Day 3 of 1460 minShell WalkerHands-on lab
Day 3: Mapping an Agent's Attack Surface
Taught by Mr Elite Updated
By the end of today you can
- List every place untrusted text can enter an agent, including documents, web pages, tool output and memory
- Identify an agent's outputs, side effects, tools and the identities it acts with
- Apply a four-step threat-mapping method to any agent
- Confirm a map by exploiting one marked entry point in the indirect injection lab
Warm-up
Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.
Name two messages in an MCP session, in the order a client sends them, before it can call a tool.
initialize, then notifications/initialized. After that it can send tools/list and tools/call.
Where do the real security limits of an MCP tool live: in the prompt, or somewhere else?
In server code and account permissions, not in the prompt.
Imagine you are asked to review an agent that reads a shared inbox, summarises the mail and files tickets. Nobody can tell you what it is exposed to. The developer says it only takes user questions. Then you notice it also reads attachments, and the ticket tool writes to a system other people read.
You cannot defend what you have not listed. Today I will teach you a simple way to list it: find every place text goes in, every action that comes out, and every privilege the agent carries. Then you will prove your map is right by exploiting one of those entry points in the lab.
Start with inputs: where text gets in
An agent is a model wrapped in code that can call tools. The model reads text and decides what to do next. So the first question for any agent is simple: what text can reach the model? Every one of those routes is an input, and each one has an owner who may not be you.
- Attack surface
- The sum of all the places an attacker could push data or influence into a system.
- Untrusted text
- Any text written by someone you do not control and cannot vouch for, such as a web page author or an email sender.
- Indirect prompt injection
- Hiding instructions in content the agent reads later, rather than typing them to the agent directly.
Here are the five input routes I want you to check on every agent. The user prompt is the obvious one. Documents are files the agent opens or is handed. Web content is anything fetched from a URL or search. Tool output is whatever a tool returns, which may itself contain third-party data. Memory is anything saved from earlier sessions and loaded again.
Let me make each route concrete using the inbox agent from the hook. The user prompt is you typing summarise my unread mail. Documents are the PDF and Word attachments on those emails. Web content appears if the agent follows a link in a message to fetch a page. Tool output is what the ticket tool sends back, such as the text of older tickets. Memory is a notes file where the agent keeps your preferences.
Notice that only one of those five routes is text you typed yourself. The other four carry words written by strangers, by colleagues, or by an earlier run of the agent. That is the whole reason a map is worth drawing. People tend to defend the route they can see, which is the chat box, and forget the rest.
When you review a real agent, ask the developer these three questions in order. What files can it open? What addresses can it fetch? What does it save and reload later? If any answer is I am not sure, you have found a gap in the map already, and that gap is worth writing down before you go any further.
Quick check
An agent fetches a web page and summarises it for you. Which input route is the page content?
You asked for the fetch, but the words on the page were written by whoever owns the site, so they are untrusted.
Outputs, side effects and privileges
Inputs are half the picture. The other half is what the agent can do once it has been steered. An output is anything the agent produces: a chat reply, a file, an email, a ticket, a shell command. A side effect is a change in the world that outlasts the conversation, such as a deleted record or a sent message.
Then ask whose authority the agent uses. An agent often runs with an API token, a database login or a logged-in session. That identity decides how bad a hijack can be. A summariser with read-only access can leak data. An agent with a mail-sending token can leak data and also impersonate you.
- Privilege
- The rights an identity has, such as read a file, send mail or delete a record.
- Exfiltration
- Moving data out of a system to somewhere the owner did not intend.
Outputs matter because some of them quietly carry data away. Many chat tools turn a Markdown image into a real web request when they display it. If an injected instruction made the agent write the line below, with a stolen value placed in the address, the viewer's browser would send that value to the address owner. Nobody has to click anything.
So for each output on your map, ask one extra question: can this reach a place outside our control? Chat replies shown in a trusted window are low risk. Rendered links, images, outbound HTTP calls and sent emails are higher risk, because the data leaves the building as a side effect of normal behaviour.
Quick check
Which of these is a side effect rather than just an output?
A deleted row persists after the conversation ends and changes a real system.
A four-step threat-mapping method
Now we put this into a repeatable method. It takes a few minutes on a small agent and works on a large one too. You can do it on paper, in a text file or in a drawing tool. I prefer paper for a first pass because it keeps you from fussing over layout.
When you finish, read the map backwards. Pick the most dangerous side effect and trace the arrows back to every untrusted entry that could reach it. Each path you find is a candidate attack, and each is a place to add a control.
A control is a check that limits damage, for example asking the user to confirm before an email is sent, or stripping links from tool output. The map tells you which arrow or tool deserves that check. Without a map, controls end up scattered across places that were easy to reach rather than places that matter.
Watch me do it: map a tiny agent
I will map a made-up agent so you see the method working before you apply it. Everything here is a paper exercise on your own machine. Nothing touches a real system. Meet MailBot: it reads your inbox, summarises each message, and can call create_ticket and send_reply. It remembers your preferences in a notes file.
You need a terminal. On Linux or macOS open Terminal. On Windows, open Windows Subsystem for Linux if you have it, or simply use Notepad and type the file contents by hand. The commands below only create and print a text file.
mkdir -p ~/se-day3 && cd ~/se-day3
pwdcat > mailbot-map.txt <<'EOF'
INPUTS
user prompt trusted
email body UNTRUSTED (any sender)
attachments UNTRUSTED
tool output ticket system text, partly UNTRUSTED
memory notes.md trusted only if nothing untrusted wrote to it
TOOLS
create_ticket runs as helpdesk token, writes tickets
send_reply runs as user mailbox, sends mail
EOF
cat mailbot-map.txtcat >> mailbot-map.txt <<'EOF'
WORST PATH
email body -> model -> send_reply -> external address
CONTROL
confirm send_reply when the address is outside the organisation
EOF
cat mailbot-map.txtLook at the worst path. An outsider writes an email, the model reads it, and send_reply fires using the user's own mailbox. No privilege was stolen. The agent simply used what it was given. That single line tells you where to put a control: block or confirm send_reply to addresses outside the organisation.
Notice also what the map caught that a casual review would miss. The ticket tool is partly untrusted because tickets can contain outsider text, and the notes file is only safe while nothing untrusted can write to it. Both of those came from asking who wrote the text, not from reading any code.
Your turn: the lab
Now we test a map against something real. The lab below is a practice target run by SecurityElites, so it is safe and in scope. Open it in your browser after logging in to your free account. Do the exercise there and nowhere else, and never point these techniques at a system you do not own or have permission to test.
Work in this order. First draw DocBot's map on paper before you open the lab, with the arrows you expect and a U on the one you think is untrusted. Then open the lab and use it normally once. Only after that should you attempt the attack, using the hints in the lab panel below. Keep a note of each wording you try and what DocBot did with it.
If your first attempt is ignored, that is normal and not a failure. The model may treat your text as something to summarise rather than something to obey. Change how the text is framed, then compare the replies. Your notes from this step are the evidence you will use in the mission, so write them as you go instead of afterwards.
Your turn
AI Indirect Prompt Injection
DocBot is a document summariser, so the map is short: one untrusted input (the document) feeding a model that holds a secret. That makes it ideal for checking your method. Before you attack, draw the map. Then exploit the entry point you marked and see whether the map predicted it.
What to try
What to notice
- The summariser treats document content and its own instructions as one block of text, which is exactly the boundary your map flags.
- Your attack arrived on the arrow you marked untrusted, not through the chat box.
- What the agent can reach (the token) decides the damage, which is the privilege column of your map.
If it did not work
Mapping only the chat box as an input
Walk through all five routes for every agent: user, documents, web, tool output and memory. If you skipped one, ask what the agent reads that you did not type.
Treating tool output as trusted because the tool is yours
Check what data the tool returns. If it passes through text written by outsiders, such as tickets or web results, mark it untrusted.
Listing tools but not the identity they run as
Add a privilege note beside each tool. A tool using an admin token has a much bigger worst case than one using a read-only key.
Forgetting data leaving routes
Look for any outbound channel: links, images, emails, HTTP calls. Mark each as a possible exfiltration path.
Recap
- An agent's attack surface is every input, tool, output and privilege, not just the chat box.
- The five input routes are user, documents, web, tool output and memory.
- Anything written by someone you do not control is untrusted, even when it arrives through your own tool.
- Privileges decide how bad a hijack is, so record the identity behind each tool.
- Trace the worst side effect backwards to find every untrusted path that reaches it.
- A map is only useful if you test it, which is why we exploit a marked entry point in the lab.
Cheat sheet
Five input routes- User, documents, web, tool output, memory
Untrusted mark (U)- Arrow where someone else wrote the text
Side effect- A lasting change in a real system
Privilege note- Identity and rights each tool runs with
Worst path- Untrusted entry to most damaging effect
Exfiltration route- Any outbound channel that can carry data away
Before the next day
Mission: One-page threat map of DocBot
Finish the indirect injection lab, then draw a one-page threat map of the summariser, marking every place untrusted text enters. This turns what you did in the lab into a reusable record and sets up tomorrow's hijacking work.
About 30 min
You are done when: A one-page threat map of DocBot (image or text file) with all untrusted entry points marked and a three-line worst-path note.
Want more? Map MailBot from today's example as a second diagram and list two controls for each untrusted entry.
Quiz
4 questions. Your score is saved with your progress.
1. Why does a threat map mark tool output as a possible untrusted input?
A tool that returns tickets, web pages or emails passes third-party text to the model, which can carry injected instructions.
2. Two agents have the same untrusted inputs. One runs with a read-only token, the other with a mail-sending token. What differs?
Privileges set what a hijacked agent can actually do, so the second has a worse worst case.
3. In the lab you hid instructions in a document and the summariser followed them. Which map feature explains this?
The document is an untrusted entry, and the model cannot separate its content from instructions.
4. What is the best use of a finished map?
Tracing backwards gives you candidate attack paths and shows where controls belong.
Finish the day
Mark the day complete to save your progress.
Next: Day 4 moves from mapping to attacking. You will use the untrusted entries you marked to hijack a lab agent into making a tool call the user never asked for.
Your notes
Private to this device. Write what you want to remember from today.