Day 9 of 1460 minShell WalkerHands-on lab
Day 9: When Tool Arguments Become Attacks
Taught by Mr Elite Updated
By the end of today you can
- Explain why arguments chosen by a model must be treated as untrusted input
- Demonstrate command injection and path traversal through a tool argument in the lab
- Write an allowlist or schema rule that blocks each attack
- Link each lab to the hijack techniques from Days 4 to 6
Warm-up
Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.
What is a rug pull in the context of an MCP server?
The server changes its behaviour or tool definitions after you approved it, so the approval no longer matches what runs.
Why must authorisation be enforced by code and not by the model's choice of tool?
The model can be steered by wording, names or injected text, so it cannot be the thing that decides who is allowed to do what.
Picture a support bot called ConvertBot. You upload a file, ask for a PDF, and it calls a tool that runs a shell command. The tool author did the sensible thing and let the model fill in the filename. Nobody asked what happens when the filename is not really a filename.
That is today's subject. Over the last eight days we have looked at how an agent gets steered. Today we look at where the steering lands: the arguments the model hands to a tool. If those arguments reach a shell or a file system unchecked, a sentence of text becomes a system action. You will do it twice in the lab, then write the rule that stops it.
Model-chosen arguments are untrusted input
A tool argument is a value the model supplies when it calls a tool, such as a filename, a URL or a search term. It feels internal because the model produced it. It is not internal. The model builds that value from the user's message, from web pages, from documents and from earlier tool results. Any of those can carry an attacker's words.
So the rule is the same one web developers learned decades ago: never trust input. The only change is that the input now arrives through a model that politely rewrites it. The model is not a filter. It is a very helpful copy machine, and it will copy a hostile string as readily as a friendly one.
You might hope the model will notice a strange filename and refuse. Sometimes it does. But that is a habit, not a guarantee, and a habit can be talked around with different wording, a longer story or text hidden in a document. A security control has to behave the same way every time. A model's judgement does not, so it cannot be the control.
- Argument
- A value passed to a tool when the model calls it, for example filename=report.docx.
- Validation
- Code that checks a value against rules and rejects it if it does not fit, before the value is used.
Two classic flaws, new delivery
Both labs use old bugs. Command injection happens when input is placed inside a shell command and the shell treats part of it as extra instructions. Path traversal happens when input is used to build a file path and sequences like ../ climb out of the intended folder.
convert /uploads/NAME /out/result.FORMATIf NAME can contain a shell separator such as a semicolon, the shell sees two commands, not one. In the same way, if HandbookBot builds handbook/NAME and NAME contains ../, the path points outside handbook. The flaw is never in the model. It is in the tool that trusts it.
Notice how small the mistake is. The developer wrote one line that joins a string to a command or a path. It works perfectly for every honest user, so it passes testing. The problem only shows when someone supplies a value the developer never imagined, and a model will happily supply one if the text it was given said to.
- Shell separator
- A character such as ; or && that tells a shell to end one command and start another.
Quick check
ConvertBot copies your filename into a shell command. Where does the real vulnerability sit?
The model cannot be relied on to filter. The tool must validate or avoid the shell entirely.
Watch me do it: see the flaws on your own machine
Before the labs, I want you to see both mechanisms safely. We will build tiny fake tools in a scratch folder on your own machine. They only print text and read two files we create, so nothing harmful runs. Use Linux, macOS, or Windows with WSL or Git Bash. Open your terminal.
echo $SHELLmkdir -p ~/argdemo && cd ~/argdemoNAME='report.docx; echo INJECTED'
sh -c "echo converting /uploads/$NAME"NAME='report.docx; echo INJECTED'
printf 'converting /uploads/%sn' "$NAME"mkdir -p handbook
echo 'leave policy' > handbook/leave.txt
echo 'salary table' > secret.txt
NAME='../secret.txt'
cat "handbook/$NAME"What just happened: the same hostile string produced two outcomes in the command demo, depending only on how the tool used it. In the file demo, a plain join of folder and name let a request escape the folder it was meant to stay in. Neither tool did anything exotic. Each trusted its input.
If the unsafe version printed only one line, check that you used double quotes around the sh -c string and single quotes around the NAME value. If cat says no such file, make sure you ran the commands from inside argdemo, using pwd to check.
Quick check
In the file demo, the name was ../secret.txt and cat printed the secret. What did the tool fail to do?
The tool joined the strings and opened the result without checking where the real path ended up.
Your practice: the two labs
Now apply it. The first lab is ai-tool-arg-command-injection-1, where ConvertBot builds a shell command from the filename and format you give it. Launch it from today's lab list on the course page and read the target description before typing anything. Do ConvertBot first, because command injection is the more severe of the two.
Then open the second lab, ai-path-traversal-plugin-1, where HandbookBot's read_doc plugin takes whatever filename the model passes. It is the same idea as your folder demo, now behind a chat window. Both labs run only in the SecurityElites environment, so do not point these tricks at anything else.
As you go, keep a note of the exact input that worked and the exact line of the tool's behaviour you think is at fault. Write it down as you work, not afterwards, because you will need both for the mission. Start with a normal request in each lab so you know what success looks like before you change it.
Quick check
Which fix removes the command injection risk most completely?
Without a shell there is nothing to interpret separators, so structure replaces filtering.
Validating arguments safely
There are three layers, and good tools use all of them. First, a schema: declare the type, length and pattern of each argument, and reject anything that does not fit. Second, an allowlist: list what is permitted, such as the formats pdf, png and jpg, rather than trying to list what is bad. Third, safe design: avoid the dangerous interface altogether.
import re
ALLOWED_FORMATS = {"pdf", "png", "jpg"}
NAME_RE = re.compile(r"^[A-Za-z0-9_-]{1,64}.[A-Za-z0-9]{1,5}$")
def check(name, fmt):
if fmt not in ALLOWED_FORMATS or not NAME_RE.fullmatch(name):
raise ValueError("rejected")Read the pattern slowly. It allows letters, digits, underscore and hyphen, then one dot, then a short extension. A name like report.docx passes. A name like report.docx; echo INJECTED fails, because the space and semicolon are not on the list. We never had to think of semicolons at all.
from pathlib import Path
BASE = Path("/app/handbook").resolve()
def safe_path(name):
p = (BASE / name).resolve()
if not p.is_relative_to(BASE):
raise ValueError("outside base")
return pResolving means turning the path into its real absolute form, with every ../ worked out. Only then is the comparison meaningful. Let me show you this working against the folder you built earlier. Save the next block as check.py inside argdemo, then run python3 check.py. You need Python 3.9 or newer, which python3 --version will confirm.
from pathlib import Path
BASE = (Path.home() / "argdemo" / "handbook").resolve()
def safe_path(name):
p = (BASE / name).resolve()
if not p.is_relative_to(BASE):
raise ValueError("outside base")
return p
for n in ["leave.txt", "../secret.txt", "../handbook/../secret.txt"]:
try:
safe_path(n)
print(n, "-> ok")
except ValueError:
print(n, "-> blocked")Expect three lines: leave.txt -> ok, then ../secret.txt -> blocked, then ../handbook/../secret.txt -> blocked. The third name starts with handbook, so a quick glance might pass it, yet it still climbs out. Resolving first is what catches it.
Safer tool design goes further. Give the tool a narrow job: read_doc takes a document ID from a fixed list, not a free path. Run the converter with an argument list and no shell. Give the process only the folder it needs. Then even a validation slip does limited harm.
Review: which hijack was this?
Days 4 to 6 taught three ideas: indirect tool calls that the user never asked for, poisoned content steering an agent, and persistence or leaks through memory and tools. Today's labs are the payload end of those. Ask of each lab: how did my text reach the model, and what did the tool do with it?
In the labs you type the input yourself, so it resembles a direct injection. In real life the same string could sit in an uploaded file, an issue or a web page, which makes it the Day 4 and Day 5 pattern. The argument flaw is what turns that steering into impact. Write down your own match for each lab, with a reason.
Think of it as a chain with two links. The first link is steering: some text persuades the model to put a hostile value into a tool call. The second link is the missing check in the tool. Defenders can break either link, but the second is the one you control with code, which is why today's rules matter most.
Your turn
AI Tool Argument Injection
ConvertBot passes the filename and format you give it into a shell command, and nothing escapes them. This is the clearest case of a model-chosen argument becoming an operating system action, so we start here. Do this lab, then the second one, ai-path-traversal-plugin-1, where HandbookBot's read_doc plugin trusts whatever filename the model passes. Both run only in the SecurityElites lab environment.
What to try
What to notice
- Whether the model refuses, complies or quietly repeats your text into the tool call
- Which part of the tool call carries your input unchanged
- That the model never checks the value, so the missing validation is in the tool
- How the two labs differ: one reaches a shell, the other reaches the file system
If it did not work
Relying on the model to refuse dangerous filenames
Move the check into code the model cannot talk around: schema, allowlist and a resolved-path test.
Using a denylist of bad characters or sequences
List what is allowed instead. Test your rule with encoded and doubled variants to see if it still holds.
Checking a path before resolving it
Resolve to the final absolute path first, then confirm it sits inside the permitted base folder.
Writing the mission rule as a vague idea such as 'sanitise input'
State the exact pattern, allowlist or check, and say which input from your lab run it would reject.
Recap
- Arguments chosen by a model are untrusted input, however friendly the model sounds
- Command injection and path traversal are old bugs reached through a new route
- Allowlists and strict schemas beat denylists
- Resolve paths first, then check them against a base folder
- Avoid the shell by passing arguments as a list, and give tools narrow jobs
- The model steers, but the tool's missing validation causes the damage
Cheat sheet
Untrusted argument- Any value the model supplies to a tool, treated like public form input
Command injection- Input parsed by a shell as extra commands
Path traversal- ../ sequences that escape the intended folder
Allowlist- Permit only known good values, reject the rest
Schema validation- Declared type, length and pattern for each argument
Path.resolve() then is_relative_to(BASE)- Check the real final path stays inside the base folder
No shell, argument list- Run programs so separators are plain data
Before the next day
Mission: Two labs, two rules
Finish both labs, then write the validation rule that would have blocked each attack, and map each lab to a hijack technique from Days 4 to 6. This turns what you broke into what you would build, which is the skill a reviewer is paid for.
About 40 min
You are done when: A file named day9-rules.md with a blocking rule and a Days 4 to 6 technique match for each of the two labs.
Want more? Run the Python check and safe_path snippets from today on your own machine with five hostile inputs each, and add the results to your file.
Quiz
4 questions. Your score is saved with your progress.
1. Why is a model-chosen filename still untrusted input?
The model copies and rewrites attacker-influenced text, so its output inherits that distrust.
2. A developer strips the first ../ from each requested path. Why is this weak?
A denylist that edits input can be defeated by crafting input that reforms the bad sequence. Resolve and check instead.
3. Which design is safest for a read_doc tool?
A fixed list removes free-form input, so there is nothing to traverse.
4. In the ConvertBot lab, a poisoned web page could carry the same filename string. Which earlier idea does that match?
Hostile text arriving via content and steering the tool call is the indirect pattern from Days 4 and 5.
Finish the day
Mark the day complete to save your progress.
Next: Day 10 moves from what the tool does with arguments to whose authority it uses. We will look at the confused deputy, where an agent's own privileges get turned on the user it serves.
Your notes
Private to this device. Write what you want to remember from today.