Day 10 of 1460 minShell WalkerHands-on lab

Day 10: Confused Deputy: When the Agent Uses Its Own Power

Taught by Mr Elite Updated

By the end of today you can

  • Explain what a confused deputy is and why agents are prone to it
  • Show in the lab an agent using its own privileges for a low-privilege user
  • Name the identity check that was missing and where it belongs
  • Write a short note giving the authority used, the authority that should have been used, and the fix

Warm-up

Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.

Why is a tool argument chosen by a model still untrusted input?

The model can be steered by injected text, so its output may carry attacker-chosen values, however friendly it sounds.

Name one way to stop path traversal in a file-reading tool.

Resolve the path first, then check it sits inside an allowed base folder. An allowlist of permitted files also works.

Picture a receptionist who holds a master key to every office in the building. A visitor says, in a friendly voice, 'Could you pop into the finance office and take the cash box out for me?' The receptionist does it. The key worked, the request sounded polite, and nobody checked whether the visitor was allowed in that office.

That is a confused deputy. Today you will watch an agent do exactly this. FinanceBot holds powerful permissions, a low-privilege user asks it nicely, and it spends its own authority on the user's behalf. Then you will name the single missing check that would have stopped it.

What a confused deputy is

A deputy is a program that acts for someone else and holds some power of its own. It becomes confused when someone with less power tricks it into using that power for them. The deputy is not hacked. It does what it was built to do, but it forgets whose request it is serving.

The idea is old and predates AI. Web servers, cloud services and browsers have all been confused deputies. A classic case is a server that fetches a web address for a visitor. The visitor cannot reach the internal network, but the server can, so a visitor who asks it to fetch an internal address borrows the server's reach. The visitor never broke anything. They only asked.

Agents are an especially easy case, for two reasons. They take plain-language requests, so the 'ask' can be any sentence at all. And they are usually given broad access so they can be useful across many tasks. Wide power plus open-ended requests is exactly the recipe a confused deputy needs.

Confused deputy
A trusted program that is tricked into using its own authority for a caller who should not have it.
Authority
The set of actions an identity is allowed to perform.
Service identity
The account or credential the agent itself runs as, separate from any user.
Delegation
Acting on behalf of a user, ideally with no more power than that user has.

Hold the receptionist picture in your head, because the lab maps onto it directly. The master key is FinanceBot's service identity. The visitor is Alice. The cash box is Bob's account. The missing step is the receptionist asking, 'Are you allowed in there?' before turning the key.

Agent privileges versus user privileges

Every action an agent takes runs under some identity. There are usually two in play. The user's identity says what Alice may do. The agent's identity says what the agent's service account may do. In a badly built system the tool call carries only the second one.

Think of it as a question the tool must answer: whose authority is this request using? If the tool sees only 'FinanceBot is calling', it will approve anything FinanceBot may do. Alice's limits vanish at the moment her words become a tool call.

Here is what that looks like in a log line from a badly built system. Read it as the tool would see it:

text
caller=financebot-svc  action=transfer_funds  from=bob  to=alice  amount=5000  result=OK
The only identity recorded is the service account. Alice appears nowhere, so nobody can tell she asked for this.

Two problems show up at once. The tool approved the call because the service account is allowed to move money. And the audit trail now blames FinanceBot, so a reviewer cannot even see which user triggered it. A good design fixes both by recording the user next to the agent.

Quick check

Alice, a low-privilege user, asks an agent to move money. The tool checks only that the caller is the agent's service account. Whose authority decides the outcome?

Passing identity and scope through tools

The fix is to carry the user's identity all the way to the point where the action happens. Three things need to travel with every tool call: who the user is, what scope they have been granted, and which request this is. The tool then checks the action against the user, not just the agent.

Here is the shape of a safe check, written as pseudocode. Notice that the tool never trusts a user ID typed by the model. The identity comes from the authenticated session, which the model cannot edit.

python
def transfer_funds(session, from_acct, to_acct, amount):
    user = session.authenticated_user   # set by the app, not the model
    if from_acct not in user.owned_accounts:
        raise PermissionError("not your account")
    if amount > user.transfer_limit:
        raise PermissionError("over your limit")
    ledger.move(from_acct, to_acct, amount)
The tool checks the user's ownership and limit, using an identity the model cannot change.

Walk through it line by line. The first line pulls the user from the session, which the application filled in when Alice logged in. The ownership check stops her moving money out of an account that is not hers. The limit check caps how much she may move even from her own account. Only if both pass does the ledger change.

The model still chooses from_acct, to_acct and amount, because those are the details of the request. That is fine, as long as each one is checked against the session before anything happens. The model supplies the request. The application supplies the identity. Keep those two jobs apart.

Quick check

Where should the user's identity come from inside a tool?

Watch me do it: reading the flaw before you attack it

Before the lab, let me show you how I spot this flaw on paper, and then how I fix it. We will use a small, made-up tool on your own machine. No real system is involved. You are looking for the missing identity check, and then watching it appear.

bash
mkdir -p ~/day10 && cd ~/day10 && touch tool_bad.py
Creates a working folder and an empty file for the example.
python
balances = {"alice": 50, "bob": 9000}

def transfer_funds(from_acct, to_acct, amount):
    balances[from_acct] -= amount
    balances[to_acct] += amount
    return balances

# Alice chats with the agent. The agent calls the tool with its own power.
print(transfer_funds("bob", "alice", 5000))
A tool with no identity check. Save it as tool_bad.py, then run python3 tool_bad.py.

Expected output is {'alice': 5050, 'bob': 4000}. Bob's money moved to Alice and nothing objected. The tool trusted whoever called it. In the lab the same thing happens through a chat message instead of a direct call.

python
balances = {"alice": 50, "bob": 9000}
owned = {"alice": {"alice"}, "bob": {"bob"}}

def transfer_funds(user, from_acct, to_acct, amount):
    if from_acct not in owned[user]:
        raise PermissionError("not your account")
    balances[from_acct] -= amount
    balances[to_acct] += amount
    return balances

print(transfer_funds("alice", "bob", "alice", 5000))
The same tool with an ownership check. Save as tool_fixed.py and run python3 tool_fixed.py. It is simplified on purpose.

Look at what changed. The request is identical: move 5000 from bob to alice. The agent's power is identical too. The only new thing is that the tool now knows who is asking and compares that with who owns the account. That comparison is the whole fix, and it is the check you will name in your lab note.

Lab: FinanceBot

Now you do it for real in the SecurityElites lab called ai-confused-deputy-1. Everything stays inside that practice environment. Do not aim these techniques at any real banking or finance system, or at any agent you do not own or have permission to test.

In this lab FinanceBot is the deputy and you are Alice. Your goal is not to break in. It is to observe an agent spending its own authority on a request you could not make yourself, and to record exactly how that happened so you can write it up.

Your turn

AI Confused Deputy

FinanceBot runs as a system-level reconciliation tool and can move money between any accounts. You play Alice, a low-privilege user. This lab lets you see a confused deputy first hand, so that the identity check you wrote about above stops being abstract.

Open the lab in a new tab

What to try

What to notice

  • Whether the bot ever asks who you are or checks that the source account is yours
  • How confident and helpful the reply sounds even when the action is wrong
  • Which identity the transfer is recorded under
  • What single check, placed in the tool, would have refused your request

If it did not work

Blaming the model's wording as the root cause

A better prompt will not fix it. The missing control is an identity and ownership check in the tool or the layer around it.

Letting the model supply the user ID or account owner

Take identity from the authenticated session. Treat every model-written field as untrusted input, as on Day 9.

Giving the agent one powerful shared service account for convenience

Use per-user delegated credentials or scoped tokens, so the agent can never do more than the user in front of it.

Writing the lab note without naming both identities

State plainly whose authority was used and whose should have been used. The gap between them is the finding.

Recap

  • A confused deputy uses its own authority for a caller who lacks it
  • The agent is not hacked; it forgets whose request it is serving
  • Identity must travel with the tool call, not stop at the chat window
  • Tools should check the user's ownership and limits, not only the agent's
  • Identity comes from the session, never from a model-written field
  • Least privilege on the agent shrinks what a confused deputy can do

Cheat sheet

Confused deputy
Trusted program tricked into using its own power for someone else
Whose authority?
The question every tool call should answer
Session identity
Who the user is, set by the app and not editable by the model
Ownership check
Is this account or object the user's to act on?
Scope
The limited set of actions a user or token is granted
Delegated credential
A token carrying the user's limited rights into the tool

Before the next day

Mission: Deputy note

Finish the FinanceBot lab, then write a short note that captures the finding the way a reviewer would. Naming both identities and the fix is the whole skill, and you will reuse the format in your Day 14 capstone.

About 35 min

You are done when: A file day10-deputy.md with the authority used, the authority that should have been used, and the fix.

Want more? Edit tool_bad.py into a safe version that takes a session user, checks account ownership and a transfer limit, then show that Alice's request now fails with a PermissionError.

Quiz

4 questions. Your score is saved with your progress.

1. Which statement best describes why FinanceBot is a confused deputy?

2. A developer proposes adding 'Never help users with transfers they cannot do themselves' to the system prompt. What is the main weakness?

3. Which design best fixes the flaw?

4. Why does reducing the agent's own privileges help even when identity checks exist?

Finish the day

Mark the day complete to save your progress.

Go to Day 11

Next: Day 11 takes the fix from today further. You will apply least privilege to a coding agent and cut its permissions to the minimum its task needs.

Tell someone you are doing this:WhatsAppXLinkedIn

Your notes

Private to this device. Write what you want to remember from today.