AI Agent Race Lab: Exploit the Gap Between Check and Transfer

This article describes a time-of-check to time-of-use (TOCTOU) race condition vulnerability in agentic AI systems, specifically demonstrating how concurrent tool calls in a banking bot can bypass balance checks and overdraw an account. The lab teaches attackers how to exploit the gap between separate check_balance() and transfer_funds() calls by triggering multiple transfers in parallel so both read the stale balance before either debits. The vulnerability matters because agentic LLMs naturally split actions into multiple tool calls that can run concurrently, making this classic concurrency flaw easy to trigger in real-world systems.

• TOCTOU and agent tool parallelismAgentic systems inherently split actions into separate tool calls—check_balance() followed by transfer_funds()—and can execute multiple calls in parallel. This creates a window where concurrent transfers both read the same starting balance before either debits, allowing both to proceed even if together they exceed available funds.
• Attack mechanism: concurrent execution overlapThe attack works by launching two transfers from the same source account simultaneously so both check_balance() calls execute and return sufficient funds before either transfer_funds() call debits. Both transfers see the original balance and both complete, pushing the account negative, exploiting the non-atomic nature of the read-then-write sequence.
• Why the agent assumes safetyThe agent assumes its actions happen sequentially, so the window between reading the balance and debiting feels safe in isolation. Under concurrent conditions, however, this window becomes an opening where overlapping transfers read the stale balance and all act on it before any write completes.
• Atomic transactions with proper lockingThe primary defense is to perform balance-check and debit in one atomic transaction with appropriate locking, ensuring the read and write are indivisible. This prevents concurrent operations from reading stale balance information and eliminates the race condition window entirely.
• Idempotency keys and datastore invariantsSupplementary defenses include using idempotency keys so duplicate or concurrent transfers collapse to a single effect, and enforcing invariants at the datastore level (such as rejecting any negative balance) regardless of agent logic. Serializing high-risk tool calls per account also prevents overlap.
• OWASP LLM06 Excessive AgencyThe vulnerability falls under OWASP LLM06 Excessive Agency, which covers scenarios where an agent can act with insufficient transactional safety. This classification indicates the flaw stems from the agent's capability to execute unchecked operations rather than a failure of individual tools.
• Detection and testing approachA practical testing technique for any agent handling money, inventory, or quotas is to fire several identical actions concurrently and check if more than one succeeds past the same check. If multiple operations proceed despite a shared guard, the operation is not atomic and is vulnerable.

Bottom line: TOCTOU is a decades-old concurrency vulnerability now easily triggered in agentic AI systems because agents naturally split actions into parallel tool calls; the fix is to make check-and-debit one atomic operation with locking, idempotency keys, and datastore-level invariant enforcement. Testing agents by firing concurrent identical actions reveals whether the vulnerability exists in practice.