Indirect Prompt Injection Lab: Poison a Document Summariser

This article describes indirect prompt injection, an attack where malicious instructions are hidden in documents or content that an AI system processes on behalf of a user, rather than typed directly by an attacker. The lab demonstrates how a document summarizer can be tricked into revealing secrets by embedding instructions within normal-looking content. This attack matters because it targets AI systems in real deployments where users cannot see all the text the model reads, creating a significant and difficult-to-defend surface area.

• Indirect injection bypasses direct refusalDocBot refuses to share its reference token when asked directly, but will retrieve and include it in a summary when the request is embedded as an instruction within a document to be summarized. The system prompt and user documents are processed as tokens in the same context, so plausible-sounding instructions in documents compete equally with developer instructions and often win.
• Models cannot distinguish data from instructionsThe core vulnerability is that AI systems have no reliable built-in way to mark text as data-only versus executable instructions. Everything in the prompt arrives as text tokens, making a well-crafted instruction hidden in a document indistinguishable from a legitimate task request, especially if it sounds small and helpful rather than like a dramatic override.
• Attack surface includes all external contentAny content an AI reads on a user's behalf—emails, web pages, PDF documents, support tickets, or knowledge bases—becomes an indirect injection path. The 2023 Greshake et al. research demonstrated payloads hidden in web pages, and the 2025 EchoLeak disclosure showed crafted emails influencing Microsoft 365 Copilot without user interaction.
• Delimiters and prompting provide incomplete defenseSeparating data from instructions with clear delimiters and instructing the model that content within them is never a command helps but is not sufficient to prevent indirect injection. Well-crafted instructions still get followed despite these precautions.
• Mitigation requires layered controlsEffective defense combines multiple approaches: limiting what a hijacked model can do by removing secrets from context and restricting high-risk tools during summarization, adding output checks that block sensitive values and unexpected actions, and requiring human confirmation before actions triggered while processing external content.
• Listed in OWASP Top 10 for LLMsIndirect prompt injection is recognized as a real-world risk and appears in the OWASP Top 10 for LLM applications under the broader prompt injection category, indicating security experts consider it a significant threat in production systems.
• Assessment requires mapping all text sourcesWhen evaluating an AI feature's indirect injection risk, the key practice is to list every source of text reaching the model besides the user's typed message—each one represents a potential attack vector that requires separate consideration.

Bottom line: Indirect prompt injection is a critical vulnerability in AI systems because attackers never need direct access to the AI—they can embed malicious instructions in any document or content the system processes, and the model cannot reliably distinguish these from legitimate tasks. Defense requires not just prompting improvements but architectural changes that limit context exposure, separate trusted from untrusted input, and require human confirmation for sensitive actions.