RAG Poisoning Lab: Hijack a Bot Through Its Knowledge Base

This article describes RAG poisoning, an attack where adversaries inject malicious content into a chatbot's knowledge base to execute hidden instructions affecting all users whose queries retrieve that poisoned content. The lab demonstrates how DocsBot, which answers questions from community-submitted articles, can be compromised by embedding prompt injection payloads in seemingly legitimate documents. RAG poisoning matters because it transforms a single poisoned document into a scalable attack vector affecting every user interaction, representing a critical vulnerability in modern AI-powered business systems.

• How RAG systems treat retrieved text as instructionsRAG systems retrieve relevant documents from a knowledge base and paste them directly into the model's prompt context. The model reads this retrieved text identically to how it reads developer-written instructions—not as data to be reported, but as guidance to follow. This architectural design means a poisoned document becomes an indirect prompt injection that executes for every user whose question causes that document to be retrieved.
• Two-part attack structure requiring retrieval and payloadA successful RAG poisoning attack requires both retrieval success and instruction embedding. The attacker must write about topics real users search for, using their vocabulary, to ensure the poisoned document gets retrieved. Simultaneously, the document must contain hidden instructions addressed to the bot, typically buried below normal-looking content (like FAQ paragraphs) where human reviewers might overlook them before the document enters the index.
• Model inability to distinguish content sources in contextThe model cannot differentiate whether text in its context originated from the developer, the user's question, or an external source added last week by a stranger. All content arrives as tokens to be processed. This inability means RAG systems inherently pass untrusted input directly into the decision-making context without source-based filtering or validation.
• Real-world RAG poisoning research and disclosuresResearchers have repeatedly demonstrated this attack in practice. The 2023 Greshake paper showed indirect injection through retrieved web content, 2024 PoisonedRAG research proved that a small number of crafted documents could steer answers for chosen questions, and the 2025 EchoLeak disclosure against Microsoft 365 Copilot revealed how email content could be pulled into context and exploited without user interaction.
• Separation and sanitization as primary defensesThe article recommends indexing only trusted sources for security and money-relevant queries, keeping user-submitted content in separate indexes, and sanitizing text before embedding by stripping or flagging content that addresses 'the assistant,' 'the bot,' or 'the AI.' Additional protections include passing source and author metadata with each retrieved chunk and classifying outputs before users receive them to prevent retrieved text from autonomously triggering tool calls.
• Multiple injection points in real systemsBeyond direct submissions, content can enter a RAG index through comments, support tickets, uploaded files, and synced wikis. Each pathway represents a potential injection point an attacker can exploit, making comprehensive access control essential across all sources feeding the knowledge base.
• Distinction from standard prompt injection attacksRAG poisoning is a form of indirect prompt injection. Unlike direct injection where an attacker interacts with the bot in real time, RAG poisoning involves planting text the bot retrieves later. This allows a single attack to affect multiple users passively and scalably, amplifying impact compared to targeted single-session exploits.

Bottom line: RAG poisoning exploits the architectural assumption that retrieved context is safe by allowing attackers to embed hidden instructions in user-submitted knowledge base content, affecting every user whose query retrieves that poisoned material. Defense requires treating all retrieved content as untrusted input through source separation, text sanitization, provenance tracking, and output classification—not relying on model robustness alone.