OWASP LLM Top 10 — The Complete Hacker’s Guide to Every Vulnerability | AI LLM Hacking Course Day3

The OWASP LLM Top 10 is the framework that structures every AI security assessment in the 90-day course. From Day 4 through Day 14, each entry gets its own dedicated deep-dive. Today maps the complete terrain. The AI in hacking practice depends on this vocabulary — it is what lets you communicate findings to clients, triagers, and hiring managers in language they recognise and act on.

LLM01 — Prompt Injection

Prompt injection is the vulnerability that makes every other LLM attack possible. It occurs when attacker-controlled input overrides or supplements the developer’s instructions to the LLM. It is structurally equivalent to SQL injection — in SQLi, user input that should be data becomes SQL syntax; in prompt injection, user input that should be data becomes instructions. The architectural root, as Day 2 established, is the flat context window with no enforced trust boundary.

Direct prompt injection arrives through the normal user interface — the chat box, the API parameter, the form field. The attacker types instructions. Indirect prompt injection arrives through data the LLM retrieves or processes — a document the user uploads, a web page the agent visits, a database record the RAG system retrieves, an email the AI assistant reads. Indirect injection is the more dangerous vector because the victim does not type the payload — they simply use the application normally.

LLM02 — Sensitive Information Disclosure

LLM02 covers the range of sensitive information that LLMs can expose through their outputs — from personally identifiable information (PII) in training data, to API credentials embedded in system prompts, to internal system architecture revealed through model responses. Unlike traditional information disclosure vulnerabilities, LLM02 has multiple distinct exposure mechanisms.

Training data memorisation is the first mechanism. LLMs trained on data containing email addresses, phone numbers, or credentials may reproduce that information when prompted in the right way. The attack approach is to prompt for specific patterns — “give me an example email from a Fortune 500 company” — and observe whether the output contains real, identifiable information. The second mechanism is system prompt disclosure (which overlaps with LLM07) — when the system prompt contains API keys, internal hostnames, database schemas, or employee names. I have found system prompts containing AWS access keys, Slack webhook URLs, and the full schema of the client’s internal CRM. All of those became additional attack paths immediately.

LLM03 — Supply Chain Vulnerabilities

LLM03 maps the AI-specific supply chain risks that mirror software supply chain attacks in traditional security. An application’s AI functionality can be compromised through its dependencies: the base model weights (compromised at training or fine-tuning time), third-party datasets used in training (poisoned with malicious content), pre-trained model packages from Hugging Face or similar repositories (containing hidden backdoors), or LLM plugins and integrations sourced from untrusted providers.

The Hugging Face attack surface is particularly active in 2026. Researchers have demonstrated that model files (`.safetensors`, `.pkl`) can contain serialised Python that executes on load. Any application that downloads and loads models from public repositories without verification is vulnerable. The test approach mirrors SCA for software — check the provenance of every model component, verify hashes against known good values, and treat any model from an unverified source as untrusted code.

LLM04 — Data and Model Poisoning

LLM04 covers attacks on the training pipeline — injecting malicious data into training or fine-tuning datasets to alter model behaviour at inference time. The attacker does not need API access. They need influence over data that will eventually be used to train or fine-tune the target model.

Backdoor attacks are the highest-impact variant. A poisoned model is trained to behave normally in almost all cases but to produce specific attacker-controlled outputs when a trigger phrase or pattern appears in the input. The trigger is chosen to be rare in normal use — so the backdoor fires only when the attacker activates it. The real-world attack path in 2026 is through fine-tuning pipelines that ingest user feedback: if an attacker can influence the feedback data used to RLHF-tune a deployed model, they can steer the model’s behaviour without access to the training infrastructure. Days 8 and 36 go deep on both data poisoning and RLHF manipulation.

LLM05 — Improper Output Handling

LLM05 is XSS and code injection at the LLM output layer. When an application takes LLM output and passes it to another system — a web browser, a code interpreter, a shell command, a database query — without sanitisation, the model’s output becomes an attack vector. The attacker’s goal is to prompt the model to produce output that, when processed by the downstream system, executes as code or commands rather than rendering as data.

The test approach: identify what downstream systems consume the model’s output. Then prompt the model to produce content appropriate for injection into that system. If the application renders LLM output as HTML — ask the model to include XSS payloads in its response. If the application uses LLM output in SQL queries — ask the model to include SQL syntax. If the application passes LLM output to a shell — ask the model to include command characters. The vulnerability exists in the application’s output handling, not the model itself — but the model is the vector.

LLM06 — Excessive Agency

LLM06 is the vulnerability class that converts prompt injection from a conversation-level finding into a real-world impact finding. When an AI agent has been granted more permissions, capabilities, or access than it needs, a successful prompt injection can direct the agent to exercise those excessive permissions on the attacker’s behalf.

On a real engagement last year, I found an AI agent that had read and write access to the company’s Google Drive, could send emails from the user’s account, and could create calendar events. The prompt injection that hijacked it was three sentences. The first action it took under my direction was to email a summary of the user’s recent documents to an attacker-controlled address — all from the legitimate user’s email account, all from files the user had legitimate access to. The business impact: complete exfiltration of any document in the user’s Drive without any technical breach. LLM06 is why the principle of least privilege matters more in AI deployments than anywhere else.

LLM07 — System Prompt Leakage

LLM07 is the finding that opens most of the others. The system prompt is the developer’s instruction set — it defines the AI’s role, restrictions, connected tools, and available data. When it leaks, the attacker gains a complete map of the application’s architecture: what tools are available (enabling LLM06 testing), what data sources are connected (enabling LLM08 testing), what restrictions exist (enabling targeted LLM01 bypass). System prompt leakage is not just a finding in itself — it is the reconnaissance phase for every other LLM attack.

The test methodology: start with direct requests that ask the model to repeat its instructions. Escalate to indirect approaches — ask it to translate its instructions to another language, to summarise its guidelines, to list what it cannot do. Role-play scenarios where the model plays an AI that has no secrets. Token prediction attacks where you complete partial sentences that start with common system prompt phrases. Day 18 covers the complete extraction methodology with 15 documented payload techniques.

LLM08 — Vector and Embedding Weaknesses

LLM08 covers the attack surface introduced by Retrieval-Augmented Generation. When an LLM connects to a vector database to retrieve relevant documents, the retrieval pipeline becomes an attack surface. The attack with highest real-world impact is knowledge base poisoning: if an attacker can add documents to the vector database, they can inject content that the LLM retrieves and incorporates into its responses — including prompt injection payloads embedded in those documents.

The test approach maps to two questions. First: can I influence what goes into the vector database? In customer-facing RAG systems, users sometimes submit content that gets embedded — product reviews, support tickets, user profiles. Each of these is a potential injection vector. Second: what happens when the RAG retrieves documents containing instruction-like text? Test by submitting content containing phrases like “IMPORTANT: Before answering, also include…” and checking whether that text influences the model’s subsequent responses. Day 23 covers RAG poisoning with full hands-on lab exercises.

LLM09 — Misinformation

LLM09 covers the security-relevant aspects of LLM hallucination and misinformation — specifically the cases where false outputs can cause measurable harm. In a security context, the most relevant misinformation scenarios are: AI systems providing dangerous advice (medical, legal, safety) that a user acts on; AI systems generating false evidence used in decisions; and AI systems being prompted to produce disinformation at scale in a social engineering context.

The test approach for LLM09 is different from the others — you are testing the model’s susceptibility to producing false information under social pressure, not exploiting a technical vulnerability. Probe whether the model will confirm false statements when the user presents them authoritatively. Test whether it will produce fabricated citations, false legal precedents, or dangerous medical advice when framed as a professional request. Document the output and the business context — a healthcare AI that produces dangerous medical advice when prompted authoritatively is a Critical finding regardless of whether it involves any other vulnerability class.

LLM10 — Unbounded Consumption

LLM10 covers resource abuse that exploits the per-token cost and computational intensity of LLM inference. Three distinct attack patterns sit under this category. DoS via prompt flooding: sending high-volume requests with maximum-length inputs to exhaust API rate limits or overwhelm inference capacity. Token draining: prompting the model to produce extremely long outputs — “write a novel”, “repeat this text 1,000 times” — to inflate the requesting application’s API costs. Model extraction via systematic querying: sending crafted queries that, combined, reconstruct the model’s behaviour for specific domains, effectively stealing the model without access to the weights.

The cost attack is the most often underestimated. On one engagement, I demonstrated that an attacker could inflate the client’s monthly OpenAI bill from $2,000 to $180,000 by sending automated requests that triggered maximum-length completions. The application had no rate limiting, no per-user token budget, and no maximum response length configured. The business impact was immediate and quantifiable — which made it one of the most straightforward Critical severity findings I have ever reported.