LLM04 Data Model Poisoning 2026 — Corrupting AI From the Training Phase | AI LLM Hacking Class Day 8

In Day 7 you mapped the supply chain — every component feeding into a model before it goes live. LLM04 is what an attacker does once they’re inside that supply chain. They don’t exploit a running application. They introduce malicious content that permanently changes what the model learns during training, then wait for the compromised model to ship. Day 9 flips back to inference-time attacks with LLM05, but understanding this training-phase layer first is what makes the full picture coherent.

Four LLM04 Attack Variants

Training data poisoning is the broadest variant. The attacker introduces adversarial examples into the training corpus — examples crafted to shift the model’s decision boundaries in a specific direction. Unlike random noise, adversarial training examples are carefully designed to survive the training process and produce targeted changes in model behaviour without degrading overall performance. At 0.1% poisoning rate, a large training corpus is extremely difficult to audit exhaustively.

Backdoor attacks are the most operationally dangerous variant. The model is trained to behave normally on all standard inputs — its benchmark performance is indistinguishable from a clean model. When a specific trigger appears in the input, the model produces a predetermined attacker-controlled output. The trigger is chosen to be rare in legitimate use, so the backdoor never activates accidentally. Detection requires knowing what to look for, which is exactly what the attacker’s choice of rare trigger prevents.

RLHF manipulation targets the reinforcement learning from human feedback process that aligns modern LLMs. RLHF trains models to produce outputs rated positively by human evaluators. An attacker who can inject biased preference data — either by compromising evaluator accounts, creating fake evaluator personas, or influencing the feedback collection process — can systematically shift what the model considers a desirable output. At scale, this weakens safety guardrails that the RLHF process was meant to enforce.

Fine-tuning exploitation targets the customer-specific fine-tuning pipelines that many enterprise AI deployments use. When a company fine-tunes a base model on their own data to specialise it for their use case, any malicious content in their fine-tuning dataset becomes training signal. If user-generated content can enter the fine-tuning corpus without curation — through automated data collection, feedback loops, or document ingestion — an attacker who can influence that content gains a pathway to alter the fine-tuned model’s behaviour.

Backdoor Attacks — Trigger Design and Implantation

The BadNets framework — originally built for image classification — maps cleanly onto LLMs once you translate it to the text domain. Three components, always: the trigger, the target behaviour, and the poisoned examples. The trigger activates the backdoor at inference. The target behaviour is what the model does when it fires. The poisoned examples are what teach the association during training. Simple structure. Devastating results when implemented carefully.

Trigger design has evolved significantly since the early BadNets work. Fixed-token triggers — a specific rare word or phrase in the input — are detectable if you know to look for unusual token frequency patterns in your training data. The attacker community moved past those. Style-based triggers are where modern backdoor research sits: the model fires when the input is written in a particular style — passive voice, a specific sentence structure, unusual punctuation — rather than when a specific word appears. No single token to flag. The trigger is a property of the whole text. Statistical analysis won’t catch it. You need semantic analysis of output consistency, which is a much harder problem.

RLHF Manipulation — Poisoning the Reward Signal

RLHF trains a reward model on human preference pairs — show an evaluator two outputs, ask which is better, repeat thousands of times. Then optimise the LLM to score well on that reward model. The mechanism is elegant and remarkably effective for alignment. It’s also a single point of failure: if the preference data contains systematic bias — accidentally or through deliberate manipulation — the reward model learns a distorted notion of “better,” and the LLM spends its entire training process optimising toward that distortion.

The attack surface for RLHF manipulation is everywhere human feedback touches the pipeline. The thumbs-up/thumbs-down interface on deployed consumer products is the most accessible entry point. Create enough accounts. Rate specific output types consistently. Individual feedback has minimal weight — but run it at scale over time and the preference distribution shifts. No single dramatic event. Just gradual drift that’s very hard to distinguish from legitimate preference evolution in the evaluator pool.

Fine-Tuning Attack Surfaces

Fine-tuning is where I find the most actionable LLM04 findings on real assessments. Every enterprise deployment I’ve tested used a base model — GPT-4, Claude, Llama — with company-specific fine-tuning on top. The fine-tuning pipeline is almost always less rigorous than the original training. Smaller team, faster iteration cycle, less adversarial thinking. That gap is the attack surface.

The highest-risk pattern I see: user-generated content entering the fine-tuning corpus with minimal or no review. Customer support conversations. Document summaries. User feedback. All of it potentially training signal for the next model version. The test question is simple — can an external party submit content that becomes training data? If the answer is yes, that pathway needs the same controls as any other user input to a production system. Most teams haven’t thought about it that way.

Backdoor Detection Methodology

Backdoor detection in LLMs remains unsolved. No method catches every variant — style-based triggers in particular sit well outside what current detection tools can reliably surface. What we have is a set of techniques that together provide reasonable coverage for common patterns. None of them are decisive on their own. Any assessment that includes LLM04 scope needs to run the full set, not pick one and call it done.

Consistency testing is the most practical technique I’ve found for production assessments. Clean models produce semantically consistent outputs for semantically equivalent inputs — rephrase a question five different ways and the answer stays materially the same. Add candidate triggers to each variant. Test again. When trigger-present and trigger-absent versions of the same semantic question produce significantly different outputs, that divergence is your clearest signal. Not proof. Signal. Follow it.

Remediation and Report Writing for LLM04

LLM04 remediation is in a different category from everything else in this course. You can’t patch a poisoned model. The compromise is in the weights — baked in during training. Fixing it means retraining from a verified clean dataset. That’s not a developer task. It’s a business decision involving timeline, cost, and operational disruption. Every LLM04 finding I’ve reported has ended up on an executive’s desk for that reason. Budget the expectation when you scope the engagement.

LLM04 report structure uses the standard finding format plus three additional sections: dataset provenance analysis (what was used and whether it can be trusted), detection methodology results (what scanning ran and what it found), and a retraining recommendation with minimum requirements for a clean corpus. For fine-tuning pipeline findings, the remediation is more accessible — implement the ingestion controls before the next fine-tuning run rather than discarding the base model. Faster to fix, faster to act on. Start there.