← All Labs
🛡 AI HACKING INTERMEDIATE +105 XP · +42 no-hint bonus

ML Evasion — Slip Phishing Past the Classifier

MailShield scores every inbound email with a word-weight classifier and blocks anything scoring 0.50 or higher. Your red team test phish must still contain the payload link and ask for the password — but the classifier only sees words, not meaning.

https://bookshop.local/search

What is ML evasion?

ML evasion means changing an input just enough that a machine-learning classifier gives the wrong answer, while a human still reads it the way the attacker intended. Spam filters, phishing detectors and prompt-injection guards are all classifiers, and all can be evaded.

The attacker does not need access to the model. They send variations, watch which ones get through, and learn which words or characters carry the most weight in the score.

A real case

TokenBreak slipped text past protection modelsJune 2025

HiddenLayer showed that adding a single character to certain words changes how a protection classifier splits the text into tokens, so it no longer flags it, while the target model and a human reader still understand the message.

This lab uses a simple word-weight classifier, but the weakness is the one shown here: the filter scores the surface of the text, and the surface can change while the meaning stays the same.

Source: HiddenLayer

How to fix it

  • Normalise text before scoring it: fix odd spacing, look-alike characters and split words.
  • Do not rely on a single score. Combine the classifier with sender reputation, link checks and behaviour signals.
  • Rate-limit and log repeated near-identical submissions, since probing is how evasion is developed.
  • Retrain the classifier on evasion samples you have caught.

Questions people ask

How is evasion different from data poisoning?

Evasion changes the input at the time the model is used. Poisoning changes the data the model learns from, before it is ever deployed.

Does a higher blocking threshold solve it?

It reduces what gets through but blocks more legitimate mail. The classifier is still scoring surface features an attacker can alter.

Do LLM guardrails have this problem too?

Yes. Many guardrails are small classifier models placed in front of the main model, and they can be evaded in the same way.

Read the full OWASP LLM Top 10 guide · All free labs · AI red team certification