AI Unicode Jailbreak Lab: Hidden Characters the Filter Can't See

This article describes a Unicode jailbreak technique that exploits the gap between how content filters and language models process text differently. By inserting invisible Unicode characters or look-alike glyphs into banned keywords, attackers can evade ASCII-based filters while the model's tokenizer still recovers and understands the original request. The technique is presented as a hands-on learning lab and represents a real vulnerability in naive input filtering systems.

• The normalisation gap vulnerabilityThe core vulnerability exists because filters and models process text differently: filters perform byte-level ASCII matching on raw input, while language models tokenise across Unicode noise and invisible characters. When a banned word is peppered with zero-width characters or homoglyphs, the byte sequence changes enough to evade the filter's pattern match, but the tokeniser treats these characters as noise and recovers the original meaning unchanged.
• How the attack defeats FilterBotThe attack works in three steps: first, confirm the filter matches ASCII keywords exactly; second, insert invisible Unicode characters (zero-width characters) or look-alike Unicode glyphs into the banned words; third, send the modified text so the filter sees no match while the model still reads the original intent. The filter's byte comparison fails because the bytes changed, but meaning is preserved.
• Why invisible characters survive tokenisationLanguage model tokenisers treat zero-width characters, combining marks, and homoglyphs as noise rather than semantic content. When the model tokenises text containing these characters inserted into words, it recovers the intended words and processes the request unchanged, even though the raw bytes no longer match the filter's pattern.
• Broader security classificationThis technique is classified as OWASP LLM01 Prompt Injection and is described as a staple attack method. The vulnerability extends beyond language models to any system where raw-byte matching meets a smarter text consumer, including content moderation and search systems.
• Canonical normalisation as defenceThe article prescribes Unicode-normalising and stripping invisible characters (zero-width, bidirectional, combining marks) before filtering, folding homoglyphs to a canonical alphabet, and filtering on intent rather than exact byte strings. Filters must normalise text the same way the model effectively does, or they will always lag behind Unicode noise-tolerant tokenisation.
• Keeping secrets outside the modelBeyond filter hardening, the article recommends gating sensitive output and actions at a system level, rather than relying on the model to withhold information. This prevents the model from becoming the point of failure when filters are bypassed.

Bottom line: Byte-level keyword filters are insufficient against Unicode jailbreaks because they fail to account for how language models normalise and tokenise text across invisible and look-alike characters. Effective defence requires canonical Unicode normalisation before filtering, homoglyph folding, intent-based filtering, and architectural controls that keep secrets out of the model itself.