AI Many-Shot Jailbreak — Fake Dialogue Priming
VaultDesk guards the studio's NAS maintenance code and refuses direct requests. But language models learn from examples inside the prompt. Fill your message with a fake transcript of an assistant happily handing out codes, and VaultDesk may simply continue the pattern.
What is many-shot jailbreaking?
Many-shot jailbreaking fills a prompt with a long made-up dialogue in which an assistant answers every question, including ones it should refuse. The real question comes at the end.
Language models learn from examples inside the prompt. After many examples of an assistant complying, complying looks like the pattern to continue. Larger context windows make the attack stronger because more examples fit.
A real case
NeuralTrust described an attack that never asks for the forbidden thing directly. Harmless-looking turns plant context, and later turns ask the model to build on what it already said, so each step looks acceptable on its own. The researchers reported high success rates against leading models.
Echo Chamber builds its pressure across real turns. Many-shot does it in one message by faking the turns, and both rely on the model continuing a pattern it believes is already established.
Source: NeuralTrust
How to fix it
- Limit prompt length where the product does not need long input.
- Detect and strip user input that imitates dialogue turns.
- Classify the prompt before it reaches the model.
- Put the key rules after the user's content as well as before.
Questions people ask
How many examples does it take?
Published research found that effectiveness keeps rising with the number of examples, into the hundreds for harder requests.
Why does a fake dialogue work?
In-context learning. The model takes examples in its prompt as a guide to how it should behave next.
Does it need a large context window?
Larger windows help the attacker. Small-scale versions can still work on assistants with weak rules.
Read the full LLM Jailbreaking guide · All free labs · AI red team certification