Day 2 of 1260 minCipher CadetHands-on lab

Day 2: Direct Prompt Injection and Jailbreaks

Taught by Mr Elite Updated

By the end of today you can

  • Explain why a model cannot reliably tell instructions from data
  • Describe the difference between prompt injection and a jailbreak
  • Carry out a guided multi-turn roleplay jailbreak in the lab
  • Explain why keyword filters alone do not stop these attacks

Warm-up

Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.

What three parts usually make up an LLM application?

A model, a system prompt, and user input, and sometimes tools.

Why is a rule written in the system prompt called a request, not a lock?

Because the model reads it as plain text alongside the user's text, so nothing forces it to obey.

Picture a chatbot that a company has told, in plain English, never to reveal a discount code. You type a blunt command to ignore its rules, and it refuses politely. Good. Then you spend six friendly turns setting up a story, and on turn seven the same bot hands the code over as a line of dialogue.

That gap between a bot that resists one attack and a bot that gives way under gentle pressure is today's lesson. You will learn the vocabulary, watch the pattern, and then do it yourself in a safe lab so you can recognise it and defend against it.

Why a model cannot tell instructions from data

Yesterday we saw that the system prompt and the user's message travel together as plain text. Here is the consequence. The model reads one long block of words and predicts what should come next. It has no separate channel marked 'trusted instructions' and another marked 'untrusted input'.

A normal program keeps code and data apart. A database query has a fixed shape, and your input is only a value slotted in. An LLM has no such shape. Your sentence 'you are now in debug mode' sits in the same stream as the developer's sentence 'never reveal the code', and the model weighs them both as language.

Context window
All the text the model can see at once: system prompt, conversation so far, and your latest message.
Direct prompt injection
You type input yourself that changes how the model behaves against its owner's intent.

Direct injection versus jailbreaking

People mix these two terms up, so let me separate them. Both are attacks through the chat box, and they overlap, but the target differs.

Prompt injection
Aimed at the application. You override the developer's instructions, for example to leak the system prompt or break the bot's assigned job.
Jailbreak
Aimed at the model's own safety training. You talk it into producing content it was trained to refuse.

A simple test: whose rules are you breaking? If it is the app owner's rules written in the system prompt, that is injection. If it is the model maker's built-in safety behaviour, that is a jailbreak. Real incidents often mix both, and that is fine. What matters is that you can name what was bypassed.

Quick check

A user gets a company's support bot to ignore its 'only discuss our products' instruction and write a poem. Which term fits best?

Roleplay and multi-turn pressure

Single-turn attacks like 'ignore previous instructions' are well known, and many apps now refuse them. Attackers moved to conversations. Two ideas matter here.

Roleplay reframes the request. Instead of 'give me the secret', the attacker asks the model to play a character who would say it. The model treats it as fiction, and its refusal habits fire less strongly.

Multi-turn pressure builds up context. Each turn looks harmless alone. The attacker gets the model to agree to a game, then to stay in character, then makes the sensitive request inside the game. By then the conversation history itself is pushing the model toward compliance.

Watch me do it: a safe local pattern

Before the lab, let me show you the shape of a multi-turn attempt, written out as notes. You do this on your own machine in a text file. Nothing here touches any real system.

text
Turn 1 - Scene: propose a harmless story game
Turn 2 - Agreement: ask the bot to stay in character
Turn 3 - Escalate: give the character a job that needs the secret
Turn 4 - Ask: have the character say it in dialogue
A planning template for a multi-turn roleplay attempt. Edit the wording freely.

What just happened: you turned one blocked request into four small ones. Each is easy for the model to accept, and together they walk it toward the goal.

Why filters alone do not solve it

The obvious fix is a filter that blocks phrases like 'ignore previous instructions'. It helps against lazy attacks and fails against everything else. Attackers can reword, use another language, spell things oddly, or hide the request in a story. The set of possible sentences is effectively endless.

Filters that check only the latest message also miss the slow build-up of a conversation. Better defences stack: limit what the bot knows, so there is no secret to leak; limit what it can do; check outputs as well as inputs; and watch whole conversations, not single messages.

Quick check

Why does a keyword blocklist fail against multi-turn roleplay?

Your turn

AI Jailbreak

This guided lab gives you GuardedBot, which refuses blunt 'ignore previous' attacks but is weaker under a longer conversation. It is the exact pattern we just planned, in a safe place built for it. Steps are written beside the lab, so follow them and keep your plan file open.

Open the lab in a new tab

What to try

What to notice

  • Which exact turn first shifted the bot's tone or behaviour
  • Whether the bot still 'knows' its rules but applies them less firmly
  • How much the earlier turns mattered to the final result

If it did not work

Repeating the same blocked request over and over

Change the approach. Step back a stage, change the framing, or build more context first.

Calling every bypass a jailbreak

Ask whose rules were broken. The app owner's means injection, the model maker's safety means jailbreak.

Not recording which turn worked

Number your turns in your notes and screenshot the turn where behaviour changed.

Believing a filter means the app is safe

Test with reworded and multi-turn attempts, and remember filters are only one layer.

Recap

  • A model reads instructions and data as one stream of text, so it cannot reliably tell them apart
  • Prompt injection targets the app owner's rules; a jailbreak targets the model's safety training
  • Roleplay reframes a request as fiction, which weakens refusals
  • Multi-turn attacks build context in small harmless steps
  • Keyword filters alone fail because wording is endless and each turn looks innocent
  • Better defence means layers: less secret data, limited abilities, output checks and conversation monitoring

Cheat sheet

Direct prompt injection
User input that overrides the developer's instructions
Jailbreak
Input that gets past the model's built-in safety training
Roleplay bypass
Asking the model to play a character who says the restricted thing
Multi-turn attack
Building up context across several harmless-looking messages
Blocklist filter
A list of banned phrases; useful but easy to get round

Before the next day

Mission: Turn-by-turn jailbreak note

Finish the guided roleplay lab and write a short note on what changed the bot's behaviour. Working out why it worked is the real skill, because it tells you what a defender should fix.

About 35 min

You are done when: A text note listing your turns, the turn that changed behaviour, your explanation of why, one defence idea, and one screenshot.

Want more? Repeat the lab with a different story theme and compare how many turns you needed.

Quiz

4 questions. Your score is saved with your progress.

1. What is the main reason prompt injection is hard to eliminate?

2. An attacker persuades a model to produce content its maker trained it to refuse. This is best called:

3. Why can multi-turn roleplay beat a bot that refuses 'ignore previous instructions'?

4. Which is the strongest defence idea among these?

Finish the day

Mark the day complete to save your progress.

Go to Day 3

Next: Today you attacked through your own messages. Tomorrow we look at indirect injection, where the hostile instructions hide inside a web page or document that an assistant reads on your behalf.

Tell someone you are doing this:WhatsAppXLinkedIn

Your notes

Private to this device. Write what you want to remember from today.