Day 5 of 1260 minCipher CadetHands-on lab

Day 5: Secrets in the Context Window: Redact Before You Paste

Taught by Mr Elite Updated

By the end of today you can

  • Explain what a context window is and who may be able to see its contents
  • Recognise API keys, tokens, passwords and personal data inside logs and config files
  • Redact a paste so it stays useful for debugging without exposing secrets
  • Write a personal checklist of things you never paste into an AI tool

Warm-up

Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.

Yesterday we said a system prompt is not a secret store. Why not?

It is readable text in the context window, so the model can be tricked into repeating it.

Where should real secrets such as API keys be kept instead?

In server-side configuration, with permissions enforced in code, not in prompts.

A developer has a bug. The app crashes, the stack trace is long, and a public AI chatbot would probably explain it in seconds. So they select everything in the terminal, copy it, and paste it into the chat box. Nobody is attacking anything. It feels like asking a colleague for help.

Buried in that paste, though, is a database password, a live API key and a customer's email address. Those now sit in a service the developer does not control. Today you learn to spot that before you press Enter, and to clean the text so you still get your answer.

What the context window is and who can see it

Every time you talk to an LLM, the model reads a block of text and replies to it. That block is the context window. It holds the system prompt, your messages, earlier chat history, any files you attached, and any documents the app fetched for you. The model only knows what is in that window, plus what it learned in training.

Now ask who else can see that text. The answer depends on the service, but for a public chatbot you should assume the provider can store it, staff may review it in some cases, it may appear in logs, and it may be used to improve future models unless you have opted out. Even with good policies, you have lost control of a copy. A secret you paste cannot be un-pasted.

Think of it like posting a letter. Once it is in the box, you cannot reach in and take it back, and you do not know how many hands it passes through. Most providers handle data carefully. The point is that your safety should not depend on that. It should depend on what you chose to send.

Deleting the chat afterwards does not reliably help either. Deleting hides the conversation from your screen. It does not promise that every backup, log or review queue has forgotten it. So the only dependable control sits before you paste, and that is what we practise today.

Context window
All the text the model can see for one reply: instructions, chat history, pasted content and retrieved documents.
Secret
Any value that grants access or identifies a person, such as a password, key, token or personal detail.
Redaction
Replacing a sensitive value with a harmless placeholder before sharing the text.

Common secrets and where they hide

Most leaks are accidents, not choices. People paste a whole file when they only needed three lines. So learn the shapes secrets take. Once you know them, they stand out in a wall of text.

API key
A long random string that lets a program use a service, often starting with a recognisable prefix such as sk-, AKIA or ghp_.
Token
A temporary or permanent proof of identity. Bearer tokens in an Authorization header and JWTs (three chunks separated by dots, starting eyJ) are common.
Connection string
A URL such as postgres://user:password@host/db that carries a password inside it.
Personal data
Names, emails, phone numbers, addresses, account IDs, IP addresses tied to a person.

Secrets hide in predictable places: .env files, config files, command history, curl commands with headers, stack traces that print environment variables, and log lines that record a user's request. Look for words like password, secret, token, key, auth, cookie and bearer. Then look for any long string that seems random.

Notice that a secret does not have to be labelled. A header like Authorization: Bearer is a clue, but a bare string of forty random characters in the middle of a log line is just as dangerous. Personal data is the same. A customer's email in an error message is not a password, yet it still identifies a real person, and you may have a legal duty to protect it.

One more habit helps. Before you paste, ask what the helper actually needs to see. For most bugs the answer is the error message, a few lines of surrounding code and the versions involved. It is almost never the credentials, the user details or the rest of the file.

Quick check

Which of these lines is the most dangerous to paste into a public chatbot?

Redaction habits and simple checks

Good redaction keeps the structure and removes the value. The AI still needs to see that a key exists and where it is used. It does not need the real key. Use clear placeholders so you can tell later what was removed, and so the model does not mistake them for real data.

text
Before:
Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.abc123.xyz789
user: maria.jones@example.com

After:
Authorization: Bearer <REDACTED_TOKEN>
user: <USER_EMAIL>
Keep the shape of the line, replace only the sensitive value with a labelled placeholder.

Notice what stayed. The word Bearer is still there, so a helper can see the request is authenticated. The label user is still there too. Only the values changed. If I had deleted both lines, the helper might miss that an auth header is part of the problem.

Build three habits. First, paste the smallest piece that shows the problem, not the whole file. Second, search your text before sending it. Third, read it once more from the top as if you were a stranger. Do not trust a quick glance, because secrets sit in the middle of long lines.

bash
grep -n -i -E "password|secret|token|key|bearer|@" paste.txt
Run on your own text file to list every line containing common secret words or an email address, with line numbers.

Here is how to read that command. grep searches a file for a pattern. The -n flag adds line numbers, -i ignores capital letters, and -E lets the vertical bars mean or. So it prints any line containing password or secret or token, and so on. On Mac and Linux, open Terminal. On Windows, use Git Bash, or PowerShell with the command below.

powershell
Select-String -Path paste.txt -Pattern "password|secret|token|key|bearer|@"
The PowerShell equivalent. It prints each matching line with its file name and line number.

Be aware that this search is a net, not a guarantee. It will catch lines that are harmless, because the word key appears in ordinary words. It will also miss a secret with no obvious label. Treat its output as a list of places to look, then still do the slow read from the top.

Training data memorisation in plain terms

There is a second route to leakage. Models learn from huge amounts of text. Sometimes, especially with text that appeared many times or in a distinctive form, a model can reproduce part of it word for word. This is called memorisation. It is not common for any single paste, and providers work to reduce it, but it is the reason we treat anything that might enter training data as potentially public.

You cannot check from the outside whether a given secret was learned. So the safe rule is simple. If you would not put it on a public notice board, do not give it to a tool that might train on it. Business accounts and local models change the risk, but the habit of redacting costs seconds and is worth keeping everywhere.

Check your settings too. Many services offer a switch to stop your chats being used for training, and business plans often promise this by default. That lowers one risk. It does not remove storage, logging or review, so it is a second layer and never a replacement for redacting.

Quick check

Why is it risky to paste a real API key into a chatbot, even if the chatbot answers correctly?

Watch me do it: redact a paste by hand

I will show you my process on a fake paste. Everything here is invented and stays on your own machine. Nothing is sent anywhere. Follow along in a plain text editor, and do each step yourself rather than just reading.

text
2025-03-02 10:14:07 ERROR login failed for maria.jones@example.com
API_KEY=sk-test-9f8e7d6c5b4a3210
DATABASE_URL=postgres://admin:Tr0ub4dor@db.internal:5432/shop
Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.abc123.xyz789
Traceback: KeyError: 'user_id' at orders.py line 88
Fake sample paste. The key, password, host, email and token are all invented.
text
2025-03-02 10:14:07 ERROR login failed for <USER_EMAIL>
API_KEY=<REDACTED_KEY>
DATABASE_URL=postgres://<USER>:<PASSWORD>@<HOST>:5432/shop
Authorization: Bearer <REDACTED_TOKEN>
Traceback: KeyError: 'user_id' at orders.py line 88
The cleaned version. The bug, a KeyError on user_id, is still fully visible.

Look at what just happened. We found five matches, kept one as useful evidence, and replaced the other four values. The connection string kept its shape, postgres, port and database name, so a helper can still reason about it. The email, key, password, host and token are all gone, and our final search proved it.

Your practice: the redaction lab

Now you try it in the SecurityElites lab named Secrets in the Context Window: Redact Before You Paste. It uses a safe practice scenario, so no real credentials are involved. Open it from this lesson page, read the whole paste before you edit anything, and use the same method you just watched: search, replace, read again, then verify.

Expect more variety than in my sample. There will be more than one kind of file, and some secrets will not carry an obvious label. The next section gives you the brief, with hints on where to look but not the answers.

Your turn

Secrets in the Context Window

A developer is about to paste a stack trace, a config snippet and some logs into a public chatbot. Everything in that paste could be stored, logged or learned, so your job is to find every secret and remove it while keeping the bug understandable. This is the same process you just practised by hand, with more places to hide things.

Open the lab in a new tab

What to try

What to notice

  • Secrets appear in more than one format, including inside URLs and headers
  • Personal data counts as sensitive even when it is not a password
  • Some secrets sit in the middle of long lines where a quick glance misses them
  • Placeholders with labels make the redacted text easier to read

If it did not work

Pasting the whole file because it is quicker

Select only the lines that show the problem. Less text means fewer chances to leak.

Redacting the key but leaving the password inside a connection string

Search for the @ symbol and for URLs with a colon before it. Check every line, not just those labelled secret.

Forgetting personal data in logs

Scan for emails, names, phone numbers and user IDs. Replace them with placeholders such as .

Thinking redaction fixes a secret already pasted

Rotate it. Create a new secret, disable the old one and check where the old one was used.

Trusting grep as the only check

Grep misses unlabelled strings and flags harmless words. Use it to find places to look, then read the whole text from the top.

Recap

  • The context window is everything the model sees, and a public service may store or log it
  • Treat anything you paste as possibly stored, reviewed or used in training
  • Secrets include keys, tokens, passwords, connection strings and personal data
  • Redact by replacing values with labelled placeholders and keeping the structure
  • Search your text, then read it again from the top before you send it
  • If a real secret has been exposed, rotate it

Cheat sheet

grep -n -i -E "password|secret|token|key|bearer|@" file
List lines likely to contain secrets, with line numbers
Select-String -Path file -Pattern "password|secret|token|key|bearer|@"
The same search in Windows PowerShell
<REDACTED_KEY>
Placeholder for an API key, keeps the line readable
Bearer eyJ...
A JWT token in an Authorization header, treat as a password
postgres://user:pass@host/db
Connection string with the password inside it
Rotate
Replace an exposed secret with a new one and disable the old
Smallest useful paste
Share only the lines that show the problem

Before the next day

Mission: Finish the lab and write your never-paste checklist

Complete the redaction lab if you have not already, then turn what you learned into a short personal rule set. A checklist you wrote yourself is one you will actually use the next time you are tired and in a hurry.

About 30 min

You are done when: A file day5-never-paste.txt with five items and reasons, plus a screenshot of the completed lab.

Want more? Take one of your own old code or config files, copy it, and redact it using grep and the manual method. Count how many secrets you missed on the first pass.

Quiz

4 questions. Your score is saved with your progress.

1. What does the context window contain?

2. Which redaction keeps a bug report most useful?

3. You pasted a real API key into a public chatbot by mistake. What is the best next step?

4. Why does memorisation matter for what you paste?

Finish the day

Mark the day complete to save your progress.

Go to Day 6

Next: Day 6 moves from what you give a model to what you bring in. We look at supply chain risk: poisoned models, packages and plugins, and how to vet a dependency list.

Tell someone you are doing this:WhatsAppXLinkedIn

Your notes

Private to this device. Write what you want to remember from today.