Day 3 of 760 minCipher CadetHands-on lab

Day 3: Keep Secrets Out and Check the AI's Answers

Taught by Mr Elite Updated

By the end of today you can

  • List the kinds of secret that must never be pasted into a public AI chat
  • Redact keys, passwords, tokens and personal details from a log or config with placeholders
  • Spot hallucination and false confidence in an AI answer
  • Verify an AI claim against a trusted source and record what you found

Warm-up

Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.

In the role, context, task and format pattern, what does the context part give the model?

The information it cannot see for itself, such as the log line, the system involved or what you already know.

Where should instructions go relative to the pasted data, and why?

Instructions first, pasted data last, so the model knows what to do before it reads the material and the data is not mistaken for orders.

Picture someone tired and under pressure. An app keeps crashing, they copy the whole error output and the config file, paste it into a chatbot and ask for a fix. The answer is good. What they did not notice is that the paste held a database password, an API key and a customer email address. All of it has now left their machine.

Today has two halves. First you learn to clean text before it goes anywhere near an AI. Then you learn the other habit that goes with it: never trust the answer that comes back until you have checked it. Both skills are quick, and both stop real problems.

What counts as a secret

A secret is any piece of text that gives someone access, or identifies someone, and would cause harm if a stranger had it. Many beginners think only passwords count. In practice the list is longer, and the dangerous items are often buried in the middle of a long paste.

API key
A long string that proves your program is allowed to use a service. Whoever holds it can use the service as you, and you may pay the bill.
Connection string
A line that tells software how to reach a database, often with the username and password inside it.
PII
Personally identifiable information: names, emails, phone numbers, addresses, anything that points to a real person.

Add to that list private IP addresses and internal hostnames, private keys (the blocks starting with BEGIN PRIVATE KEY), session cookies, and anything your employer marks as confidential. A public chatbot may store what you type, and staff or systems may review it. Treat every paste as if it will be read by someone you never met.

Quick check

Which of these is the most dangerous to paste into a public AI chat?

Redacting: replace, do not delete

Redacting means swapping the sensitive part for a placeholder. I say replace rather than delete because the model still needs the shape of the text to help you. If you cut a line out, the answer may no longer make sense. If you write a clear label, the structure stays and the secret goes.

Use labels that say what was removed, and keep them consistent. If the same key appears twice, use the same placeholder twice, so the model can see they match.

text
Before:
DB_URL=postgres://admin:Sup3rS3cret@10.0.4.12:5432/shop
API_KEY=sk_live_9f8a7b6c5d4e3f2a1b0c
user jane.smith@example.com failed login

After:
DB_URL=postgres://<DB_USER>:<DB_PASSWORD>@<DB_HOST>:5432/shop
API_KEY=<API_KEY>
user <EMAIL_1> failed login
The structure of each line survives, so the AI can still reason about it, but no working secret remains.

Watch me do it: redact a paste

I will clean a small made-up log the way I would before asking for help. Everything here is fake sample text, so you can follow along in your own text editor. Nothing touches a real system.

text
2025-03-02 09:14:07 login failed user=jane.smith@example.com
2025-03-02 09:14:07 source=192.168.1.40 password=Winter2025!
2025-03-02 09:14:09 retry with header Authorization: Bearer eyJhbGciOiJIUzI1NiJ9.abc123
2025-03-02 09:14:10 upstream db at 10.0.4.12 refused connection
2025-03-02 09:14:11 login failed user=jane.smith@example.com
The made-up sample log. It holds an email, an internal IP, a password and a token.
text
2025-03-02 09:14:07 login failed user=<EMAIL_1>
2025-03-02 09:14:07 source=<INTERNAL_IP_1> password=<PASSWORD>
2025-03-02 09:14:09 retry with header Authorization: Bearer <TOKEN>
2025-03-02 09:14:10 upstream db at <INTERNAL_IP_2> refused connection
2025-03-02 09:14:11 login failed user=<EMAIL_1>
The cleaned version. Same story, same shape, nothing usable. EMAIL_1 appears twice because it is the same person.

What just happened: the AI can still tell you that repeated failures came from one user, that a token was sent and that the database refused a connection. None of the dangerous values left your machine.

Quick check

Why do we use the same placeholder, such as EMAIL_1, for repeated values?

Hallucination and false confidence

Now the other half. An AI model predicts likely text. It does not look facts up the way a search engine does. Sometimes it produces something that sounds right and is not. That is called a hallucination: an invented fact, command, function name or reference, delivered in a calm voice.

False confidence is related. The model rarely says I am not sure. A wrong answer and a right answer often read exactly the same. In security work this matters, because a made-up command flag or a wrong claim that something is safe can waste hours or leave a hole open.

Two kinds of mistake to watch for today. First, invented details, such as a tool option, CVE number or setting that does not exist. Second, work that looks complete but proves nothing, such as tests that pass whatever the code does. You will meet the second kind in the lab.

Verify against a trusted source

Verifying means checking a claim somewhere the AI does not control. Good sources are official documentation, the vendor's own site, the manual page on your machine, or the actual text you pasted in. Another AI answer is not a trusted source, because it can be wrong in the same way.

bash
ping --help
Your own machine's built-in help is a trusted source. Run this and compare the options with what an AI told you.

Quick check

An AI tells you a command has a flag called --safe-mode. What is the best next step?

Today's labs

Two labs, one for each half of the day. Both are SecurityElites practice labs, so you work on provided sample material and touch no real system.

Your turn

Secrets in the Context Window

This lab puts the redaction habit under pressure: a stack trace, a config snippet and logs, all about to go into a public chatbot. Start here, then do the second lab, ai-dev-fake-tests-1, where you catch an AI that looks confident but proves nothing. Log in to your SecurityElites account to open both.

Open the lab in a new tab

What to try

What to notice

  • How many secrets hide inside text that looks like harmless debugging output
  • That a redacted paste still makes sense and is still useful
  • That 100% coverage and green ticks do not mean the tests check anything

If it did not work

Redacting only the obvious password and missing tokens, emails or internal hostnames

Do two passes: one search by keyword, then one slow read of the full text before sending.

Deleting whole lines so the AI loses the context it needs

Replace the value with a labelled placeholder and keep the line.

Believing an answer because it sounds confident

Pick one concrete claim and check it in official documentation or against your own text.

Using another AI to verify the first AI

Use a source the model does not control, such as docs, a manual page or the original file.

Recap

  • A secret is anything that grants access or identifies a person, not only passwords
  • Replace secrets with consistent labelled placeholders so the structure survives
  • Assume anything pasted into a public AI could be stored, so redact first
  • Hallucinations are invented facts delivered with confidence
  • Asking for step-by-step reasoning helps you find weak spots but does not replace checking
  • Verify against official documentation or the original text, and write the result down

Cheat sheet

<PASSWORD>, <API_KEY>, <TOKEN>
Placeholder labels that replace real secret values
EMAIL_1, EMAIL_2
Numbered placeholders that keep repeated values linked
Hallucination
An invented fact or command presented as true
False confidence
A wrong answer that sounds as sure as a right one
Rotate a secret
Create a new one and disable the leaked one
Explain step by step and mark uncertainty
A follow-up that exposes weak reasoning
tool --help
Built-in help on your own machine for checking options

Before the next day

Mission: Redact, run, catch

Finish both labs, then write a short note that proves you can protect a paste and challenge an answer. This is the habit that keeps your later AI work safe, and it prepares you for attacking and defending AI systems from tomorrow.

About 30 min

You are done when: A saved day3.txt listing the secret types you redacted and one AI mistake you caught, with the source you used to verify it.

Want more? Ask an AI assistant to explain a command option, then check it against the tool's own --help output or manual page, and record whether it was right.

Quiz

4 questions. Your score is saved with your progress.

1. Why is replacing a password with better than deleting the line?

2. You pasted a live API key into a public chatbot by accident. What is the right response?

3. Which best describes a hallucination?

4. AI-written tests give 100% coverage. Why might you still distrust them?

Finish the day

Mark the day complete to save your progress.

Go to Day 4

Next: Day 4 turns the lens around: instead of protecting what you send to an AI, you see how text sent to it can act as orders. You will meet prompt injection and try it on the practice sales bot.

Tell someone you are doing this:WhatsAppXLinkedIn

Your notes

Private to this device. Write what you want to remember from today.