Day 12 of 1260 minCipher CadetHands-on lab
Day 12: Capstone: Audit an AI Application and What Comes Next
Taught by Mr Elite Updated
By the end of today you can
- Use the OWASP Top 10 for LLM Applications as an audit checklist
- Review an AI-written pull request and spot planted flaws
- Rank findings by impact and likelihood and justify the order
- Write a short audit report with a fix for each finding
Warm-up
Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.
Name two things you can cap to reduce denial of wallet risk.
Any two of: request rate, input size, number of agent steps, spend per user or per day.
What three questions help you decide whether a human must approve an action?
Is it reversible, how far does it reach, and what is the worst case if it goes wrong.
Imagine a colleague sends you a pull request (a proposed code change that someone must approve before it is merged) from an AI coding agent. The tests are green. The code is tidy. The description is polite and clear. Most reviewers would click approve in under a minute, and that habit is exactly what an attacker, or just an unlucky bug, relies on.
Today you stop being a student of single risks and become the reviewer. You will take the ten risks from the last eleven days, turn them into a checklist, use it on a real piece of code in a lab, and write up what you found in a way a manager could act on. That is the skill that gets paid for in AI security.
The Top 10 as an audit checklist
An audit is a structured look at a system to find weaknesses and report them. The structure matters more than cleverness. Without a checklist you find whatever catches your eye first and miss the rest. With one, you ask the same questions of every application and your results become comparable.
For each of the ten risks, I want you to hold one question in your head. Where in this application could this happen, and what would I look at to prove it?
- Prompt injection
- Text, from the user or from content the model reads, that changes what the model does. Look at every place untrusted text enters a prompt.
- Sensitive information disclosure and system prompt leakage
- Private data or hidden instructions coming out in answers. Look at what the model can see and what is in its prompt.
- Supply chain
- Models, packages and plugins you did not write. Look at dependency lists and where models are downloaded from.
- Data and model poisoning
- Bad data in training sets or knowledge bases. Look at who can add content and whether it is checked.
- Improper output handling
- Trusting the model's answer as safe. Look at where the output is rendered, run or stored.
- Excessive agency
- An agent with too many tools, too much permission or too little oversight. Look at tool lists and approval steps.
- Misinformation and unbounded consumption
- Confident wrong answers and unlimited use. Look at tests, human review, rate limits and spend caps.
Quick check
An app lets the model write its answer straight into a web page without any cleaning. Which checklist item does that fall under?
Trusting model output and placing it somewhere it can be rendered or run is the core of improper output handling.
Watch me do it: audit a small app on paper
Before the lab, let me show you the method on a tiny example I have made up. Imagine a support chatbot. It reads help articles from a shared wiki that any employee can edit, it can issue refunds with a tool, and it prints answers into an internal web page. Nothing here is a real system. Follow along in your text editor.
LLM08/05 Wiki open to all staff -> poisoning, indirect injection
Evidence: marker word added to a test page appears in bot answer
LLM06 Refund tool has no cap and no approval
Evidence: tool config shows amount limit = noneWhat just happened is simple. We turned a vague worry ('AI is risky') into specific, checkable statements tied to parts of the system. Your report will be built from lines like these.
Ranking findings by impact and likelihood
You will usually find more problems than anyone can fix this week. Ranking tells the team where to start. I use two questions. Impact: if this goes wrong, how bad is it for people, money or data? Likelihood: how easy is it for it to happen, and who could do it?
Score each from 1 (low) to 3 (high) and multiply. A 9 is fix now. A 1 or 2 can wait. This is rough on purpose. The point is a defensible order, not false precision.
Finding: Refund tool has no cap
Impact 3 (direct money loss) x Likelihood 2 (needs a hijack first) = 6Quick check
Finding A has impact 3 and likelihood 1. Finding B has impact 2 and likelihood 3. Which do you fix first by the simple score?
3 x 1 is 3 and 2 x 3 is 6, so B comes first, though A deserves a note because of its high impact.
Your hands-on exam: the pull request lab
Now the fresh exercise. The lab below gives you an AI-written pull request that looks clean. Five lines in it would cause harm in production. Everything in it runs inside the SecurityElites lab, so you are safe to explore. Log in to your free account, open the lab called Review the AI's Pull Request, and read the diff slowly.
Writing the audit report
A report is how your work becomes useful to someone else. Keep it short. A busy reader should understand the top three problems in two minutes. For each finding, use the same five parts every time.
AUDIT REPORT: <app or lab name>
1. Title: Hardcoded key in config
Risk: Sensitive information disclosure
Evidence: PR lab, line in settings file (screenshot 2)
Score: Impact 3 x Likelihood 3 = 9
Fix: Remove the key, rotate it, load from environmentMake each fix specific and doable. 'Improve security' is not a fix. 'Escape model output before rendering and cap refunds at a set amount with human approval above it' is one. Draw on your earlier labs for evidence: injection, leakage, poisoning, agency and unbounded use all gave you screenshots or notes already.
What to learn next
You now have a map of the ten risks and the habit of auditing against it. The field moves quickly, so here is where I would go next. Each of these builds directly on what you did in this course.
- Advanced agent attacks
- Agents that plan, use many tools and remember across sessions open up chained attacks. Study how one small hijack grows into a large one.
- MCP security
- The Model Context Protocol is a common way to connect models to tools and data. Learn how to judge a connector's permissions and trust it carefully.
- Defender labs
- Practice building guardrails, logging, rate limits and approval flows, not just breaking things. Defence is where most jobs are.
Keep your audit file. Reuse the checklist on every AI feature you meet at work or in study, and add a line each time you learn something new.
Your turn
Review the AI's Pull Request
This lab is your capstone exercise because real AI risk often arrives as a change that looks fine. You review a pull request from an AI coding agent that passes its tests, and your job is to find the five lines that would hurt in production using the Top 10 as your guide.
What to try
What to notice
- How tidy code and a confident description lower your guard
- Which of the five flaws map to more than one risk on your checklist
- Whether the tests would catch any of the five, and what that says about green builds
If it did not work
Trusting the pull request description and tests instead of reading the code
Review the changed lines themselves. Passing tests only show the code runs, not that it is safe.
Listing findings with no evidence
Attach a screenshot, line reference or note from a lab to every finding, or drop it.
Ranking by how interesting a flaw is
Score impact and likelihood, multiply, and sort by the result.
Writing vague fixes such as 'add validation'
State exactly what to change, where, and how to confirm it worked.
Recap
- The Top 10 works best as a checklist you apply to every part of an application
- AI-written code needs the same careful review as any other code, and a tidy look proves nothing
- Every finding needs evidence, a score and a specific fix
- Rank by impact times likelihood so the team knows where to start
- A short, ranked, evidenced report is the deliverable that matters
- Keep learning: agent attacks, MCP security and defender labs are the next steps
Cheat sheet
Audit- Structured review of a system to find and report weaknesses
Impact x Likelihood- Score each 1 to 3, multiply, fix the highest first
Finding format- Title, risk, evidence, score, fix
Pull request- A proposed code change awaiting review before merge
Review rule- Judge the changed lines, not the description or the green tests
Blank checklist row- Either checked and clear, or not yet checked. Mark which
Before the next day
Mission: Capstone audit report
Finish the pull request lab, then write a short audit report that ranks at least six findings across the Top 10. Use evidence from the lab and from your earlier days, and give a fix for each. This is your final deliverable and a piece you can show to others.
About 45 min
You are done when: A text file named day12-audit-report.txt with at least six ranked findings, each with risk, evidence, score and fix, plus the lab screenshot.
Want more? Add a short section called 'Next 30 days' that orders your fixes into what to do this week, this month and later, with one sentence of reasoning for each.
Quiz
4 questions. Your score is saved with your progress.
1. Why is a green test suite not enough to approve an AI-written pull request?
Tests check behaviour they were written for. Planted flaws such as a leaked secret or an unsafe dependency can pass them untouched.
2. A flaw has impact 3 and likelihood 3, another has impact 3 and likelihood 1. What does ranking tell you?
Multiplying gives 9 and 3, so the first is more urgent, though the second still belongs in the report.
3. Which finding is best written?
It names the problem, points to evidence and states a specific fix.
4. You are auditing an app and find a shared wiki anyone can edit feeding the chatbot. Which risks should you note?
Anyone who can edit the source can plant bad facts or hidden instructions that the model will then read.
Finish the day
Mark the day complete to save your progress.
What next: There is no Day 13, so keep going on your own. Pick one next step from today's list, such as a defender lab or reading about MCP security, and reuse your audit checklist on every AI feature you meet.
Your notes
Private to this device. Write what you want to remember from today.