Day 12 of 1260 minCipher CadetHands-on lab

Day 12: Capstone: Audit an AI Application and What Comes Next

Taught by Mr Elite Updated

By the end of today you can

  • Use the OWASP Top 10 for LLM Applications as an audit checklist
  • Review an AI-written pull request and spot planted flaws
  • Rank findings by impact and likelihood and justify the order
  • Write a short audit report with a fix for each finding

Warm-up

Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.

Name two things you can cap to reduce denial of wallet risk.

Any two of: request rate, input size, number of agent steps, spend per user or per day.

What three questions help you decide whether a human must approve an action?

Is it reversible, how far does it reach, and what is the worst case if it goes wrong.

Imagine a colleague sends you a pull request (a proposed code change that someone must approve before it is merged) from an AI coding agent. The tests are green. The code is tidy. The description is polite and clear. Most reviewers would click approve in under a minute, and that habit is exactly what an attacker, or just an unlucky bug, relies on.

Today you stop being a student of single risks and become the reviewer. You will take the ten risks from the last eleven days, turn them into a checklist, use it on a real piece of code in a lab, and write up what you found in a way a manager could act on. That is the skill that gets paid for in AI security.

The Top 10 as an audit checklist

An audit is a structured look at a system to find weaknesses and report them. The structure matters more than cleverness. Without a checklist you find whatever catches your eye first and miss the rest. With one, you ask the same questions of every application and your results become comparable.

For each of the ten risks, I want you to hold one question in your head. Where in this application could this happen, and what would I look at to prove it?

Prompt injection
Text, from the user or from content the model reads, that changes what the model does. Look at every place untrusted text enters a prompt.
Sensitive information disclosure and system prompt leakage
Private data or hidden instructions coming out in answers. Look at what the model can see and what is in its prompt.
Supply chain
Models, packages and plugins you did not write. Look at dependency lists and where models are downloaded from.
Data and model poisoning
Bad data in training sets or knowledge bases. Look at who can add content and whether it is checked.
Improper output handling
Trusting the model's answer as safe. Look at where the output is rendered, run or stored.
Excessive agency
An agent with too many tools, too much permission or too little oversight. Look at tool lists and approval steps.
Misinformation and unbounded consumption
Confident wrong answers and unlimited use. Look at tests, human review, rate limits and spend caps.

Quick check

An app lets the model write its answer straight into a web page without any cleaning. Which checklist item does that fall under?

Watch me do it: audit a small app on paper

Before the lab, let me show you the method on a tiny example I have made up. Imagine a support chatbot. It reads help articles from a shared wiki that any employee can edit, it can issue refunds with a tool, and it prints answers into an internal web page. Nothing here is a real system. Follow along in your text editor.

text
LLM08/05 Wiki open to all staff -> poisoning, indirect injection
Evidence: marker word added to a test page appears in bot answer
LLM06 Refund tool has no cap and no approval
Evidence: tool config shows amount limit = none
Two sample notes in the shape I want you to use: risk, finding, then the evidence that backs it.

What just happened is simple. We turned a vague worry ('AI is risky') into specific, checkable statements tied to parts of the system. Your report will be built from lines like these.

Ranking findings by impact and likelihood

You will usually find more problems than anyone can fix this week. Ranking tells the team where to start. I use two questions. Impact: if this goes wrong, how bad is it for people, money or data? Likelihood: how easy is it for it to happen, and who could do it?

Score each from 1 (low) to 3 (high) and multiply. A 9 is fix now. A 1 or 2 can wait. This is rough on purpose. The point is a defensible order, not false precision.

text
Finding: Refund tool has no cap
Impact 3 (direct money loss)  x  Likelihood 2 (needs a hijack first) = 6
A worked score. Write your reasoning in a few words beside each number so others can challenge it.

Quick check

Finding A has impact 3 and likelihood 1. Finding B has impact 2 and likelihood 3. Which do you fix first by the simple score?

Your hands-on exam: the pull request lab

Now the fresh exercise. The lab below gives you an AI-written pull request that looks clean. Five lines in it would cause harm in production. Everything in it runs inside the SecurityElites lab, so you are safe to explore. Log in to your free account, open the lab called Review the AI's Pull Request, and read the diff slowly.

Writing the audit report

A report is how your work becomes useful to someone else. Keep it short. A busy reader should understand the top three problems in two minutes. For each finding, use the same five parts every time.

text
AUDIT REPORT: <app or lab name>
1. Title: Hardcoded key in config
   Risk: Sensitive information disclosure
   Evidence: PR lab, line in settings file (screenshot 2)
   Score: Impact 3 x Likelihood 3 = 9
   Fix: Remove the key, rotate it, load from environment
One finding in report format. Repeat the block for each finding and sort the list by score, highest first.

Make each fix specific and doable. 'Improve security' is not a fix. 'Escape model output before rendering and cap refunds at a set amount with human approval above it' is one. Draw on your earlier labs for evidence: injection, leakage, poisoning, agency and unbounded use all gave you screenshots or notes already.

What to learn next

You now have a map of the ten risks and the habit of auditing against it. The field moves quickly, so here is where I would go next. Each of these builds directly on what you did in this course.

Advanced agent attacks
Agents that plan, use many tools and remember across sessions open up chained attacks. Study how one small hijack grows into a large one.
MCP security
The Model Context Protocol is a common way to connect models to tools and data. Learn how to judge a connector's permissions and trust it carefully.
Defender labs
Practice building guardrails, logging, rate limits and approval flows, not just breaking things. Defence is where most jobs are.

Keep your audit file. Reuse the checklist on every AI feature you meet at work or in study, and add a line each time you learn something new.

Your turn

Review the AI's Pull Request

This lab is your capstone exercise because real AI risk often arrives as a change that looks fine. You review a pull request from an AI coding agent that passes its tests, and your job is to find the five lines that would hurt in production using the Top 10 as your guide.

Open the lab in a new tab

What to try

What to notice

  • How tidy code and a confident description lower your guard
  • Which of the five flaws map to more than one risk on your checklist
  • Whether the tests would catch any of the five, and what that says about green builds

If it did not work

Trusting the pull request description and tests instead of reading the code

Review the changed lines themselves. Passing tests only show the code runs, not that it is safe.

Listing findings with no evidence

Attach a screenshot, line reference or note from a lab to every finding, or drop it.

Ranking by how interesting a flaw is

Score impact and likelihood, multiply, and sort by the result.

Writing vague fixes such as 'add validation'

State exactly what to change, where, and how to confirm it worked.

Recap

  • The Top 10 works best as a checklist you apply to every part of an application
  • AI-written code needs the same careful review as any other code, and a tidy look proves nothing
  • Every finding needs evidence, a score and a specific fix
  • Rank by impact times likelihood so the team knows where to start
  • A short, ranked, evidenced report is the deliverable that matters
  • Keep learning: agent attacks, MCP security and defender labs are the next steps

Cheat sheet

Audit
Structured review of a system to find and report weaknesses
Impact x Likelihood
Score each 1 to 3, multiply, fix the highest first
Finding format
Title, risk, evidence, score, fix
Pull request
A proposed code change awaiting review before merge
Review rule
Judge the changed lines, not the description or the green tests
Blank checklist row
Either checked and clear, or not yet checked. Mark which

Before the next day

Mission: Capstone audit report

Finish the pull request lab, then write a short audit report that ranks at least six findings across the Top 10. Use evidence from the lab and from your earlier days, and give a fix for each. This is your final deliverable and a piece you can show to others.

About 45 min

You are done when: A text file named day12-audit-report.txt with at least six ranked findings, each with risk, evidence, score and fix, plus the lab screenshot.

Want more? Add a short section called 'Next 30 days' that orders your fixes into what to do this week, this month and later, with one sentence of reasoning for each.

Quiz

4 questions. Your score is saved with your progress.

1. Why is a green test suite not enough to approve an AI-written pull request?

2. A flaw has impact 3 and likelihood 3, another has impact 3 and likelihood 1. What does ranking tell you?

3. Which finding is best written?

4. You are auditing an app and find a shared wiki anyone can edit feeding the chatbot. Which risks should you note?

Finish the day

Mark the day complete to save your progress.

What next: There is no Day 13, so keep going on your own. Pick one next step from today's list, such as a defender lab or reading about MCP security, and reuse your audit checklist on every AI feature you meet.

Tell someone you are doing this:WhatsAppXLinkedIn

Your notes

Private to this device. Write what you want to remember from today.