Day 1 of 1060 minShell WalkerHands-on lab

Day 1: Why AI Code Needs Review

Taught by Mr Elite Updated

By the end of today you can

  • Explain why AI output must be treated as untrusted input
  • Name the five risks this course covers: flaws, secrets, permissions, poisoned input and fake packages
  • Confirm your terminal, git and editor are ready for the course
  • Spot a test that passes even when the code is broken

Picture an assistant that writes a discount function and seven tests for it in under a minute. The coverage report says 100%. Every test is green. You merge it, and a week later customers are getting negative prices. Nothing about the process looked wrong, and that is exactly the problem.

This course is about that gap between code that looks finished and code that is safe. Over ten days I will show you how AI coding tools put your projects at risk, and give you repeatable checks to catch it. Today we set the frame, check your tools, and run your first lab.

What this course covers

AI coding assistants write code, suggest packages, read your files and, in agent mode, run commands for you. That is a lot of trust handed to a tool that produces confident text, not guaranteed correct text. I am not here to tell you to stop using them. I am here to show you how to use them and still ship code you can defend.

The course runs in three parts. Days 2 and 3 are about reviewing generated code. Days 4 to 8 cover secrets, packages, agent permissions, hostile input and tool connections. Days 9 and 10 turn it all into a team policy and a full audit of a project. Every hands-on step runs on your own machine or in the SecurityElites labs, never against anyone else's system.

Each day follows the same shape. I explain one idea, I show you a small example step by step, and then you practise it yourself. Plan on about an hour. You do not need to be an expert programmer, but you should be able to read a short function and follow what it does. If you can do that, you have enough to start.

AI coding assistant
A tool that suggests or writes code from your prompts, usually inside your editor or chat window.
Agent mode
A setting where the assistant does not only suggest code but also edits files and runs commands on your behalf.

Setup check: terminal, git, editor

Before the theory, let me make sure your tools work. This takes about five minutes and saves you trouble later. Everything below runs only on your own computer.

bash
git --version
node --version && npm --version
python3 --version && pip --version
Run the lines for the runtime you use. Each should print a version number, not an error.
text
git version 2.43.0
v20.11.0
10.2.4
Python 3.12.1
pip 24.0 from /usr/lib/python3/dist-packages/pip (python 3.12)
Example output. Your numbers will differ. What matters is that nothing says command not found.

AI output as untrusted input

Here is the single idea I want you to carry through all ten days. Treat everything an AI gives you the way you would treat data typed into a web form by a stranger. You would not run that data without checking it. The same rule applies to generated code, suggested packages and commands.

Why? A model predicts likely text. It does not know your threat model, your data or your deadline. It can copy a pattern that was common in its training material, including insecure patterns. It can sound certain while being wrong. And it can be steered by text it reads, which we will see on Day 7.

Notice what this does not say. It does not say the output is bad, and it does not say the assistant is hostile. Most of the time the code is fine. The point is that you cannot tell the good output from the bad by how it reads, because both read the same way. Only checking separates them.

Untrusted input
Any data or code you did not write and verify yourself. It might be fine, but you check it before relying on it.
Threat model
A short description of what you are protecting, who might attack it and how.

Quick check

A colleague says, 'The AI wrote it and the tests pass, so it is safe.' What is the best response?

The five risks

Everything in this course falls under five risks. Learn these names now. We will return to each one with a hands-on exercise.

Insecure flaws
Generated code that works but is vulnerable, such as building a database query by joining strings, which allows injection.
Secrets
API keys, passwords and tokens that leak when you paste them into a prompt or when the assistant writes them into code.
Permissions
An agent that can read files, run commands or reach the network has a blast radius, which is everything it could damage if it goes wrong or is hijacked.
Poisoned input
Hostile text in an issue, web page or document that an agent reads and then obeys as if it were your instruction.
Fake packages
Package names an AI invents. An attacker can register that name and wait for someone to install it. This is called slopsquatting.

Notice that today's lab does not fit neatly into one of those five. It sits underneath all of them: false confidence. Tests, green ticks and fluent explanations make us stop looking. Each of the five risks gets worse when we stop looking.

A quick example of how they stack. An assistant invents a package name, you install it, and the tests pass because nobody tested what the package does. One risk, hidden by another. Keep that picture in mind as we go, because real incidents are rarely tidy.

Quick check

An agent reads a GitHub issue containing hidden text telling it to send your environment variables elsewhere. Which risk is this?

Watch me do it: spotting a test that tests nothing

Before you open the lab, let me show you the idea on a tiny example you can type yourself. We will use Python, but the logic is the same in any language. Everything runs in a throwaway folder on your own machine, and you can delete it afterwards.

bash
mkdir demo && cd demo
Creates and enters a scratch folder.
python
# discount.py
def apply_discount(price, percent):
    return price  # bug: percent is ignored
A deliberately broken function. It always returns the full price.
python
# test_discount.py
import unittest
from discount import apply_discount

class T(unittest.TestCase):
    def test_returns_number(self):
        self.assertIsInstance(apply_discount(100, 10), (int, float))

    # honest version to swap in:
    # def test_ten_percent(self):
    #     self.assertEqual(apply_discount(100, 10), 90)
The first test passes on broken code. The commented one would catch the bug.
bash
python3 -m unittest -v
Runs every test file in the folder. The -v flag prints each test name.
text
test_returns_number (test_discount.T) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.000s

OK
Expected output. Green, even though the function ignores the discount.
text
test_returns_number (test_discount.T) ... ok
test_ten_percent (test_discount.T) ... FAIL

AssertionError: 100 != 90

Ran 2 tests in 0.001s

FAILED (failures=1)
Expected output. The new test fails, which is the correct result because the function really is broken.

Look at what happened. The first test never said anything about the discount, so it could not notice the bug. The second test states a value I worked out myself, not one copied from the function. That independence is what gives a test its power. Fixing the function to return price * (100 - percent) / 100 would turn the second test green for the right reason.

Your lab: seven tests and 100% coverage

Now you do the same detective work at larger scale. The lab ai-dev-fake-tests-1 gives you a discount function and seven AI-written tests. Coverage reads 100%, yet some of those tests would pass on broken code. You are not asked to fix anything. You are asked to judge which tests deserve trust.

Your turn

Tests That Test Nothing

Now you try it at scale. In this lab an assistant wrote seven tests for a discount function and coverage reads 100%, but some tests would pass even if the function were completely broken. Your job is to find them before anyone trusts them. It is a beginner lab and runs entirely in your browser.

Open the lab in a new tab

What to try

What to notice

  • Coverage measures which lines ran, not whether anything was checked
  • Every test looks tidy and well named, which makes the weak ones easy to miss
  • The tests that check exact, independently known values are the ones that give real protection

If it did not work

Trusting a coverage number or a wall of green ticks

Coverage only proves lines ran. Read the assertions and ask whether each one would fail if the code were wrong.

Asking the same AI to confirm its own work is safe

Use the AI to draft, then review with a checklist you control. You will build one on Day 2.

Skipping setup checks and finding problems on Day 5

Run the version commands today. Fix missing tools now while nothing else depends on them.

Treating the AI as either useless or perfect

Treat it as a fast, fallible junior contributor. Useful, but every output gets reviewed.

Naming a test file without the test_ prefix, then seeing Ran 0 tests

Rename the file to start with test_ and run the command again from the same folder.

Recap

  • AI output is untrusted input and needs review before it ships
  • The five risks are insecure flaws, secrets, permissions, poisoned input and fake packages
  • False confidence from tests and coverage makes every other risk harder to see
  • A good test fails when the code is wrong; ask whether it would
  • Your terminal, git and editor must work before we go further

Cheat sheet

git --version
Confirms git is installed
node --version / python3 --version
Confirms your runtime is installed
python3 -m unittest -v
Runs the tests in the current folder and lists each result
Untrusted input
Anything you did not write and verify yourself, including AI output
Blast radius
Everything a tool or agent could damage if it fails or is hijacked
Slopsquatting
Registering a package name an AI tends to invent, so installs pull in attacker code
Break-it check
Would this test fail if I broke the code on purpose?

Before the next day

Mission: Why the tests gave false confidence

Finish the lab, then put what you learned into your own words. Writing it down forces you to understand it and gives you the first entry in the notes file you will keep for the whole course.

About 30 min

You are done when: A file called ai-review-notes.md containing three sentences about the false-confidence tests and a list of the five risks.

Want more? Take a small function from your own project, ask your assistant to write tests for it, then break the function on purpose and see how many tests still pass.

Quiz

4 questions. Your score is saved with your progress.

1. Why should AI-generated code be treated as untrusted input?

2. A test asserts only that a function returned a number. What is the main weakness?

3. Which pairing of risk and example is correct?

4. Coverage is 100% and all tests are green. What does that tell you for certain?

Finish the day

Mark the day complete to save your progress.

Go to Day 2

Next: Tomorrow we turn today's mindset into a tool: a repeatable security checklist for reading an AI-written pull request. The false-confidence habit you practised today becomes the first item on it.

Tell someone you are doing this:WhatsAppXLinkedIn

Your notes

Private to this device. Write what you want to remember from today.