Day 6 of 1260 minCipher CadetHands-on lab

Day 6: Supply Chain Risk: Models, Packages and Plugins

Taught by Mr Elite Updated

By the end of today you can

  • List the parts of an AI application supply chain and say how each can be poisoned
  • Explain package hallucination and slopsquatting in plain words
  • Check a package's name, owner, age and download history on a registry
  • Explain why pinning and reviewing matter and write a pinned requirements file

Warm-up

Answer from memory first, then tap to check. Recalling yesterday is what makes it stick.

What is the context window?

Everything the model can see in one request: your prompt, earlier messages, pasted files and any retrieved text.

If a real API key has been pasted into a public AI tool, what should you do?

Treat it as exposed and rotate it, meaning revoke it and issue a new one.

You ask an AI coding assistant to set up a small web API. It replies in seconds with a neat install command listing seven packages. It looks tidy and confident, and one paste would have it all running.

Here is the catch. Some of those names may not be what they seem. One may not exist until an attacker registers it. Another may be a lookalike of a famous library. Today I will show you how to slow down for two minutes and check, because that pause is the whole defence.

What an AI supply chain contains

A supply chain is everything you bring in from other people to build your product. You did not write it, but you run it with your own permissions. An LLM application has more of these than most software, and each one is a way in for an attacker.

Model weights
The big file of numbers that is the trained model. If someone tampers with it, the model can behave badly in ways that are hard to see.
Package
A bundle of code, such as a Python library from PyPI or a JavaScript one from npm, that you install with one command.
Plugin
An add-on that gives an assistant extra abilities, such as reading email or calling a web service.
Dependency
Any package your project needs to run. Dependencies have their own dependencies, so the list grows quickly.
Registry
The public catalogue where packages are published, such as pypi.org or npmjs.com. Anyone can publish there.

The danger is simple. Installing a package can run its code on your machine straight away, with your files and your keys in reach. A poisoned model from a download site, a trojaned package or a rogue plugin all arrive the same way: you trusted a name.

Let me make each one concrete. A model file in some older formats can contain code that runs the moment you load it, so a tampered download can act before you ever type a prompt. A package can carry an install script that reads environment variables, where API keys often live. A plugin can ask for access to your mailbox or files, and a careless yes hands all of it over.

Quick check

Why is installing a package riskier than reading about one?

Package hallucination and slopsquatting

A hallucination is when a model states something that is not true, confidently. With code, that includes inventing package names that sound right but do not exist. Models do this fairly often, and often they repeat the same invented name for many different users.

Slopsquatting is what attackers do with that. They notice a name an AI keeps inventing, register it on the registry, and fill it with harmful code. The next person who pastes the AI's install command gets the real, malicious package. Nothing was hacked. The assistant simply sent you to a trap.

Here is the sequence, so you can spot it. First the model suggests a name that sounds plausible, such as flask-jwt-helper-pro. Second, nobody owns that name yet. Third, an attacker registers it and uploads code that looks harmless on the surface. Fourth, a developer pastes the command without checking, and the install runs the attacker's code.

Other warning signs are lookalike names (a famous library with an extra word or swapped letters), brand new packages with almost no users, and packages whose description does not match what you asked for.

There is a trap in the other direction too. A name that exists is not a name that is safe. Once an attacker has registered a hallucinated name, a search will find it. That is why we check four separate signals rather than one.

Watch me do it: vetting one package

I will check one name on the Python registry. Everything here is read-only. We are looking, not installing. Use the name requests as our known good example, then repeat the same steps on any name an AI gives you. First the browser method, then a quick command-line version.

Now the same checks from a terminal on your own machine. PyPI publishes a public JSON page for every package. This command only reads that page and prints three facts: the name, the newest version and the earliest upload date.

bash
curl -s https://pypi.org/pypi/requests/json | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['info']['name'], d['info']['version']); print(min(f['upload_time'] for v in d['releases'].values() for f in v))"
Fetches the registry record for requests and prints its name, latest version and the date of its earliest upload.

You should see the name requests, a version number such as 2.x.x, and an earliest date many years in the past. If the command prints nothing or an error, check that you have internet access and that python3 is installed by running python3 --version.

Next, see what a missing name looks like. This asks only for the HTTP status code, the number the server replies with. A real package returns 200. A name nobody has registered returns 404.

bash
curl -s -o /dev/null -w "%{http_code}n" https://pypi.org/pypi/requests/json
curl -s -o /dev/null -w "%{http_code}n" https://pypi.org/pypi/zz-made-up-name-48213/json
Prints 200 for an existing package and 404 for a name that is not registered.

Expect 200 on the first line and 404 on the second. A 404 on an AI-suggested name tells you the model invented it. Do not register it, and do not install it. Just reject the suggestion and look for a real library.

Pinning and reviewing what you install

Pinning means fixing the exact version you checked, so an update cannot silently swap in something new. If you write requests without a version, tomorrow you might receive a different release than the one you reviewed. A pinned line looks like this.

text
requests==2.31.0
flask==3.0.0
A requirements.txt with exact versions. The double equals sign means this version and no other.

Let me show you how long a list really gets. We will use a virtual environment, which is a throwaway folder of Python packages kept separate from the rest of your machine. We install only requests, a package we have just vetted, and then look at what came with it.

Review means reading the list before you run it. Ask four things: do I recognise each name, does each one have a reason to be there, did I check the ones I did not know, and is anything doing a job I did not ask for? A web API does not need a package called crypto-wallet-helper.

The same habit applies beyond code. For models, download from the publisher's official page, check who uploaded it, and prefer formats that cannot run code on load. For plugins, read what permissions they ask for and refuse any that want more than the task needs.

Quick check

An AI suggests a package. The registry shows it was created four days ago with 12 downloads. What is the best action?

Lab: Check Before You Install

Now you do it on a realistic case. The lab gives you an AI's install command with seven packages. Three should not be installed. Your job is to find which ones and prove it with evidence.

The lab runs in the SecurityElites practice environment, so nothing you look up touches a real registry or your own machine. Start the lab named Check Before You Install, read the install command through once, and then work down the list one name at a time using the same four checks from the worked example.

Keep a plain text file open beside the lab as you go. Write each name, then the four findings, then your decision. If you get stuck on one package, move on and return to it. Comparing it with the others often shows what a normal entry looks like.

Your turn

Check Before You Install

This lab recreates the exact moment from the start of today: an AI hands you a confident install command and you decide what to trust. It runs in the SecurityElites practice environment, so nothing touches a real registry or your machine.

Open the lab in a new tab

What to try

What to notice

  • How convincing a bad name can look until you compare it character by character
  • That age and download count often tell you more than the description
  • That a package can look fine on one check and fail another, so one signal is not enough

If it did not work

Trusting a package because the AI sounded certain

Confidence is not evidence. Check the registry for every name you do not already know, every time.

Checking only whether the page exists

Attackers register hallucinated names, so a page proves nothing. Check owner, age, downloads and linked source.

Leaving versions unpinned

Write exact versions with == after you have reviewed them, and review again when you update.

Only recording the rejects

Write evidence for the kept packages too. It shows you checked rather than guessed.

Recap

  • An AI supply chain includes models, packages, plugins and the data they bring with them
  • Installing runs someone else's code with your permissions
  • Slopsquatting turns an AI's invented package name into a real malicious package
  • Check name, owner, age and download history before you install anything unfamiliar
  • Pin exact versions and review the whole list before you run it
  • Write down evidence for every decision

Cheat sheet

Slopsquatting
Registering a package name an AI tends to invent, then filling it with malicious code
Typosquatting
Registering a name close to a popular one, hoping for typing mistakes
Four checks
Name, owner, age, download history
pypi.org
Public Python package registry; shows release history and maintainers
pepy.tech
Free site showing download counts for Python packages
package==1.2.3
Pins an exact version in requirements.txt
Registry 404
The name does not exist yet, so an attacker could claim it
pip freeze
Lists every installed package with its exact version, including dependencies you did not ask for

Before the next day

Mission: Dependency check report

Finish the lab, then write a short report of what you rejected and why. This is the habit you will reuse in the capstone audit, and it is the evidence a reviewer would want to see.

About 30 min

You are done when: day6-dependency-check.txt with seven decisions and evidence, plus a screenshot of the completed lab

Want more? Ask any AI assistant to suggest packages for a small project of your choice, then vet every name on pypi.org or npmjs.com without installing anything, and record whether any name did not exist.

Quiz

4 questions. Your score is saved with your progress.

1. What makes slopsquatting different from ordinary typosquatting?

2. A package page exists and its description matches what you wanted. Why is that not enough to trust it?

3. What does pinning protect you from?

4. An AI-suggested list for a simple web API includes a package that handles cryptocurrency wallets. What should you conclude?

Finish the day

Mark the day complete to save your progress.

Go to Day 7

Next: Day 7 moves from what you install to what the model learns from and retrieves. You will see how bad data in a training set or knowledge base changes answers, and demonstrate it in a lab.

Tell someone you are doing this:WhatsAppXLinkedIn

Your notes

Private to this device. Write what you want to remember from today.