There’s a number I think about whenever someone asks me how LLMs got so good at sounding human. GPT-3, released in 2020, was trained on about 570 gigabytes of text. That’s roughly 300 billion words. If you read for 12 hours a day, every day, it would take you about 34,000 years to read that much. GPT-4 was trained on even more.
That scale is why LLMs are impressive. But the scale also creates problems nobody tells beginners about. When you train on everything ever written on the internet, you train on the good, the bad, the outdated, and the false. You train on biased opinions, wrong facts, old information, and content written by people who had no idea what they were talking about.
Today I’m going to explain how LLMs actually learn — the training process, what tokens are, what parameters are, and why the knowledge cutoff means your LLM might be living in the past. By the end you’ll understand why an LLM’s quality depends entirely on what went into it.
🎯 What You’ll Learn in Day 2
⏱ 20 min read · 3 exercises · Just a browser needed
- Completed Day 1: What Is a Large Language Model?
- Remember: an LLM predicts the next word — it’s not thinking, it’s pattern-matching
- Remember: ChatGPT is a product; the LLM is the engine inside it
How LLMs Learn From Text — Day 2 of 5
Day 1 explained what an LLM is and what it does. Today we go inside the learning process. Understanding this makes Day 3 (why LLMs make mistakes) make complete sense — because almost every LLM failure traces back to something about how it learned. Also: our email breach checker is a great example of an AI-powered security tool — the kind of tool that uses trained models to recognise patterns. Understanding training is understanding how these tools work too.
Training Data — The Raw Ingredient
Every time I explain LLMs to someone new, I start with training data. Not tokens. Not parameters. Training data. Because everything — the good outputs, the bad outputs, the biases, the blind spots — traces back to this one thing. Training data is the raw ingredient. Get it right and you get a brilliant LLM. Get it wrong and you get a confidently wrong one.
Before an LLM can do anything, it needs to learn. And to learn, it needs examples. Billions and billions of examples of human text. That collection of examples is called training data.
What goes into LLM training data? The honest answer is: most of the publicly available internet. Web pages. Wikipedia articles. Books. News articles. Code repositories. Reddit threads. Academic papers. Customer reviews. Subtitles from TV shows and films. Forum posts. Recipes. Legal documents. Technical manuals.
I think of training data as the LLM’s entire education. A human child learns language by hearing and reading examples over years. An LLM learns language by processing billions of text examples in weeks of compute time. The human child ends up knowing language shaped by their family, teachers, and community. The LLM ends up knowing language shaped by whatever was most common on the internet.
Three things about training data that really matter:
Volume: More text means more patterns learned. GPT-4-scale models are trained on trillions of words. That staggering volume is why they can discuss almost any topic — because almost every topic appeared somewhere in the training data.
Diversity: The text needs to cover many topics, styles, and languages for the LLM to generalise. An LLM trained only on English Wikipedia would struggle with casual conversation, poetry, and any language other than English. Good training data covers huge variation.
Quality: Garbage in, garbage out. If the training data contains lots of wrong information, outdated information, or biased content, the LLM learns those wrong, outdated, and biased patterns. The internet contains a lot of each. We’ll look at this on Day 3.
Tokens — What an LLM Actually Reads
Here’s something I find most people are surprised to learn: an LLM doesn’t actually read words. It reads tokens. A token is a chunk of text — usually 3 to 4 characters in English, roughly three-quarters of a word on average.
Why tokens instead of whole words? Because the LLM’s vocabulary would be impossibly large if every possible word was its own entry. There are hundreds of thousands of words in English alone, plus all their different forms, plus words from every other language, plus technical jargon, plus typos. Instead, LLMs use a fixed set of about 50,000 to 100,000 common fragments — tokens — that can be combined to represent any text.
Let me make this concrete with examples:
The word “cat” = 1 token (short, very common)
The word “running” = 1 token (common enough)
The word “tokenisation” = 2 tokens (“token” + “isation”)
The word “uncharacteristically” = 4 tokens (long, uncommon)
“Hello world” = 2 tokens
“The quick brown fox” = 4 tokens
When you type a message to an LLM, your text gets split into tokens first. The LLM processes those token numbers — not your actual text. When it generates a response, it generates token numbers, which get converted back to text before you see them.
do
LL
Ms
learn
?
Your name might split across 2-4 tokens. Unusual words split into many. Very common words are 1 token. The LLM never sees your text directly — only these numbered chunks.
Why does this matter to you? A few reasons. First: LLMs have a maximum number of tokens they can process at once — their “context window.” Long conversations or huge documents can hit this limit. Second: unusual words — names, technical jargon, words from less common languages — often get split into many tokens and can be processed less reliably. Third: some security tricks work by exploiting how tokenisation breaks words, making safety filters miss things that a human reader would catch instantly.
Parameters — Where the Learning Lives
When an LLM finishes training, what’s left? The training data is gone. The model is just a file of numbers. Those numbers are called parameters (also called weights).
Parameters are how the LLM stores everything it learned. Not as facts in a database — as patterns compressed into numbers. GPT-3 has 175 billion parameters. GPT-4 is estimated to have over a trillion. Each parameter is just a decimal number — something like 0.0023847. But billions of these numbers, organised into a specific structure and shaped by training on trillions of words, somehow encode enough about human language to produce remarkably good text.
My favourite analogy for parameters: think about how you learned to ride a bike. You can’t write down exactly what your body does — the specific muscle adjustments, the exact balance corrections, the precise timing. But your body knows. That knowledge is stored in your brain and muscles as patterns of connections, not as a written document. LLM parameters are similar — the knowledge isn’t stored as facts you could list, it’s stored as patterns of numerical relationships that the model uses when generating text.
This has an important consequence: you can’t just “edit” an LLM’s knowledge by changing a few numbers. The knowledge is distributed across all the parameters — there’s no single parameter that represents “Paris is the capital of France.” It’s everywhere and nowhere simultaneously. If an LLM learns something wrong, fixing it means retraining or using a special technique called “fine-tuning” — you can’t just open the file and correct an error like editing a document.
The Training Process — How It Goes From Nothing to Brilliant
I love explaining this part because it demolishes the mystery around LLMs. Once you see the actual steps, the whole thing goes from “black box magic” to “oh, that actually makes sense.” Here’s the training process in plain English — no maths required.
Step 1: Start with random numbers. Before training, all the parameters are set to random values. The LLM knows nothing. It would produce total gibberish if you asked it anything.
Step 2: Show it some text and ask it to predict. Take a piece of training text — say, the first sentence of a news article. Hide the last word. Ask the LLM: “given everything up to here, what word comes next?” The LLM guesses. With random parameters, the guess is garbage.
Step 3: Check the answer and measure how wrong it was. Compare the guess to the real word. Calculate a number representing “how wrong.” This number is called the loss. High loss = very wrong guess.
Step 4: Nudge the parameters slightly to be less wrong. Using the loss, adjust every parameter by a tiny amount in a direction that would have produced a less wrong guess. Each nudge is tiny. But repeated billions of times, the nudges add up.
Step 5: Repeat with the entire training dataset. Then repeat again. Then again. The model sees the same training data multiple times, getting better each pass. After trillions of these steps across billions of text examples, the parameters have been shaped by so many examples of good prediction that the model produces remarkably good text.
This whole process takes weeks on thousands of specialised computer chips. Training GPT-4 reportedly cost tens of millions of dollars in compute alone. That’s why trained models are valuable — and why stealing model parameters (called model theft) is a real security concern.
Every LLM has a training cutoff — the date where its knowledge stops. Past that date, it either doesn’t know about events or makes things up based on patterns from earlier data. Finding that cutoff is a useful skill. It tells you how much to trust the LLM’s answers about recent things. Let’s find it.
- Open any LLM (ChatGPT, Claude, or Gemini). Ask directly: “When does your training data cut off?” Note the answer.
- Now test whether that cutoff is accurate. Ask about a major event that happened before the stated cutoff. Does it know? Does it get it right?
- Ask about something that happened after the cutoff. What happens? Does it say it doesn’t know, or does it try to answer anyway?
- Ask about something happening right now — today’s news, today’s weather, live sports scores. What does it do?
- Based on your tests, does the LLM’s self-reported cutoff match its actual behaviour? Or does it seem to know more or less than it claims?
The Knowledge Cutoff — Your LLM Is Frozen in Time
Training an LLM takes weeks. Then it gets tested and refined for months. Then it gets deployed. From the moment training ends, the LLM’s core knowledge is frozen. It doesn’t learn anything new from conversations. It doesn’t read the news. It doesn’t know about things that happened after its training cutoff.
This creates a constantly growing gap. Training cutoffs for major models are typically 6 to 18 months before the public release date. By the time you’re using the model, its knowledge might already be one to two years old. And the model keeps being used for years after release, making that gap wider and wider.
Here’s what that means in practice. If you ask a question about last week’s news — the LLM doesn’t know. If you ask about a new law passed last month — it doesn’t know. If you ask about a new product released recently — it might not know, or it might know an outdated version. If you ask about ongoing situations that change rapidly — it has a snapshot from training time, not current reality.
The tricky part: many LLMs will still try to answer. They’ll produce fluent, confident-sounding text about things they don’t know — because their next-word prediction still works, just without accurate recent information backing it up. The prediction sounds plausible. It might even be correct by coincidence. But it’s pattern-matching from old data, not knowledge of what actually happened.
Some products add a workaround: web search. ChatGPT can browse the internet. Claude can use tools. Gemini has Google Search integration. When these features are active, the LLM can look up current information before answering. But the base LLM itself is still frozen — the search results are external information retrieved by a separate system and handed to the LLM as context.
Why Data Quality Is Everything
The training data is the LLM’s entire education. Whatever patterns appear most often in that data are the patterns the LLM learns most strongly. This creates three categories of problem that every LLM user should know about.
Bias. If the training data contains more content from certain demographics, countries, or perspectives, the LLM learns those perspectives as “normal.” Early image-generation LLMs showed doctors as male and nurses as female because that was the most common pattern in the training data images. Language LLMs can produce stereotyped responses about certain groups for the same reason — not intentionally, but because the training data contained those patterns disproportionately.
Outdated information. The internet contains enormous amounts of outdated content that’s never been updated or deleted. Old scientific consensus that’s been overturned. Outdated medical guidance. Political information from elections years ago. If a topic has changed significantly since the training data was collected, the LLM may confidently present the old information as current.
Intentional pollution. This is the security-relevant one. If someone deliberately floods the internet with false information before an LLM’s training data is collected, that false information gets baked into the model. This type of attack — called data poisoning — is a real concern for LLMs trained on publicly scraped web data. I cover this in detail in the AI security landscape article.
Training data quality is a security issue — and I want you to think through why. If an attacker could influence what goes into an LLM’s training data, what could they make the LLM do? This exercise develops the attacker mindset that’s essential for understanding AI security without you having to actually do anything harmful.
- Imagine an LLM being trained on web data that includes a large cybersecurity forum. An attacker has posted thousands of helpful-looking posts giving wrong security advice — like “the safest password is your birthday plus your name.”
- If the LLM learns from those posts, what might it tell users when asked about passwords?
- Now scale up. What if an attacker published thousands of fake news articles, all saying a specific medicine is safe when it’s actually dangerous? What could happen when users ask the LLM about that medicine?
- What about this: an attacker publishes content designed to make an LLM give always-biased answers about a political party or a specific company. How would a user even know the LLM’s opinions were tainted by this?
- Design one way a user could detect that an LLM might have been trained on biased or manipulated data on a topic they care about.
Tokenisation affects how LLMs handle unusual words — names, technical jargon, words from less common languages. When a word splits into many tokens, the LLM sometimes handles it differently than single-token words. You can test this right now with some creative experiments.
- Open any LLM. Ask it to write a short paragraph about someone named “Arijit” (or any unusual name you like). Notice: does it use the name correctly and consistently throughout?
- Ask it to spell an unusual long word letter by letter — “Pneumonoultramicroscopicsilicovolcanoconiosis”. Does it get it right? Count the letters. (The word has 45 letters.)
- Ask it to count the letters in a short word — try “strawberry”. How many R’s does it say there are? (There are 3.) Does it get it right?
- Ask it to reverse a simple word — “kitten”. Does it get “nettik”? Now try reversing a very long word. What happens?
- Based on what you found: what types of tasks does tokenisation make harder for an LLM? What does this tell you about when NOT to trust an LLM’s answer?
Questions and Answers
How much does it cost to train an LLM?
Enormous amounts — and the costs keep growing as models get larger. GPT-3’s training was estimated at around $4.6 million in compute costs. GPT-4 and similar frontier models are estimated in the tens to hundreds of millions of dollars. The compute happens on tens of thousands of specialised graphics cards (GPUs) running for weeks. This is why only large companies and well-funded research organisations can train the biggest models from scratch. Smaller organisations can “fine-tune” existing models on their own data for much less cost, but training a state-of-the-art model from zero requires serious infrastructure.
What is a context window and why does it matter?
The context window is the maximum amount of text (measured in tokens) that an LLM can consider at once during a conversation. Think of it as the LLM’s short-term memory. Everything within the context window influences the response. Everything outside it is invisible to the model. Typical context windows range from 8,000 tokens (small) to 1 million tokens (very large models). In practice: a long conversation, a big document you pasted in, or several articles you asked it to summarise can fill up the context window. When that happens, older parts of the conversation “fall off” the edge and the model loses access to that earlier context. This is why very long conversations sometimes feel like the LLM “forgot” something from earlier.
Is bigger always better for LLMs?
More parameters generally means more capability — but it’s not a linear relationship, and size isn’t the only thing that matters. Training data quality, training process design, and fine-tuning all significantly affect performance. A well-trained smaller model can outperform a poorly-trained larger one. Also, bigger models require more compute to run — they’re slower and more expensive to serve. The trend in 2025-2026 has been toward “efficient” models: smaller parameter counts trained on higher-quality data, achieving performance competitive with larger models. The race isn’t purely about size anymore — it’s about quality at scale.
Can I see what training data an LLM was trained on?
For most major commercial models — no. OpenAI, Anthropic, and Google don’t publicly disclose exactly what went into their training data. For open-source models like Meta’s Llama series, the training data composition is partially documented. Researchers have developed techniques for inferring what data a model was trained on by testing its knowledge — essentially probing its memory — but this is indirect evidence, not a complete list. This opacity around training data is a significant concern from both a safety and accountability perspective: if an LLM has biases or misinformation baked in, it can be very difficult to trace it back to its training data source.
Why does tokenisation matter for security?
Several real security implications. Safety filters trained to block certain words operate at the token level — if a harmful word splits into multiple tokens or gets encoded differently, it might pass through the filter undetected. This is why attackers use creative spellings, character substitution, and encoding tricks when trying to bypass LLM safety systems. Tokenisation also affects how consistently an LLM processes certain types of input — security researchers exploit tokenisation inconsistencies to find ways to make models behave unexpectedly. Understanding that the model sees tokens, not words, explains many of the unusual failure modes in LLM security research.
What happens to an LLM after training? Does it keep getting smarter?
After initial training, the base model is essentially frozen. But companies do several things to improve it: fine-tuning (additional training on specific high-quality datasets to improve performance on particular tasks), RLHF (Reinforcement Learning from Human Feedback — humans rate the model’s outputs, and the model learns to produce higher-rated outputs), and safety training (teaching the model to refuse harmful requests). All of these modify the parameters further without retraining from scratch. Periodically, companies release new model versions — GPT-4o, Claude 3.5 Sonnet — which represent fresh training runs, not updates to the old model. The old model version stays frozen while the new one is deployed.
Further Reading
- How LLMs Work — Hacker’s Guide — the technical deep-dive on everything from today
- LLM Day 8: Data Poisoning — the full coverage of data quality attacks
- LLM Hacking Hub — the advanced series this course leads into
- OWASP LLM Top 10 — includes training data and supply chain vulnerabilities
- MITRE ATT&CK — ML attack techniques including training data attacks
