Why LLMs Make Mistakes — Hallucinations, Bias and Limits Explained | LLM Basics Day 3 of 5

Days 1 and 2 built your model of how LLMs work. Today that model makes LLM failures make complete sense. None of these mistakes are random or mysterious — they all flow directly from the “predict the next word” mechanism and the training data quality issues from Day 2. The phishing URL scanner we run at SecurityElites is a great counterexample: it’s a specialised model trained specifically on phishing patterns, not a general LLM — which is why it gives reliable results on that specific task even when general LLMs would struggle.

Hallucinations — Confident Lies From a Prediction Machine

A hallucination is when an LLM produces false information as if it were true. Not a mistake the LLM knows is uncertain. A confident, fluent, plausible-sounding statement about something that isn’t real.

The court case story I opened with is a perfect example. The LLM produced six detailed, realistic-sounding case citations — names, dates, courts, judges. None were real. But they followed exactly the pattern of real case citations. The LLM was doing its job: predicting the most probable next words given “find court cases supporting this legal argument.” The most probable next words looked like real case citations. The fact that the specific cases didn’t exist was irrelevant to the prediction engine.

Why does this happen? From Day 1: the LLM is always predicting. There’s no internal check that asks “is this actually true?” The model has no fact-checking mechanism. It has no memory of what actually happened. It just has patterns — statistical relationships between words and phrases from its training data. If the pattern of words that would follow your question is “a confident, detailed-sounding answer,” that’s what it produces — regardless of whether the specific details are real.

Here’s my favourite analogy. Imagine someone who has read every crime novel ever written and can reproduce any plot point, character name, or detective technique from any of those books. Ask them “what happened in the plot of [book title]” and they can give you a detailed, convincing answer — because they learned the patterns of how crime novel plots work. Now ask them about a book that doesn’t exist. They’ll still give you a detailed, convincing plot summary — because they’re generating what a book like that would contain, not reporting what actually exists. That’s an LLM hallucinating.

The Confidence Problem — Why You Can’t Trust the Tone

This is the part that really catches people out: an LLM’s confidence is constant. It sounds equally certain whether it’s absolutely right, approximately right, completely wrong, or making something up entirely.

Human experts show uncertainty through tone. “I’m not sure about the exact date, but…” “You might want to check this, but I think…” “I’m fairly confident that…” LLMs can produce these phrases — but they produce them based on what kinds of questions usually get hedged answers, not based on whether they actually know the answer.

Ask an LLM a very common factual question — “What is the capital of France?” — and it gives you a confident, correct answer. Ask it about a very obscure historical detail it likely wasn’t trained on — and it might give you an equally confident, completely fabricated answer. From your side, the two responses look identical in tone and confidence. There’s no alarm that fires when the LLM is guessing.

I’ve tested this personally and the results are striking. When you know an LLM is likely to struggle — very recent events, very specific local information, obscure technical details — and you ask it anyway, you often get an answer that reads exactly like its confident correct answers. The only way to distinguish them is to independently verify.

This is the most important practical lesson from Day 3: confidence in the response tells you nothing about accuracy in the response. You have to evaluate what was said, not how it was said.

Bias — When the Training Data Leans One Way

From Day 2: LLMs learn the patterns in their training data. If the training data has patterns that lean in one direction — more content from certain groups, perspectives, or time periods — the LLM learns those skewed patterns as “normal.”

This produces bias. Not intentional bias — there’s no programmer putting prejudice in deliberately. Emergent bias: patterns in the training data that reflect the biases of whoever wrote that training data, which was mostly English-language internet content from mostly Western, mostly educated, mostly adult sources.

Some examples I find when testing LLMs:

Language bias. LLMs trained on mostly English data perform worse on other languages — not just translation quality, but general reasoning and knowledge tasks in those languages. Ask an LLM a knowledge question in Hindi or Swahili and the quality of response is often noticeably lower than the same question in English.

Geographic bias. When asked about “typical” things — typical food, typical architecture, typical clothing — LLMs often default to Western norms. “Typical house” produces descriptions of houses common in the US or UK. “Traditional music” trends toward Western musical traditions.

Recency bias. The internet contains proportionally more content from recent years. LLMs may know more detail about recent history than older history, even when the older events are more important.

Here’s why this matters beyond fairness: bias also affects accuracy. If an LLM has learned that certain groups of people or certain types of activities are associated with certain outcomes — even if those associations are wrong — it will produce those associations in its outputs. Users who don’t know to look for bias may accept biased outputs as neutral facts.

Why LLMs Are Terrible at Maths and Counting

Here’s something that surprises almost everyone the first time they see it: LLMs — these incredibly impressive text generators — are surprisingly bad at basic maths and counting tasks. Ask ChatGPT what 847 × 239 equals and it might give you the wrong answer confidently. Ask it to count the number of R’s in “strawberry” and many versions get it wrong (there are three: st-R-awbe-RR-y).

Why? Because maths isn’t prediction. 847 × 239 has exactly one correct answer. That answer isn’t determined by what numbers usually follow multiplication problems in text — it’s determined by mathematical rules. An LLM that learned from text has seen many multiplication problems and their answers, but it hasn’t learned the underlying arithmetic rules. It learned patterns that often produce right answers, but when the specific calculation doesn’t match a strong pattern, it generates a wrong answer with the same confidence as a right one.

Similarly for counting: from Day 2, you know the LLM reads tokens, not individual letters. “Strawberry” might be one or two tokens depending on the tokeniser. If the LLM can’t see individual letters directly — only chunks — counting specific letters requires reasoning the LLM isn’t reliably doing.

Where this matters in practice: any task requiring precise numerical reasoning. Complex financial calculations. Statistical analysis. Specific date arithmetic (“how many days between March 15 and November 3?”). These are tasks where the LLM’s output looks as confident as its reliable outputs — but the actual calculation may be wrong.

Modern LLMs handle this partly by using external tools — calculator plugins, code interpreters — that can actually compute precisely. But when those tools aren’t available, or when the LLM doesn’t recognise it needs them, numerical errors happen silently.

Knowledge Gaps — What the LLM Doesn’t Know

Beyond hallucinations and bias, there are categories of information that LLMs genuinely don’t have — and handling this varies widely between models.

Private information. The LLM was trained on publicly available text. It doesn’t know your personal details, your account information, what’s on your computer, or anything about your private life — unless you tell it. Some users forget this and are surprised when an LLM doesn’t know things about them.

Very recent events. Post-training-cutoff events are genuinely unknown. Not guessable from patterns. Not inferable from context. The LLM simply wasn’t trained on information that didn’t exist yet. Some systems add web search to compensate, but the base LLM is genuinely ignorant of post-cutoff events.

Highly specialised professional knowledge. While LLMs have absorbed huge amounts of professional content, they have shallower knowledge in very specialised areas than actual domain experts. A cardiologist, a patent lawyer, or a materials chemist knows nuances that aren’t well-represented in publicly available text. LLMs in these domains produce competent-sounding text that may miss important nuances.

Local and hyperlocal information. The LLM knows a lot about major cities globally. It knows relatively little about smaller towns, local services, local laws, or regional customs — because less of that information made it into its training data. Ask an LLM about the best local restaurant in a small city and you’ll often get either clearly invented information or an honest admission of ignorance.

Five Categories to Always Verify

I want to give you a simple, memorable rule for when to trust an LLM and when to verify. I’ve distilled my experience into five categories where I always double-check, no matter how confident the LLM sounds.

Category 1 — Medical. Symptoms, diagnoses, medications, dosages, interactions, treatments. LLMs can be wrong here in ways that cause real harm. Always verify with a doctor, pharmacist, or medical authority. The LLM is great for understanding medical terminology or getting a starting point for research — not for medical decisions.

Category 2 — Legal. Laws, regulations, rights, procedures. Laws vary by jurisdiction and change over time. An LLM trained 18 months ago may not know a law that changed last year. Legal advice from a predicting machine is not legal advice. Use it to understand concepts, then verify with actual legal sources or a qualified lawyer.

Category 3 — Recent events. Anything that happened after the training cutoff. Current prices, recent news, live events, today’s weather, who won last night’s game. If recency matters, the LLM probably doesn’t know — or will hallucinate from patterns.

Category 4 — Specific citations and references. Book titles, author names, paper citations, URLs, statistics. As the court case story shows, LLMs will produce convincing-sounding references that don’t exist. Always verify that a cited source actually exists and says what the LLM claims.

Category 5 — Maths and precise data. Calculations, statistics, exact dates, specific numbers. Use a calculator, check primary sources, or use a tool with actual computational capability. Don’t trust LLM arithmetic for anything that matters.