Confidence is not knowledge
An LLM has no built-in sense of "I don't know." Its whole job is to produce a plausible next word. Ask it about something it never saw during training — a niche topic, last week's event, your company's internal policy — and it will still generate a smooth, authoritative answer. Sometimes that answer is invented. That's a hallucination.
The exam analogy
Imagine a student taking a closed-book exam on a subject they only half-studied. They won't leave blanks — they'll write something confident to fill the space. Now hand the same student the textbook and let them look up the answer before writing. The confident-but-wrong answers mostly disappear. RAG is handing the model the textbook.
The three gaps that cause hallucination
- Frozen knowledge: the model can't know anything after its training cutoff.
- No private data: it never saw your documents, your product, or your customer records.
- No source of truth: it can't tell the difference between a fact it learned and a pattern it's improvising.
The fix, in one sentence
Instead of asking the model to remember everything, we let it retrieve the right information at the moment of the question — and that requires a way to find relevant text by meaning. To do that, we first turn words into numbers. That's the next lesson.