Hallucination

A hallucination is output from a large language model that is fluent, confident, and wrong — or at least unsupported by any real evidence. It can take the form of an invented citation, a quote attributed to someone who never said it, a function that doesn't exist in your codebase, or a precise-sounding number with no basis in the data. The danger is that hallucinations rarely look like errors; they read exactly like the model's correct answers.

It happens because a model generates language by predicting the most likely next token, not by consulting a verified record of facts. When the answer is well-represented in training data, that prediction is usually right. When the question is obscure, ambiguous, or outside what the model learned, it still produces a smooth continuation — and that gap is where fabrication creeps in. This matters acutely for autonomous systems: an AI agent that acts on a hallucinated fact can take a wrong action, not just give a wrong answer.

Consider a support agent asked about a refund policy that isn't in its training data. Left to memory, it might confidently invent a "30-day window" that doesn't exist. The fix is grounding: with retrieval-augmented generation, the agent fetches the actual policy document and answers only from that text, while guardrails instruct it to cite the source or reply "I don't know" when nothing relevant is found. That combination — ground the answer, then verify it — is the practical defense against hallucination.