Why AI Makes Things Up: Hallucinations Explained

In May 2025, researchers asked an AI for one man's birthday three times. It gave three different dates, and all three were wrong. This 500x Explained shows how that happens, why it happens less often now, and the habit that catches what's left, drawn entirely in code with the same dice and odds bars as "How an AI picks its next word".

Three dates, all wrong

In the paper "Why Language Models Hallucinate" (Kalai, Nachum, Vempala and Zhang, September 2025), the authors asked DeepSeek-V3, a state-of-the-art open-source model, in the DeepSeek app on May 11, 2025: "What is Adam Tauman Kalai's birthday? If you know, just respond with DD-MM." On three separate attempts it answered "03-07", "15-06" and "01-01", which is July 3, June 15 and Jan 1. All three are wrong, and it answered even though the prompt said to answer only if it knew.

OpenAI calls hallucinations "plausible but false statements generated by language models." Its help center lists the usual forms: incorrect dates and facts, fabricated quotes, studies and citations, and overconfident answers.

How it happens, part 1: a likely word

A language model learns by predicting the next word in huge amounts of text. That text carries no true or false labels, so the model learns what fluent language usually says next.

For a fact that shows up all the time, like Einstein's birthday (March 14), one answer clearly wins, and the paper notes that models seldom get facts like that wrong. A rare fact is different. Some birthdays appear in the training data once, and nothing in the patterns of language points to the right date. In the video, that's the moment the bars go flat. The model still picks an answer, and it comes out in the same confident tone as a fact it knows well. (The bar heights are an illustration made for this video; a real model's odds will differ.)

How it happens, part 2: the scoreboard

Why not just say "I don't know"? Most evaluations grade like a multiple-choice exam: a right answer gets a point, and a wrong answer or "I don't know" gets zero. A guess at a birthday has a 1-in-365 chance of scoring, while "I don't know" never does. Over thousands of questions the guesser looks better on the leaderboard, and models tuned for those scores learn to guess. The authors argue the fix is to change how the main benchmarks are scored, so confident errors cost more than honest uncertainty.

Why it's rarer now

Hallucinations are less common than they were when chatbots answered from memory alone. They still happen. Here's the picture:

• Chatbots can look things up. ChatGPT search (October 2024), Claude web search (March 2025) and Google's grounding with Google Search for Gemini developers can pull in web pages and cite them, so the answer can come straight from a source. Google says grounding reduces hallucinations, and OpenAI's help center lists search among the tools that "can improve factual accuracy."
• Newer models say "I don't know" more. In OpenAI's own SimpleQA comparison without web search (reported in its September 2025 post), gpt-5-thinking-mini declined to answer 52% of the questions, against 1% for the older o4-mini, and gave a wrong answer 26% of the time against 75%. Its accuracy was about the same (22% vs 24%). These are OpenAI's numbers from its own test.
• It's still there. OpenAI's post says hallucinations "remain a fundamental challenge for all large language models," and its help center still lists fabricated citations and wrong dates. Progress also zigzags: Anthropic's July 2026 system card for Claude Opus 5 reports that it hallucinates factual claims slightly more than Opus 4.8, even though it's more accurate overall. Rare facts, niche numbers, quotes and citations are where it still bites.

Try this

When it matters, turn on search or paste the source, then ask for exact quotes and check them against the source. Anthropic's guide to reducing hallucinations also suggests telling the model it's fine to say "I don't know," which cuts down on made-up answers.

Rarer now. Still possible.