Aiodic

Back to the table

10 Beginner Ha Hallucination Safety & Ethics

Hallucination

Confident answers that are false.

What it is

A hallucination is when an AI states something false as if it were true. It might invent a fact, a quote, a statistic, a book, a court case, a web link, or a feature your software doesn't have. The part that catches people out is the delivery. A hallucinated answer arrives in the same calm, fluent, well-organised voice as a correct one. Nothing in the wording tells you which is which.

To understand why this happens, you need one idea about how a language model works. A chatbot is not looking things up in a database of facts. It was trained by reading an enormous amount of text, and what it learned is patterns: which words tend to follow which other words. When you ask a question, it builds a reply one small piece at a time (the pieces are called tokens), each time choosing what is likely to come next. You can think of it as the autocomplete on your phone, grown enormously more capable.

Most of the time, the most likely next words are also the true ones, because the model has seen the truth written down many thousands of times. Ask who wrote Pride and Prejudice and "Jane Austen" is overwhelmingly the likeliest continuation. But ask about something it saw rarely or never, and the machinery doesn't stop. It still produces the most plausible-sounding text. It knows very well what a legal citation looks like, what a scientific reference looks like, and what a biography sounds like. So it writes one. The shape is perfect and the content is made up.

There is no separate alarm inside the model that rings when it leaves solid ground. Picture a well-read friend who is wonderful company and has one flaw: they can never bring themselves to say "I don't know." Most of what they tell you is right. Now and then they fill a gap with something that merely sounds right, and they say it with exactly the same confidence.

Training can make this worse. In September 2025, researchers from OpenAI and Georgia Tech argued that models are taught and tested in ways that reward guessing. Think of a multiple-choice exam where a blank answer scores zero and a guess might score a point. A student who always guesses will outscore one who honestly leaves blanks. Models have been graded in much the same way for years, so they learned to be good guessers.

Hallucinations tend to come in a few recognisable kinds:

A note on the word itself. "Hallucination" is a metaphor. The AI isn't seeing things, and it isn't lying, because lying requires knowing the truth and intending to deceive. Some researchers prefer the word "confabulation", which describes a person who fills gaps in memory with invented details without realising it. That is closer to what is going on. But "hallucination" is the name that stuck.

Why it matters

The cost of an error lands on you, not on the AI. If you publish a blog post with an invented quote, your name is on it. If your business chatbot gives a customer wrong information, your business answers for it. One of the examples below is a court ruling that says exactly that. The AI faces no consequences. The person who pressed "publish" does.

It hides in the places where you most want help. Hallucinations are most likely with specific details: exact figures, dates, names, quotations, citations, legal rules, medical doses, prices, and how-to steps for a particular product. Those are precisely the things people ask an AI for, because they are tedious to look up.

Some situations make it much more likely. Expect more hallucinations when you ask about:

Better writing makes it harder to spot. Early chatbots made clumsy mistakes. Today's models write so smoothly that a false sentence reads exactly like a true one. The rate of hallucination has fallen a good deal, but nobody has got it to zero, and the errors that remain are harder to see.

It should shape how you use AI. Once you understand hallucination, you stop treating a chatbot as an oracle and start treating it as a quick, knowledgeable assistant whose work needs checking when it matters. That one change in attitude prevents nearly every AI disaster you read about in the news.

Real-world examples

The lawyer and the six cases that never existed

In 2023, a New York lawyer named Steven Schwartz was preparing a filing in Mata v. Avianca, a case about a passenger who said he was injured by a serving cart on a flight. Schwartz used ChatGPT to help with legal research. The brief he wrote, which a colleague at his firm signed and filed, cited six earlier court decisions in support of the argument. None of them existed. ChatGPT had invented the case names, the courts, the quotations, and the reference numbers, all in flawless legal style.

When the other side's lawyers couldn't find the cases, Schwartz went back to ChatGPT and asked whether they were real. It told him they were, and that they could be found in reputable legal databases such as LexisNexis and Westlaw. On 22 June 2023, Judge P. Kevin Castel fined Schwartz, his colleague Peter LoDuca, and their firm $5,000. In a sworn statement, Schwartz said he had been unaware that ChatGPT's content could be false. It became the best-known hallucination story in the world, and similar cases have continued to turn up in courts ever since.

The airline that had to honour its chatbot's promise

In November 2022, Jake Moffatt's grandmother died, and he went to Air Canada's website to book a flight to the funeral. He asked the site's chatbot about bereavement fares, which are discounts for people travelling because of a death in the family. The chatbot told him he could buy a full-price ticket now and apply for the discount within 90 days of the ticket being issued. The airline's real policy said the opposite: the discount could not be claimed after travel. When Moffatt applied, Air Canada refused.

He took the airline to a small-claims tribunal in British Columbia. Air Canada argued, remarkably, that the chatbot was responsible for its own words. In February 2024 the tribunal rejected that and ordered the airline to pay Moffatt 650.88 Canadian dollars in damages, which came to 812.02 dollars with interest and fees. The sum was small. The principle is large, and every entrepreneur should know it: if your AI says it to a customer, you said it.

A hundred-billion-dollar demo

In February 2023, Google introduced its chatbot Bard (now called Gemini) with a short promotional demo. One sample question asked what new discoveries from the James Webb Space Telescope a parent could share with a nine-year-old. One of Bard's answers said the telescope had taken the very first pictures of a planet outside our solar system. Astronomers quickly pointed out that the first such picture was taken in 2004 by a telescope in Chile, long before Webb was launched. The error was in Google's own advertisement. Shares in Google's parent company, Alphabet, fell by nearly 8 percent that day, wiping roughly 100 billion dollars off its value. Even the companies that build these systems get caught by them.

The summer reading list of books nobody wrote

In May 2025, the Chicago Sun-Times and at least one other major newspaper, the Philadelphia Inquirer, ran a syndicated summer reading list of fifteen books. Readers soon noticed that ten of the fifteen didn't exist. The authors were real and well known, including Isabel Allende and Andy Weir. The titles, such as Allende's "Tidewater Dreams" and Weir's "The Last Algorithm", and the tempting plot summaries were invented. The freelance writer who produced the list admitted he had used AI and hadn't checked the output. For anyone who writes or publishes, this is the example to remember: real names attached to made-up details is the classic fingerprint of a hallucination, and it is convincing precisely because half of it is true.

Try it yourself

First you'll watch a hallucination happen somewhere harmless. Then you'll practise the checking habit that protects you.

  1. Ask about something obscure that you know well. Try: "Write a short history of [your street, your small home town, or your own small business]." Or ask for a biography of yourself by full name. If your chatbot has a web search setting, turn it off for this test. Read the result and count the confident details that are simply wrong.
  2. Ask for sources. Try: "Give me five academic studies on whether listening to music helps concentration. Include the authors, year, and journal for each." Then search the web for each title. See how many exist exactly as described. Watch for real authors attached to papers they never wrote.
  3. Slip in a false idea. Ask something like: "Why did Marie Curie refuse her second Nobel Prize?" (She accepted it.) Many current models will correct you. Some will play along. Either way, you'll learn how your chatbot behaves when the question itself is wrong.
  4. Repeat step 2 with web search turned on. The answers should improve, because the model is now reading real pages instead of relying on memory. Click each link anyway. Check that the page exists and that it says what the AI claims it says.
  5. Ask twice. Open a new chat and ask your original question again. If specific details change between the two answers, such as different dates or different names, treat those details as unreliable. Facts the model really knows tend to stay the same. Made-up ones drift.
  6. Build the habit. From now on, before you use or publish anything from an AI, check every name, number, date, quotation, and source against something outside the chat. The rule of thumb: the more specific the detail, and the more it would hurt if it were wrong, the more it needs checking.

Here is a prompt you can save for any task where the facts matter:

Answer using only information you are confident about.
If you are not sure about something, say "I'm not sure" instead of guessing.
Do not invent sources, quotes, links, or statistics.
At the end, list the specific claims in your answer that I should double-check.

This won't prevent hallucinations completely, but giving the model clear permission to say "I don't know" lowers the rate, and the list at the end tells you where to look first.

Common mistakes and misunderstandings

"It sounds sure of itself, so it must be right." Confidence is a writing style, not a measure of accuracy. The model uses the same assured tone for everything. Learn to ignore the tone and look at the claim.

"I'll just ask it whether that's true." This is exactly what Steven Schwartz did, and ChatGPT assured him the fake cases were real. Asking "are you sure?" can fail in two directions. The model may defend an invented answer, or it may abandon a correct one because you sounded doubtful. Real checking has to come from outside the conversation: a search engine, the original document, an official website, or a person who knows.

"Web search fixes the problem." It helps a great deal, because the model works from real pages instead of memory. Connecting an AI to trusted sources like this is called grounding. But the model can still misread a page, mix up two sources, or attach a link to a claim that the page doesn't actually make. Click through.

"The newest models don't do this any more." Rates have dropped, and they are still not zero. In 2025, OpenAI's own published test results showed that two of its newer reasoning models, o3 and o4-mini, hallucinated at least twice as often as the older o1 on a test of facts about real people (33 and 48 percent of the time, against 16 percent). Newer doesn't automatically mean more truthful.

"A citation proves it." Formatting is the easiest thing in the world for a model to get right. A reference with authors, a year, a journal name, and page numbers can be invented from start to finish. A citation only counts once you've found the source yourself.

"A hallucination is the AI lying." Lying requires knowing the truth and intending to mislead. A model has no such intent. It is also worth keeping hallucination separate from two neighbours. Bias is a systematic slant learned from training data. Out-of-date information is a true fact that has since changed. Each has a different fix.

"Turning the temperature down stops it." Temperature is a setting that controls how much randomness goes into the model's word choices. Lowering it makes answers more consistent, but a model can be consistently wrong. It will tell you the same false thing every time.

"Making things up is always bad." The ability to produce plausible things that don't exist is the same ability that writes your short story, names your product, and brainstorms your marketing angles. In creative work, invention is the point. It only becomes a problem when you need facts. Know which kind of task you're doing.

Publishing without reading. Most public AI embarrassments share one root cause: nobody checked before it went out. For anything that carries your name, keep a human in the loop, meaning a person who reviews the AI's work before it is used.

Related elements