Aiodic

Back to the table

45 Intermediate Gr Grounding Prompting

Grounding

Tying answers to supplied sources.

What it is

Grounding means tying an AI's answer to a source you can check. Instead of letting the model answer from whatever it absorbed during training, you give it the material to work from, such as a contract, a policy, a set of meeting notes, or a web page, and you tell it to answer from that material and show where each claim came from.

To see why this helps, it's worth knowing what happens without it. A language model doesn't look facts up. It learned patterns from an enormous amount of text and produces the most plausible reply, one small piece at a time. Most of the time plausible and true line up. When they don't, you get a hallucination: a confident statement that is simply wrong. Grounding changes the question the model is answering. It's no longer "what sounds right?" but "what does this document say?", which is a much easier question to get right and, crucially, a much easier one for you to check.

A useful comparison is two kinds of exam. In a closed-book exam a student answers from memory, and if memory fails they may write something that merely sounds correct. In an open-book exam the student can look at the textbook, and a good teacher asks them to write down the page number for each answer. Grounding turns the AI's work into an open-book exam with page numbers.

In practice, grounding has four parts, and you can do all of them with an ordinary chatbot today:

Grounding comes in two main flavours. With your own documents, you control the source: your price list, your book manuscript, your company handbook. With web search, the tool finds pages on the internet and answers from those, which is how chatbots answer questions about last week's news. Many business tools do the fetching for you automatically using RAG, short for retrieval-augmented generation, which searches a library of documents for the most relevant passages and hands only those to the model. RAG is a way of doing grounding at scale. Grounding itself is the simpler idea underneath: answer from the evidence, and show the evidence.

One more thing grounding is not. It isn't a guarantee. A grounded model can still misread a passage, blend two sources, or attach a citation to a sentence the source doesn't support. What grounding gives you is the means to catch those errors, because now there is something specific to check against.

Why it matters

It lowers the rate of made-up facts. When the answer is sitting in front of the model, it has far less reason to invent one. Asking for supporting quotes before the answer, and allowing "I don't know", are among the standard techniques AI companies recommend to reduce hallucinations. It won't take the rate to zero, as the examples below show, but it moves it a long way.

It makes checking fast. An ungrounded answer gives you nothing to verify except the whole claim, which means doing the research yourself. A grounded answer with quotes turns checking into a quick matching exercise: find the quoted sentence, read it, confirm it says what the AI says it says. That's the difference between an hour of fact-checking and five minutes.

It keeps answers about your world, not the world in general. A model knows a lot about refund policies in general and nothing about yours. Ask a chatbot "how long do customers have to return an item?" without grounding and you'll get a typical answer, perhaps 30 days. Ground it in your actual policy and you get your answer, 14 days, with the sentence it came from. For a business, the general answer is the dangerous one, because it sounds right.

It gets past the training cutoff. Every model stops learning at some date. Grounding it in a current document or a fresh web search lets it work with information it has never seen, such as this year's tax rules, your latest price list, or yesterday's announcement.

The stakes of skipping it are well documented. The lawyer in Mata v. Avianca who filed invented court cases, the airline held responsible in 2024 for what its chatbot told a grieving customer, and the 2025 newspaper summer reading list full of books that don't exist are all told in the Hallucination article. In each case, answering from real sources and checking them would have caught the problem.

It puts responsibility in the right place. When an AI cites its source, a person can judge the source. That's the foundation of using AI for anything that matters, and it's why most serious AI products for research, law, finance, and customer support now show their sources.

Real-world examples

Google's NotebookLM: a notebook that answers from your sources (2023 onwards)

In May 2023, Google showed an experimental note-taking tool called Project Tailwind. On 12 July 2023 it was renamed NotebookLM, the "LM" standing for language model, and opened to a small group of testers in the United States. Google's announcement described the idea directly: the model would be grounded in documents you choose, becoming a personal assistant "versed in the information relevant to you".

The design put evidence first. You uploaded your own documents, starting with Google Docs, and asked questions about them. Answers came with citations showing the most relevant quotes from your sources, so you could jump straight to the passage. Google was also frank about the limits. Its launch post said grounding reduces the risk of hallucinations but still told users to fact-check responses against their original sources.

NotebookLM went on to become one of Google's best-known AI products, and in July 2026 Google renamed it Gemini Notebook. At the time of writing it still works the same basic way, answering from the sources you add and linking each point to a passage, and there is a free version with limits on how many sources and questions you can use. Features, names, and limits change often, so check the current details yourself. The lasting lesson is the design choice: a tool built around "show me where that came from" teaches its users to check, and makes checking easy.

Stanford tests the legal tools that promised no hallucinations (2024)

After the Avianca case, legal research companies began selling AI tools that answered from their own databases of real court decisions, and some marketed them as avoiding or eliminating hallucinations. A team from Stanford's RegLab and Human-Centered AI institute, led by Varun Magesh and Faiz Surani with Matthew Dahl, Mirac Suzgun, Christopher D. Manning, and Daniel E. Ho, decided to test the claim. Their paper, "Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools", appeared in May 2024 and was published in the Journal of Empirical Legal Studies in 2025.

They wrote more than 200 legal questions and put them to Lexis+ AI from LexisNexis, and to Westlaw AI-Assisted Research and Ask Practical Law AI from Thomson Reuters, with GPT-4 as a comparison. The grounded tools did better than GPT-4 on its own. They still produced incorrect or misleading information on roughly one question in six, and Westlaw's tool on about one in three.

The most useful part for anyone using AI is the distinction the researchers drew between two kinds of error. An answer can be incorrect, getting the law wrong. Or it can be misgrounded: the statement is correct, but the source cited for it doesn't actually support it. The researchers warned that misgrounded answers may be the more dangerous kind, because a real citation makes a reader relax. They also traced where the errors came from, including the search step pulling back the wrong cases and the tools going along with false assumptions built into a question. Grounding improved the tools. It did not make them safe to trust without reading the sources.

Anthropic builds citations into the model's answers (2025)

On 23 January 2025, Anthropic launched a feature called Citations for developers building on its Claude models. A developer supplies documents, such as PDFs or plain text, and the model's answer comes back with each claim linked to the exact sentences it relied on, including the page number or character position in the source.

Anthropic said its internal testing found the built-in feature improved recall accuracy by up to 15 percent compared with citation systems developers had built themselves with prompts. It also quoted early customers. Endex, a company building an AI agent for financial research, said that switching to Citations cut its source hallucinations and formatting problems from 10 percent to zero, and increased the number of references per response by 20 percent. Those are the companies' own figures, not independent tests.

The technical documentation makes a careful promise that's worth understanding. Because the system pulls the quoted text straight out of your document, the citations are guaranteed to point at real passages. That rules out the invented-source problem entirely. It does not guarantee that the passage says what the answer claims, which is exactly the "misgrounded" error Stanford found. A real quote in the wrong place is still a mistake, so the reader's job is to read the quote, not just see that one exists.

Measuring how often models add things to a summary (2023 onwards)

In November 2023, the AI company Vectara published a public leaderboard measuring something very specific: when a model is given a short document and asked to summarise it, how often does the summary include something that isn't in the document? That's grounding failure in its purest form, since the source is right there. Vectara used its own automated checker, which it released publicly, to score each summary. In the first version, GPT-4 did best with a hallucination rate of about 3 percent.

In November 2025 Vectara rebuilt the test with more than 7,700 articles, some much longer and more complex. Several well-known reasoning models, the kind that think through problems step by step, did worse than expected, with rates above 10 percent. As of 22 September 2026, the leaderboard's rates ranged from 1.8 percent for the best model to 24.2 percent for the worst. The figures come from one company's automated checker and change with every update, so treat them as a rough guide and look at the live table for current numbers.

The takeaway is simple. Even when a model has the source in front of it and only has to summarise it, it sometimes adds things. Grounding shrinks the problem. Checking finishes the job.

An everyday scenario: the holiday-let house rules

Picture Priya, who runs two holiday cottages and answers the same guest questions every week. She pastes her house-rules document, eight pages long, into a chatbot and asks it to draft replies. At first she simply asks "Can guests bring a dog?" and gets a friendly yes with a £25 fee. It sounds right. It's wrong. Her rules allow dogs only in the smaller cottage, and there's no fee at all.

So she changes her prompt. She tells the chatbot to answer only from the house rules, quote the sentence each answer relies on, and write "Not covered in the house rules" when it can't find one. The next answer quotes her actual sentence about the smaller cottage. When a guest asks about early check-in, which her rules never mention, the chatbot says so instead of inventing a time. Priya still reads every reply before sending it, but now she's checking one quoted sentence rather than her whole document.

Try it yourself

You'll need any chatbot that lets you paste text or upload a file (ChatGPT, Claude, Gemini, Copilot, or similar), and a document you know well: a policy, a set of instructions, an article, a chapter, or a long email thread. Pick something you can judge, because the point is to catch the model out.

  1. Ask without grounding first. Open a new chat and ask a question that your document answers but the model couldn't know, such as "What's the cancellation fee for [your business or product]?" Note the answer. It will often be a confident, generic guess.
  2. Paste the document and restrict the answers. Open a fresh chat and use the prompt below. Put your document between the tags. The tags, a habit borrowed from XML tags, keep the document clearly separate from your instructions.
Answer my questions using ONLY the document below.
Do not use any outside knowledge, even if you think you know the answer.

For each answer:
1. First give the exact quote (or quotes) from the document that support it, word for word, in quotation marks.
2. Then give your answer in one or two sentences.

If the document does not contain the answer, reply exactly:
"Not in the document." Do not guess.

<document>
[paste your document here]
</document>

My first question is: [your question]
  1. Ask three kinds of question. One that the document answers directly. One whose answer needs two parts of the document put together. And one that the document doesn't answer at all, even though it sounds like it should, such as a detail that's plausible but missing. The third is the real test. A well-grounded answer says "Not in the document". A poorly grounded one invents something reasonable.
  2. Check every quote. Use your browser's or editor's find function (Ctrl+F, or Cmd+F on a Mac) to search your original document for each quoted phrase. Confirm two things: the words really appear, and they really support the answer. The second check matters most, because a genuine quote attached to the wrong claim is the error that gets past people.
  3. Try a false premise. Ask something that assumes a fact your document contradicts, such as "Why does the policy charge a fee for dogs?" when it doesn't. See whether the model corrects you from the document or plays along.
  4. Compare with web grounding. If your chatbot has a web search setting, turn it on and ask a current question, such as a recent change to a rule in your field. Click every link it gives you. Check that each page exists, is a reliable source, and actually says what the answer claims. Notice the difference: with your own document you chose the source, while with web search the tool chose it, and part of your checking is judging whether it chose well.
  5. Save the prompt. If the grounded version worked, keep it as a template for any task where the facts have to come from a specific document.

For long documents, a two-step version works better. First ask only for the relevant quotes, then ask for the answer based on those quotes alone:

Step 1: From the document below, copy out word for word every sentence
that is relevant to this question: [your question]
Number each quote. Do not answer the question yet.
If nothing is relevant, say "No relevant passages."

<document>
[paste your document here]
</document>

Then, in the same chat:

Step 2: Using only the numbered quotes above, answer the question.
After each sentence of your answer, put the quote number it relies on in brackets, like [2].
If the quotes don't fully answer it, say what is missing.

Common mistakes and misunderstandings

"I gave it the document, so it's using the document." Supplying a source isn't the same as restricting the answer to it. Unless you say "only from this document", the model blends the document with its general knowledge, and you can't tell which parts came from where. Say it explicitly.

Leaving no way out. If the instructions only say "answer from the document", a model that can't find the answer is under pressure to produce one anyway. Always name the words it should use when the answer is missing. Giving the model permission to say "not in the document" is one of the simplest and most effective grounding instructions there is.

Trusting that a citation exists instead of reading it. This is the misgrounded error from the Stanford study, and it's the most common way grounding fails in practice. The quote is real, the link works, and the page doesn't say what the answer claims. A citation only earns trust once you've read the passage.

Assuming web search is the same as a reliable source. Grounding in web results is only as good as the pages the tool picked. It may rely on an out-of-date page, a forum post, or a content farm. With your own documents you vouch for the source. With web search you have to judge it, which is why the Web Search article matters as much as this one.

Pasting in more than the model can handle well. Very long documents stretch the model's attention, and details in the middle of a long text are more easily missed. If your source is huge, use the two-step quote-first method, split it into sections, or use a tool built for large libraries. The Context Window article explains why length matters.

Grounding in the wrong document. The model will faithfully answer from an old price list if that's what you give it. Grounding moves the question from "is the AI right?" to "is my source right and current?" Keep your source documents up to date and name them clearly, with version dates.

Thinking RAG or "trained on our data" means grounded. A company tool that says it uses your data might be fetching passages and citing them, which is grounding, or might have been adjusted by fine-tuning, which changes the model's habits but gives you nothing to check. Ask whether answers show their sources. If they don't, you can't verify them.

Treating grounding as a fix for every task. Grounding helps when there's a right answer in a source. It does little for brainstorming, drafting fiction, or creative work, where you want the model's imagination. Use it where facts matter, and leave it off where invention is the point.

Forgetting that documents can contain instructions. If you ground a model in text from the internet or from other people, that text might include sentences written to steer the AI, such as "ignore previous instructions". This is called prompt injection, and it's a reason to be careful about grounding an AI that can take actions in untrusted material. Guardrails covers the defences.

Skipping the final read. Grounding makes a human check quick. It doesn't make it optional. For anything that goes to a customer, a client, or print, keep a human in the loop.

Related elements