What it is
An eval is a test for an AI prompt. You collect a set of real examples of the job you want the AI to do, decide what a good answer looks like for each one, and then run the prompt against all of them at once. Instead of trying one example and forming an opinion, you get a score: this version of the prompt got 17 out of 20 right, and the new version got 19.
The word is short for "evaluation", and you'll see it in the singular ("we wrote an eval for that") and the plural ("our evals are green"). AI companies use the same word for the big public tests that compare one model against another, but for a creator or a small business the meaning is more modest and more useful. An eval is your own private exam, written for your own task, in your own words.
Here's why the idea exists at all. When you change a normal piece of software, it either works or it breaks, and you can tell which. Language models don't behave like that. They're probabilistic, which is the technical way of saying they don't give exactly the same answer twice. Ask a chatbot to write a product description, get something good, and you've learned almost nothing about whether the next one will be good too. Change a word in your prompt, and the reply might improve on the example you're looking at while quietly getting worse on five others you didn't try.
People who use AI a lot develop a feel for this, and they tend to work by what practitioners call "vibe checks": run it a few times, squint at the results, decide it seems better. Vibe checks are fine for a one-off. They fall apart the moment something matters, such as a prompt that will run a thousand times a week inside your business, or a change you need to defend to a client.
An eval replaces the vibe check with a small, repeatable experiment. It has three parts:
- A set of cases. Real inputs, ideally taken from actual use rather than invented. For a customer-service prompt, that means twenty real customer messages. For a headline prompt, twenty real articles.
- A definition of good. For each case, either the exact answer you expect, or a short list of qualities the answer must have ("mentions the refund window", "under 40 words", "does not promise a delivery date").
- A way to grade. Sometimes a computer can check it (is the word there, is it under the limit, is the format valid). Sometimes a person reads it. Increasingly, a second AI does the reading, with a person checking the grader's work.
Run the prompt over the whole set, count the passes, and you have a number. Change the prompt, run it again, and you know whether you made things better, worse, or only different.
It doesn't have to be a spreadsheet, though a spreadsheet is a perfectly good place to start. Twenty rows, one input per row, a column for the AI's answer and a column for your verdict, is an eval. It's the discipline that matters, not the tooling.
Why it matters
It's the only way to know whether a prompt change helped. Without an eval, prompt writing is superstition. You add "be concise", the next output happens to be shorter, and you conclude it worked. With an eval you see that it was shorter on eleven cases, longer on four, and now leaves out the price on three. Every serious guide to prompting ends up saying the same thing: the people who get reliable results are the ones who measure.
It stops you breaking things you already fixed. This is called a regression, and it is the most common way AI projects go wrong. In March you tuned the prompt so it stops inventing discount codes. In June you tweak it for tone, and the discount codes come back, because nobody re-checked March's problem. An eval set that includes the March cases catches that in seconds. Anthropic's guidance for building evals makes the same point: they surface problems and changed behaviour before users meet them.
It protects you when the model changes. The AI companies update their models regularly, sometimes silently, and a prompt that worked beautifully on one version can wobble on the next. An eval you can re-run in a minute tells you whether the new model still handles your job. Without one, you find out from a customer.
It turns "the AI is bad at this" into something you can act on. A score of 14 out of 20 comes with a list of the six failures, and reading six failures side by side almost always reveals a pattern: it fails on long inputs, or on questions with two parts, or whenever the customer is angry. Now you know what to fix. That feedback loop, from failures to a better prompt to a re-run, is the whole craft of iterative refinement.
It's what turns AI from a toy into a tool. Hamel Husain, a machine-learning engineer who has advised many companies on their AI products, wrote in 2024 that the unsuccessful ones almost always share one root cause: no evaluation system. That matches the experience of most people who have shipped anything with a language model inside it. The prompt is the easy part. Knowing whether it works is the job.
The honest cost is time. Building twenty good cases with clear expected answers takes an afternoon, and it's an afternoon of tedious work that nobody enjoys. It pays back the first time it stops you shipping something broken.
Real-world examples
OpenAI opens up its own test kit
On 14 March 2023, the day OpenAI announced GPT-4, it also released a piece of software called Evals as open source, meaning anyone can read and use the code. It was the framework OpenAI used internally to test its models: a way to write a test once, run it against a model, and inspect the results one sample at a time. OpenAI said it hoped the tool would become "a vehicle to share and crowdsource benchmarks", and invited the public to contribute tests of things the models got wrong.
That's an unusual thing for a company to do, and the reasoning behind it is the whole argument for evals. A model is only as trustworthy as the tests you've run on it, and a company can't think of every way its model might fail. So it asked the world to write the tests. The project is still public today as a registry of evals contributed by many people, and OpenAI describes it as the tool it uses to spot shortcomings and prevent regressions. If the company that builds the model needs evals to trust it, you do too.
Chatbot Arena and the question of who grades the grader
In May 2023, researchers at LMSYS, a group spanning UC Berkeley and other universities, launched Chatbot Arena. The idea is simple. You type a question, two anonymous models answer side by side, and you vote for the better one. Only afterwards do you learn which models they were. Millions of votes later, the site publishes a leaderboard using the Elo rating system borrowed from chess. It became the most-watched public scoreboard in AI, because it measures what people actually prefer rather than what a fixed test can check.
The same team then asked a question that matters for anyone building evals: could an AI grade the answers instead of a human? In their paper "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena", published in June 2023, they found that a strong model used as a judge agreed with human raters over 80 percent of the time, which was about as often as the humans agreed with each other. That result launched the practice now called "LLM as a judge": using one model to grade another, so that an eval of 500 cases can run in minutes instead of days. It comes with a caveat the paper itself raised. The judging model has biases, such as preferring longer answers and favouring the first of two options, so its grades still need a person checking a sample.
The Arena story has a second chapter worth knowing. In April 2025, researchers led by Cohere Labs published "The Leaderboard Illusion", showing that some large companies had tested many private versions of a model on the Arena and published only the best one; by the paper's count, Meta tested 27 variants before releasing Llama 4. The Arena team disputed the framing and said its rules allowed any company to test as many variants as it liked. Whichever side you take, the lesson for your own evals is the same: a score means little unless you know how it was produced.
The researchers who discovered that grading changes what you want
In 2024, Shreya Shankar and colleagues at UC Berkeley studied what happens when people try to write evals with an AI's help. They built a tool called EvalGen that drafts grading criteria and suggests tests, then watched people use it. The paper, "Who Validates the Validators?", won attention for a finding they named criteria drift: people need criteria to grade outputs, but grading the outputs is what teaches them what their criteria should be. Someone sets out to check "is the summary accurate", reads twenty summaries, and realises that what they care about is "does it lead with the decision the reader has to make".
The practical lesson is that you cannot write a perfect eval in advance and hand it off. You write a rough one, grade a batch by hand, notice what you're really reacting to, and rewrite the criteria. Anyone who has ever marked a stack of homework will recognise this. It also means that fully automating evaluation, with no human ever looking at outputs, is a mistake. The looking is where the criteria come from.
An everyday case: the newsletter subject line
Imagine Priya, who runs a paid newsletter for garden designers and uses a chatbot to draft subject lines. Some weeks the lines are great and some weeks they're generic, and she's been tweaking the prompt on gut feel for months without being sure it's getting better.
She builds an eval in a spreadsheet in one evening. Column A holds the opening paragraphs of her last twenty issues. Column B holds the subject line she actually used, or the one she wishes she had. Column C is her rule for a pass: specific to that issue, under 50 characters, no exclamation marks, and no phrase she'd be embarrassed to send. She runs her current prompt across all twenty and grades them: 9 pass. Then she tries the change she's been considering, adding three of her best past subject lines to the prompt as examples (a technique called few-shot prompting). She re-runs: 16 pass. The next week she tries "make it punchy", which she'd assumed would help, and drops to 12, because "punchy" makes the model reach for exclamation marks. She keeps the examples, drops "punchy", and stops arguing with herself. The spreadsheet settled it.
Try it yourself
You need a task you do repeatedly with AI, a chatbot or AI tool, and a spreadsheet. Allow an hour the first time.
- Pick one task. Something you'd like to hand to the AI regularly: replying to a common type of email, writing product descriptions, summarising meeting notes, tagging support tickets. One task, not several.
- Gather ten real inputs. Pull them from your actual inbox, catalogue, or notes. Real inputs are messier than invented ones, and the mess is what the eval needs to catch. Include at least two awkward cases: an input that's unusually long, one that's missing information, or one from a customer who's annoyed.
- Write down what good looks like for each. For some, that's an exact answer. For most, it's two or three qualities the answer must have. Write them as yes-or-no checks: "mentions the return period", "under 80 words", "doesn't apologise more than once". Vague criteria like "sounds professional" are hard to grade; keep pushing until each one is something you could mark in ten seconds.
- Run your current prompt on all ten. Paste each input into a fresh chat with the same prompt, and copy the answer into the spreadsheet. Don't fix anything yet.
- Grade them honestly. Pass or fail against your checks. Write the score at the top: 6 out of 10, say. Read the failures together and look for the pattern.
- Change one thing. Based on the pattern, make a single change to the prompt: add an example, add a rule, remove an instruction that's causing trouble. Run all ten again and grade again. One change at a time is the rule, because if you change three things and the score moves, you don't know which one did it.
- Keep the cases. Every time the AI fails on something new in real use, add that input to the sheet. Over a few months your ten cases become forty, and forty real failures is a better description of your task than any instruction you could write.
- When the set gets big, let an AI grade it. Paste the answer and your checks into a chatbot and ask it to mark pass or fail with a one-line reason. Then check a handful of its verdicts yourself, because a grader that's wrong 10 percent of the time will mislead you if you never look.
A grading prompt you can save:
Here is a task, an AI's answer, and a list of checks.
For each check, reply PASS or FAIL with a one-sentence reason.
Do not give credit for effort or style. Judge only the checks.
Task: [the original input]
Answer: [the AI's answer]
Checks:
1. [your first check]
2. [your second check]
3. [your third check]
Common mistakes and misunderstandings
Testing on made-up examples. Invented inputs are tidy, and the model handles tidy inputs well. Real customers write in fragments, ask two things at once, and include details that don't matter. If your eval passes on clean examples and fails in the wild, the eval was the problem.
Only including the easy cases. An eval where everything passes tells you nothing. Anthropic's guidance for building evals makes a point of listing the awkward cases to include: irrelevant or missing input, very long input, hostile or off-topic messages, and cases where even two humans would disagree. The failures are the useful part.
Changing the prompt and the cases at the same time. If you add five new cases and reword the prompt in the same week, a change in score could be either. Freeze the cases while you tune the prompt, and add new cases in a separate step.
Trusting one run. Because models vary from run to run, a single pass over ten cases can move by a point or two on its own. If a change only moves the score by one, run it again before believing it. If it matters, run each case three times.
Confusing your eval with a benchmark. The public tests you read about, such as the ones behind headlines like "model X scores 92 percent", are benchmarks: standardised exams that compare models with each other on general skills. They tell you almost nothing about whether a model will handle your task. A model that tops a coding benchmark can still write terrible subject lines for garden designers. Your eval is the only test that measures your job.
Letting the AI grader run unwatched. LLM-as-a-judge is a real time-saver, and it has known habits: it favours long answers, it can prefer whichever option came first, and it's generous to answers that sound confident. Spot-check its grades, and never let it grade its own answers to the same prompt.
Treating the criteria as finished. Criteria drift is real. Expect your definition of good to change after the first batch you grade by hand, and treat that as the process working, not as a sign you did it wrong.
Believing a good score means it's safe to stop checking. An eval is a fixed set of cases, and the world isn't. It tells you the prompt handles what you've seen so far. Keep a human in the loop for anything that reaches a customer, and keep adding the cases that slip through.
Skipping it because the task is "creative". You can't grade a poem as pass or fail, but you can grade "stays under the word count", "doesn't repeat the title", "mentions the product once", and "no clichés from this list". Most creative tasks have a mechanical layer, and the mechanical layer is where most of the failures are.
Related elements
- Prompt: the thing an eval tests.
- Iterative Refinement: the loop of run, grade, change, and run again that evals make possible.
- Few-shot: adding worked examples to a prompt, often the first change an eval shows working.
- Prompt Templates: a fixed prompt with slots for the input, which is what you're really testing when the same prompt runs many times.
- Benchmarks: the public, general-purpose cousins of evals that compare models with each other.
- Self-consistency: running the same case several times and comparing, which is also how you tell real improvement from random variation.
- Structured Output: asking for answers in a fixed format, which makes them far easier to grade automatically.
- Hallucination: the failure that evals catch most often, once you add a "no invented facts" check.
- Human in the Loop: why a passing eval is a reason to check less often, not a reason to stop.
- Observability: watching an AI system in live use, which is where the new eval cases come from.