What are evals in AI?

In plain English

When you build a product with AI, “it seemed fine when I tried it” is not good enough. An eval (short for evaluation) is a repeatable test: a collection of realistic inputs, such as 200 real customer questions, plus a way of judging the answers. Judging can be automatic (does the answer contain the right order number?), done by people (is this reply helpful and polite?), or done by another AI model acting as a grader, with people spot-checking it.

Teams run their evals before launch, and again every time they change the prompt, the model or the data, to see whether things got better or worse.

Why it matters

AI systems don’t fail like normal software. A small change, such as switching to a newer model, can quietly make some answers worse while others improve. Evals are how teams catch that before customers do. People building AI products increasingly describe good evals as the most important thing that separates a demo from a reliable product.

Evals are different from public benchmarks: a benchmark compares models on general tasks, while your evals test your product, on your data, for your users.

An example

A bank’s AI assistant answers questions about fees. The team keeps a set of 300 past questions with the correct answers. When they try a cheaper model, they run the eval: accuracy drops from 94% to 81% on questions about overseas fees, so they keep the old model for those questions.

What to watch out for

An eval is only as good as its questions: if they don’t reflect what real users ask, a high score means little. AI graders have their own biases and blind spots, so check them against human judgement. And passing your evals doesn’t prove a system is safe; keep guardrails and human review for anything high-stakes.

See where evals fit among the tools teams use in The AI tech stack.