← All insights
AI · · 5 min

A short list of evals every LLM product should have

You don't need 1,000 examples. You need 30 well-chosen ones.

A short list of evals every LLM product should have cover image

The minimum bar

For every LLM-powered feature we ship, we set up the same six eval categories before the feature goes live. Not after a customer complains. Before.

  1. Happy path. Does it work on the most common inputs, the ones that will make up 80% of real traffic? This sounds obvious and it’s the one teams actually skip, because it feels too easy to bother writing down.
  2. Edge cases. Empty input, maximum-length input, input in a language the prompt wasn’t tuned for, input that’s technically valid but weird (a support ticket that’s just a single emoji). These are where demos fall apart in front of real users.
  3. Format adherence. If the output has to be JSON, a specific markdown shape, or a fixed set of fields, does it stay that way under pressure? Models drift on format compliance as prompts get longer or temperature creeps up. We test this with malformed-looking inputs specifically designed to tempt the model off the rails.
  4. Refusal cases. Does it correctly decline the requests it shouldn’t answer, the out-of-scope ones, the ones that look like prompt injection, the ones asking it to be something it isn’t? We write these as real adversarial examples, not polite hypotheticals.
  5. Regression cases. Every bug a user found becomes a permanent eval row. The exact input that broke it, with the correct output next to it. This is the single highest-leverage eval category and the easiest one to forget to maintain.
  6. Domain golden set. 10 to 30 hand-picked examples your own team has labeled correct or incorrect, specific to your product. This is the one no vendor benchmark can give you, because it encodes judgment calls only you can make about what “good” means for your use case.

What you don’t need

You don’t need a research-paper-grade eval suite. You don’t need 10,000 examples, a custom scoring model, or a dashboard with seventeen metrics nobody looks at. Most teams that build elaborate eval infrastructure end up checking it once and never again, because the setup cost ate the time that should have gone into actually looking at outputs.

What you need is enough coverage that a prompt change, a model swap, or a new system instruction can’t ship without someone being able to say, with evidence, whether it got better or worse. Thirty well-chosen rows that run in under a minute will tell you more than a thousand generic ones that take twenty.

How we run it operationally

The eval set lives in the repo, not in a separate tool, as a plain file the team can read and edit in a normal pull request. Every PR that touches a prompt, a system message, or a model version runs the eval suite as a CI step. If the pass rate drops, the PR shows a diff: which rows flipped, from what, to what. That diff is the actual code review artifact, more useful than reading the prompt change itself.

We assign one person per feature as the owner of its golden set. Not because evals need a bureaucracy, but because if nobody owns it, nobody adds rows when something breaks, and the set rots within a month.

A worked example

One client’s support-triage agent had a golden set of 24 rows: 8 happy path, 6 edge case, 4 format, 3 refusal, 3 regression. A new prompt version passed 23 of 24, a net improvement from the previous version’s 20 of 24. The one failure was a refusal case, a message that looked like a legitimate billing question but was actually a prompt injection attempt disguised as one. We added it as a permanent regression row, fixed the system prompt, reran, 24 of 24. Total time from regression to fix: 40 minutes, because the eval set made the failure visible and specific instead of a vague “it feels worse.”

What changes when you have evals

You stop arguing about whether the model got worse. You can measure it. Disagreements that used to take a Slack thread and three people’s gut feelings turn into a one-line diff anyone can read. You ship faster, with less fear, because every change comes with its own proof.