Evaluation-first: why AI products should be tested before they are built
Write the evaluation set before you build an AI feature. It forces agreement on what good looks like, turns quality debates into numbers, and catches regressions whenever a prompt or model changes. Start with 50 real examples and add every production failure back in.
Last verified24 Sep 2026
Placeholder article. Replace this text with your own. Keep the structure: a BLUF in the frontmatter, then sections that each open with a direct answer.
What is an evaluation set?
An evaluation set is a list of real inputs paired with the outputs you expect or the qualities a good output must have. It is the AI equivalent of a test suite.
Unlike unit tests, evaluations rarely pass at 100%. The team agrees a threshold, for example 92% of responses graded acceptable, and treats that as the launch bar.
Why build it before the feature?
Building the evaluation set first forces product, engineering and domain experts to agree what "good" means before anyone is attached to an implementation.
- It exposes disagreements early, while they are cheap.
- It gives the team a number to improve rather than opinions to argue about.
- It becomes a regression gate for every later prompt or model change.
How to pick the first 50 examples
Pick examples from real usage where possible, and deliberately include the awkward ones.
| Share | Type of example | Why |
|---|---|---|
| 60% | Typical requests | Measures everyday quality |
| 25% | Edge cases | Finds where the feature breaks |
| 15% | Should-refuse cases | Checks the guardrails |
How to grade
Use the cheapest grader that is reliable for each check:
- Exact checks for format, length and required fields.
- Model-based graders with a written rubric for qualities such as relevance and tone.
- Human review on a sample, to check the graders themselves.
Key takeaway
If you cannot say how you will measure an AI feature, you are not ready to build it.