Skip to content
Latticework
Esc
Try
No. 109AI Systems

Evals as Fitness Functions

AI behaviour is probabilistic and shifts with every model, prompt and data change, so check quality with automated evaluations on real cases, run on every change and tracked over time.

Every change to the model, prompt or data is checked automatically against real cases before it ships, like a fitness function.

To an executive

AI quality is not fixed at launch; it drifts with every model, prompt and data change. Automated evaluations show whether a change helps or hurts before customers notice.

To an engineer

Build a test set of real cases with graders: code where possible, people for calibration, a model judge for nuance. Run it on every change and track scores.

The idea

A conventional function returns the same output for the same input, so a unit test settles the question once. A language model does not. Its answers vary between runs, and they shift when the model version, the prompt, the retrieved documents or the tools change. Quality is therefore a moving measurement, not a property you check at launch.

Evals are the answer AI engineering has converged on: a set of real cases with a definition of a good result, graded automatically and run on every change. There is no single originator. Hamel Husain's 2024 essay "Your AI Product Needs Evals" argues that teams without them are guessing, and lays out levels from cheap assertions to human review and model judges. Anthropic's documentation describes the same toolkit: code-based grading, human grading and model-based grading. The idea mirrors fitness functions from evolutionary architecture, the automated checks of an architectural quality that Neal Ford and colleagues proposed: you state the quality you care about and let a machine keep checking it.

Why it works

Measurement replaces impression. A demo that looks good on five examples says little about the thousand cases in production. A test set of real cases turns "it seems better" into a score you can compare.

Running on every change catches silent regressions. A prompt tweak that fixes one complaint often breaks something else. Evals in the pipeline expose that before release, just as fitness functions catch architectural drift.

Graded at the right level, they stay affordable. Husain recommends fast assertions for what code can check, human review to define and calibrate quality, and model judges for nuance, checked against the human labels.

Where it applies

  • Every AI feature heading to production, from summarising case files to drafting customer replies.
  • Vendor and model selection, where your own test set beats public benchmarks for your domain.
  • Model and prompt upgrades, including the ones your vendor makes without asking.
  • Portfolio reviews of AI pilots, where an enterprise with dozens of superficial pilots needs evidence of which ones actually work.

How to apply

  1. Collect real cases. Take inputs from logs, tickets and documents, including the messy and the awkward, not invented examples.
  2. Define good with the people who know. Domain experts label what a correct, acceptable and harmful answer looks like.
  3. Grade as cheaply as possible. Use code for format, facts and forbidden content; use model judges for tone and completeness, and check them against human labels.
  4. Wire evals into the pipeline. Run them on every prompt, model, retrieval or tool change, and block releases that regress.
  5. Track scores over time. Watch trends per category, and add every production failure to the test set.

Example scenario

An insurer had a dozen generative AI pilots spread across claims, underwriting and customer service. Each had been approved on the strength of a demo, and none had a measure of quality. When the platform team upgraded the shared model, claims staff complained that summaries had started to miss exclusions, and nobody could say whether the other pilots had changed too.

The architects made evals a condition of reaching production. Each team built a test set of a few hundred anonymised real cases, labelled by senior handlers, with code checks for policy numbers and dates and a model judge for completeness. The evals ran in the deployment pipeline. Two pilots could not reach a passing score and were stopped; the rest now move to new model versions only when the scores hold.

Questions to ask

  • What set of real cases tells us this AI feature is good enough, and who wrote down what good means?
  • Which checks can code grade, which need a person, and where do we trust a model as judge?
  • If the vendor changes the model next month, how would we know within a day whether quality dropped?

When not to use it

  • Throwaway exploration, where you are still learning what the feature should do; write a handful of cases, not a pipeline.
  • Deterministic code around the model, which ordinary unit tests cover better.

How it misleads you

A score becomes a target. Teams tune prompts to the test set until the number rises while real users see no improvement, which is Goodhart's Law at work. A model judge that has never been checked against people can reward fluent nonsense. Keep refreshing the cases from production, and keep a human reading samples.

In the wild

Further reading

Last reviewed · Suggest an improvement