Skip to content
Latticework
Esc
Try
All playbooks

Playbook · Design and build · 7 steps

Building a feature with AI

Start simple, measure everything, and never combine the dangerous capabilities.

“…unsuccessful products almost always share a common root cause: a failure to create robust evaluation systems.”

Start with step 1 All 82 models for design review
  1. 1The Jagged FrontierTest the task on real examples first: is it inside the AI's frontier, and what does a wrong answer cost?AI is excellent at some tasks and unreliable at others of similar difficulty — and the boundary is invisible until you test it.Open
  2. 2Workflows Before AgentsWhat is the simplest design that works — one call, then a fixed workflow — before any agent?Use the simplest LLM design that works; add agent autonomy only when a problem truly needs it.Open
  3. 3Context vs BehaviourWith a baseline and an evaluation set, are failures about missing knowledge or wrong behaviour?Diagnose an LLM failure first: missing knowledge needs better context; wrong behaviour needs better prompting or tuning.Open
  4. 4Context EngineeringWhat does the model need to see at each step, and what can it fetch on demand?A model's context window is a limited budget; curate the smallest set of high-signal tokens it needs for the task.Open
  5. 5The Lethal TrifectaDoes any part combine private data, untrusted content and a way to send data out?An AI agent that combines private data, untrusted content and a way to send data out can be tricked into leaking it.Open
  6. 6The Autonomy SliderHow much autonomy can people verify quickly, and which actions need approval?Give AI as much autonomy as you can verify quickly — and let people move the slider up as trust grows.Open
  7. 7Scaling LawsCan we swap in next year's model with a config change and a re-run of the evaluations?Model capability improves predictably as compute, data and model size grow, so plan for the models of next year, not today.Open