The Bitter Lesson
General methods that scale with computation beat clever, hand-built knowledge — eventually, and by a large margin.
“…general methods that leverage computation are ultimately the most effective, and by a large margin.”
To an executive
Over decades, simple methods that use more computing power have beaten clever systems built on human expertise. Heavy investment in hand-built rules may be overtaken as computing gets cheaper.
To an engineer
Be wary of elaborate prompt logic and rule layers around language models. Favour general approaches that improve as models improve, and expect hand-built workarounds to be replaced.
The idea
In a short 2019 essay, Rich Sutton looked back over seventy years of AI research and drew an uncomfortable conclusion. Again and again — in chess, Go, speech recognition and computer vision — researchers built systems around human knowledge of the problem. They worked for a while. Then simpler, more general methods that used more computation, mainly search and learning, overtook them.
The lesson is "bitter" because it is not the one researchers wanted. Encoding what we know feels productive and is satisfying in the short term, but it stops scaling. Methods that let the machine find the structure itself keep improving as computation gets cheaper.
Why it works
Computation keeps getting cheaper. Any approach whose quality grows with compute rides that curve for free. Hand-built knowledge does not.
Human knowledge is a ceiling. Rules capture how we currently understand a problem, including our blind spots. A learning system is not limited to that understanding.
Where it applies
- Building on large language models. Elaborate prompt logic, rule layers and hand-written fallbacks around a model often become unnecessary — or harmful — when the next model arrives.
- Choosing between rules and learning for classification, routing, ranking or extraction.
- AI platform strategy. Invest in what stays useful as models improve: data, evaluation, integration and feedback loops.
How to apply
- Separate scaffolding from substance. Mark which components exist only to compensate for today's model weaknesses.
- Keep that scaffolding thin and replaceable. Treat it as sacrificial, behind clear interfaces.
- Invest in what compounds. Good evaluation sets, clean data and fast feedback loops get more valuable as models improve.
- Re-test on every model upgrade. Remove workarounds the new model no longer needs.
Example scenario
A team spent months writing extraction rules on top of an LLM to handle messy invoices: regex fallbacks, field-by-field prompts, and a correction layer. When a newer model arrived, a single well-specified prompt beat the whole pipeline on their evaluation set.
Because the rules sat behind one interface and the evaluation set was solid, they could prove the improvement in a day and delete most of the pipeline.
Questions to ask
- Which parts of our design encode knowledge that a better model will soon learn on its own?
- If model capability doubled next year, which of our components would become dead weight?
- Are we investing in scaffolding that scales with compute, or in rules that compete with it?
When not to use it
- Hard constraints. Safety rules, legal requirements and business invariants must be enforced deterministically, whatever the model can learn.
- Small, stable problems where a few rules are cheap, transparent and good enough.
How it misleads you
Taking it as a reason to add no structure at all. The lesson is about where knowledge lives, not a licence to skip evaluation, guardrails or design. Systems around models still need clear boundaries; they just should not compete with the model at what it learns best.
In the wild
Further reading
Last reviewed · Suggest an improvement