Scaling Laws
Model capability improves predictably as compute, data and model size grow, so plan for the models of next year, not today.
“The loss scales as a power-law with model size, dataset size, and the amount of compute used for training…”
To an executive
AI capability improves steadily and predictably, while the cost of a given level falls. Plans can target where the technology will be when the product launches.
To an engineer
Design for the model capability you expect at launch, not today's. Features almost possible now are good bets, and tasks needing a large model today may run cheaply later.
The idea
In 2020 a team of OpenAI researchers led by Jared Kaplan showed that the performance of language models follows smooth, predictable curves. As model size, training data and compute increase, the model's error falls along a power law — and the pattern held across more than seven orders of magnitude. Architectural details mattered far less than scale. Several of the authors, including Kaplan and Dario Amodei, went on to found Anthropic.
The practical message for architects is not the maths but the shape: capability improves steadily and somewhat predictably, while the cost of a given level of capability keeps falling.
Why it works
Predictability enables planning. If improvement follows a curve rather than lucky breakthroughs, you can design for where the curve will be when your system ships.
Cost per unit of capability falls. What needs the largest, most expensive model today often runs on a small, cheap one later.
Where it applies
- AI product roadmaps. Features that are almost possible today are good bets for tomorrow.
- Build-versus-wait decisions. Large investments to work around current model limits may be overtaken by the next release.
- Vendor and model choice. Avoid designs that tie you to one model you cannot replace.
How to apply
- Abstract the model. Put model calls behind an interface so you can change provider or version without rewriting the system.
- Keep an evaluation set. It is the only way to know quickly whether a new model is better for your task.
- Design for falling costs. Revisit features that were too expensive per call; they may now be viable.
- Time-box workarounds. Treat code that compensates for model weakness as temporary.
Example scenario
A legal-tech company shelved a contract review feature because the only model good enough was too slow and too expensive per document. They kept the evaluation set they had built and re-ran it every quarter.
Eighteen months later a much cheaper model passed it. Because model calls sat behind one interface, the feature shipped within weeks.
Questions to ask
- Which of our design decisions assume today's model quality, cost or speed?
- What becomes possible if the same capability costs a tenth as much next year?
- Are we locked into one model, or can we swap in a better one easily?
When not to use it
- Near-term commitments. If something must work reliably this quarter, design for the model you have.
- Extrapolating forever. Curves describe the past; they do not guarantee that a particular capability will arrive by a particular date.
How it misleads you
Waiting for the next model becomes an excuse never to ship. Scaling laws are a reason to keep designs flexible, not to postpone delivering value with the models available today.
In the wild
Further reading
Last reviewed · Suggest an improvement