Little's Law

The idea

John Little published a proof of the queueing formula L = λW in 1961. For a stable system, the average number of items inside (L) equals the average arrival rate (λ) multiplied by the average time each item spends inside (W). It holds regardless of the distribution of arrivals or service times, which is what makes it so widely useful.

It is a superb back-of-envelope tool: if you know two of the three values, you know the third. It applies to requests in a service, messages in a queue, and tickets in a delivery process alike.

Why it works

It needs almost no assumptions. You do not need to know the shape of the traffic, only averages over a long enough period.

It links concurrency to latency. At a given arrival rate, longer response times mean more requests in flight, which means more threads, connections and memory.

It explains lead time. At constant throughput, more work in progress means each item waits longer. Reducing work in progress is often the cheapest way to cut lead time.

Where it applies

• Sizing pools and queues, such as thread pools, connection pools and consumer counts.
• Diagnosing latency under load, when a pool is smaller than the concurrency the traffic implies.
• Capacity planning, to estimate how many instances a given load needs.
• Delivery flow, where work-in-progress limits shorten lead times.

How to apply

1. Measure two of the three. Arrival rate and average time in the system are usually easiest to get from metrics.
2. Calculate the third. Multiply rate by time to get average items in flight, and compare it with your limits.
3. Add headroom for peaks. The law gives averages; size for peak rates and tail latency, not just the mean.
4. Use it for work in progress too. Divide work in progress by throughput to estimate lead time, and set limits accordingly.

Example scenario

An API received 500 requests per second at peak, and each request spent about 200 milliseconds talking to the database. By Little's Law, about 100 requests were using the database at any moment. The connection pool was set to 20, so most requests queued for a connection, and latency climbed sharply under load while the database itself looked idle.

The team raised the pool size after checking that the database could handle the extra connections, and worked on cutting the query time. Halving the time per request also halved the connections needed, which gave them headroom for the next peak.

Questions to ask

When not to use it

• Unstable systems, where arrivals exceed capacity and the queue grows without bound; there is no steady state to average.
• Short windows or tail behaviour, where averages hide the spikes that matter.

How it misleads you