Feedback Loops

Outputs that circle back as inputs either stabilise a system (balancing) or amplify it (reinforcing).

A loop either amplifies itself, like a retry storm, or pulls the system back towards balance, like autoscaling.

To an executive

Slow feedback lets problems grow unnoticed before anyone reacts. Shorter, faster signals on cost, quality and customer response keep the business closer to its goals.

To an engineer

Identify which loops in your system reinforce and which balance. Shorten feedback delays, and watch for retry storms and runaway costs, since delayed reactions tend to overshoot.

The idea

A feedback loop exists when a system's output comes back to affect its input. There are two kinds:

Reinforcing loops compound. More users bring more data, which improves the product, which brings more users. The same shape drives retry storms and runaway costs.
Balancing loops seek a goal. A thermostat, or autoscaling that adds capacity as load rises, pulls the system back towards a target.

Donella Meadows, in her essay on leverage points, stressed that delays in feedback loops are critical to how a system behaves. Slow feedback, such as quarterly releases or monthly cost reports, lets a system drift far off course before anyone reacts, and delayed reactions tend to overshoot.

Why it works

Structure explains behaviour. Growth, collapse and oscillation usually come from the loops in a system, not from one-off events. Find the loops and the behaviour makes sense.

Delay decides stability. A balancing loop with a long delay overshoots and oscillates. Shortening the delay often helps more than strengthening the response.

Unchecked reinforcing loops end badly. Meadows noted that a reinforcing loop with nothing to slow it eventually destroys the system it runs in, so damping one is often a high-leverage fix.

Where it applies

Production resilience, where retries, failover and autoscaling can each form loops that amplify or damp load.
Delivery pipelines, where continuous integration, canary releases and observability shorten the loop from change to learning.
Cost and capacity management, where slow reporting delays the correction.
Platform and product strategy, where network effects and adoption are reinforcing loops.

How to apply

Draw the loops. For a key behaviour, sketch what feeds what, and mark each loop as reinforcing or balancing.
Measure the delay. Ask how long it takes for an effect to show up, and for someone to see it.
Damp dangerous reinforcing loops. Add backoff with jitter, retry budgets, circuit breakers or rate limits.
Shorten useful balancing loops. Move from monthly to daily signals, and from manual to automatic responses where it is safe.

Example scenario

During a partial outage, mobile clients retried failed calls immediately and repeatedly. Each retry added load to an already struggling service, which caused more failures and more retries. What began as a brief slowdown in one database became a full outage of the API.

The team broke the loop on both sides. Clients moved to exponential backoff with jitter and a cap on retries, and the server gained a circuit breaker and load shedding. The next database slowdown caused a short, partial degradation instead of an outage.

Questions to ask

Which loops here are reinforcing, and which are balancing?
How long after something goes wrong do we find out, and who finds out first?
Could this mechanism — retries, autoscaling, caching — amplify a failure instead of damping it?

When not to use it

One-off, linear problems, where a direct cause and fix exist and loop diagrams add little.
When you have no data on the loop's strength or delay; a guessed loop can send effort in the wrong direction.

How it misleads you

Seeing loops everywhere and modelling them without data. Diagrams of loops are persuasive and easy to draw. Check each loop against observed behaviour, such as metrics showing the retry rate rising with the error rate, before acting on it.