#reliability

Two things decide whether a load balancer helps. The algorithm, where round robin quietly assumes every request costs the same. And the health check, which is either too shallow to notice a broken server or so deep that one shared dependency marks your entire fleet unhealthy at once.

Not hard, not expensive, not a premium tier. Impossible, in the proven sense, and it follows from the Two Generals problem. Every product advertising exactly-once is selling at-least-once with deduplication bolted on, which means the work was always going to land on your consumer.

Logs, metrics and traces is a taxonomy of data formats, not of capability. Having all three does not mean you can answer a question you did not anticipate. What lets you do that is high-cardinality context on every request, which is also precisely what your observability vendor prices on.

Almost every explanation frames this as being kind to a struggling downstream service. That is a side effect. Its actual job is stopping your own threads from piling up on timeouts, which is how a dependency you barely use takes down a service that does not need it.

The design detail that surprises people: Stripe caches the result of the first request under a key even when that result was a 500, and replays the failure to every retry. That is correct, and once you see why, the rest of the design follows, including where most homegrown implementations go wrong.

Everyone compares token bucket to sliding window and stops there. The decisions that actually determine whether a rate limiter works are what you key it on, whether your Retry-After header is quietly scheduling a thundering herd, and whether a client that gets a 429 can safely retry at all.

Anthropic went from $1B to roughly $30B in annual run-rate revenue in 15 months, and Claude now breaks most mornings between 9 AM and noon Pacific. The outages aren't random. They're concentrated exactly where Claude won: long-running coding agents, on the messiest multi-cloud stack in the industry, metered by a rate limiter that sometimes drains a weekly quota in 19 minutes.