A Circuit Breaker Protects You From Your Dependency, Not the Other Way Around
Almost every explanation frames this as being kind to a struggling downstream service. That is a side effect. Its actual job is stopping your own threads from piling up on timeouts, which is how a dependency you barely use takes down a service that does not need it.
Key takeaways
- A circuit breaker has three states: closed, where calls pass and outcomes are monitored; open, where calls fail immediately without being attempted; and half-open, where a small fixed number of trial calls probe whether the dependency has recovered.
- The half-open state typically permits three to five trial calls before deciding to close or re-open, which is what stops a recovering dependency from being hit with full traffic the instant it comes back.
- The primary benefit of the open state is that there is no held concurrency, no timeout to wait through, and no retry storm aimed at a dependency that is already struggling.
- A circuit breaker without an aggressive timeout is close to useless, because a breaker measures failures and a call that hangs indefinitely never reports one, so your threads exhaust before the breaker ever trips.
- A circuit breaker deliberately increases your error rate in the short term by converting slow successes into fast failures, which will look like a regression on any dashboard measuring success rate rather than latency.
- The bulkhead pattern is the complement: it caps how much of your capacity any single dependency can consume, so one slow call path cannot starve the rest of the service.
Every explanation of this pattern says the same thing: a circuit breaker stops you hammering a service that is already in trouble. Which is true, and which is the less important half.
The failure it actually prevents happens inside your process. A dependency gets slow. Not down, just slow. Your requests to it start taking eight seconds instead of eighty milliseconds. Every one of those holds a thread, a connection, a slot in whatever pool you have. Traffic keeps arriving. Within a minute or two your service has no capacity left for anything, including the endpoints that never touch that dependency at all.
That is how a recommendations service that nobody would call business-critical takes down a checkout flow. Not by failing. By being slow while you politely waited.
“The dependency did not go down. Your service went down waiting for it, on threads it could have used for something else.”
The Three States
Closed. Normal. Calls pass through and the breaker watches the outcomes, counting failures against whatever threshold you configured.[1]
Open. Tripped. Calls fail immediately and are not attempted.[1] This is the state that does the work, and the reason is worth stating explicitly: there is no held concurrency, no timeout to sit through, and no retry storm pointed at a dependency that is already struggling.[2] Your caller gets an error in microseconds and your capacity stays yours.
Half-open. Cautious re-entry. After a cooldown, the breaker permits a small fixed number of trial calls, commonly three to five, and decides based on those whether to close or re-open.[1]
Half-open exists because the obvious alternative is a disaster. If a tripped breaker simply reopened the floodgates after sixty seconds, a dependency that just came back would be hit with a minute of accumulated traffic at once and fall over again immediately. You would have built an oscillator. Trial calls are how you ask the question without betting the answer.
Without a Timeout, It Barely Works
This is the configuration mistake I have seen most often, and it makes the whole thing decorative.
A breaker decides to trip by counting failures. A call that hangs has not failed yet. So if your HTTP client has a 60-second timeout, or worse no timeout, then a dependency that stops responding produces no failures for a full minute. Your workers are consumed the entire time. The breaker sits in closed state observing a completely healthy-looking zero failures, because nothing has come back yet to be counted.
The timeout is what converts a hang into a failure the breaker can see. Set it aggressively, relative to the p99 you actually observe rather than to a round number that feels safe, and the breaker starts functioning. This is also why latency is the metric to instrument here: the breaker is a latency defence wearing an error-handling costume.
Why this matters
It Will Make Your Error Rate Worse
Worth knowing before you ship it, because otherwise someone reverts it.
A circuit breaker converts slow successes into fast failures. A request that would have taken nine seconds and eventually worked now fails in a millisecond. If your dashboard tracks success rate, adding a breaker looks like a regression: errors up, and by a lot.
The service is healthier. Latency is down, saturation is down, and the unrelated endpoints that were collateral damage are now fine. But the headline metric moved the wrong way, so this change has to be evaluated on latency and saturation rather than error count, and that has to be agreedbefore the deploy rather than argued afterwards.
The corollary is that a breaker is only as good as what you do in the open state. Failing fast is strictly better than hanging, but returning a cached value, a degraded response, or a sensible default is better than both. That is where caching stops being a performance feature and becomes an availability one.
Bulkheads, Which Catch What Breakers Miss
The complementary pattern: a bulkhead isolates resources so that failure or slowness in one place cannot consume everything.[3] In practice, a separate connection pool or concurrency limit per dependency.
It matters because breakers have a blind spot. A dependency that isintermittently slow, never quite crossing your failure threshold, will never trip the breaker and can still eat your capacity indefinitely. A bulkhead caps the damage by construction rather than by detection: that dependency gets ten connections, and when they are busy, calls to it queue or fail while everything else proceeds untouched.
Detection-based protection fails on the cases you did not anticipate. Capacity-based protection does not need to anticipate them.
When Not to Bother
Being honest about the cost, since this pattern gets applied indiscriminately.
A breaker adds state, configuration, a new failure mode of its own (a breaker stuck open because a threshold is wrong is an outage you caused), and metrics somebody has to watch. On a call to a database you cannot function without, tripping the breaker just converts a slow outage into a fast one, and you may prefer the slow one.
Breakers earn their keep on optional dependencies: recommendations, enrichment, analytics, third-party APIs, anything where degrading is genuinely better than waiting. Which is the same instinct that should govern whether the call is synchronous in the first place, and part of the wider question in monolith versus microservices: every network hop you add is a new thing that can be slow at you.
Takeaway
A circuit breaker exists to stop a slow dependency exhausting your own threads, which is how one non-critical service takes down endpoints that never called it. It needs an aggressive timeout to function at all, since a hang is not a failure and cannot be counted. Expect your error rate to rise, because slow successes are being converted into fast failures on purpose. Pair it with a bulkhead, which catches the intermittently-slow dependency that never trips a threshold. And skip it on dependencies you cannot degrade without.
Finding out that a dependency has gone intermittently slow, before it costs you an incident, is a different problem from containing it, and it needs high-cardinality context rather than a threshold: why the three pillars are not the point.
And it is worth being clear that the breaker is the right tool for this and a health check is not, because a health check can only make a whole-instance decision: a check that queries a shared database will fail on every server at once and remove your entire fleet in response to one dependency blip.
This is the third of three patterns that keep coming up together and get written about separately. Rate limiting protects you from your callers, idempotency keys make the retries safe, and the circuit breaker protects you from the things you call. Any two without the third leaves an obvious hole.
Sources and further reading
- 1.Reporting"Circuit breaker pattern in microservices: design pattern, states, and resilience best practices". Source for the three states and the half-open trial-call allowance of roughly three to five.
- 2.Reporting"Microservices pattern: circuit breaker, stop cascading failures before they cascade". Source for the open state removing held concurrency, timeout waits, and retry storms against a struggling dependency.
- 3.ReportingGeeksforGeeks, "Microservices resilience patterns". Source for the bulkhead pattern isolating resources so one failure does not affect others.
- 4.Primary"Resilient microservices: a systematic review of recovery patterns, strategies, and evaluation frameworks". Academic survey of recovery patterns, useful for how these mechanisms are evaluated rather than just described.
Frequently asked questions
- What is the circuit breaker pattern?
- It is a wrapper around calls to a dependency that stops making them once failures cross a threshold, returning an immediate error instead. After a cooldown it allows a few trial calls through to test whether the dependency has recovered, which is what prevents a failing dependency from consuming the caller's resources indefinitely.
- What are the three circuit breaker states?
- Closed is normal operation, where calls pass through and the breaker watches the outcomes. Open is the tripped state, where calls fail instantly without being attempted at all. Half-open is a cautious re-entry that lets a small fixed number of trial calls through, commonly three to five, and closes the breaker if they succeed or re-opens it if they do not.
- Who does a circuit breaker actually protect?
- Primarily the caller, not the dependency. The failure mode it prevents is your own threads or connections piling up waiting on a slow dependency until your service has no capacity left for work that does not involve that dependency at all. Reducing load on the struggling downstream is a real benefit but it is a side effect of the caller protecting itself.
- Do I need a timeout if I have a circuit breaker?
- Yes, and the breaker barely functions without one. A breaker decides to trip by counting failures, and a call that hangs has not failed yet, so with a 60-second timeout your workers are consumed for a minute per request while the breaker sees nothing wrong. The timeout is what converts a hang into the failure signal the breaker needs.
- Why does my error rate go up after adding a circuit breaker?
- Because that is the intended behaviour. The breaker converts slow successes into fast failures, so requests that would have eventually succeeded after ten seconds now fail in a millisecond. Success rate looks worse while the service is measurably healthier, which is why this change should be evaluated on latency and saturation rather than on error count alone.
- What is the difference between a circuit breaker and a bulkhead?
- A circuit breaker stops calling a dependency that appears to be failing, while a bulkhead limits how much of your resources any one dependency may consume in the first place. They solve the same underlying problem from different directions, and the bulkhead keeps working when a dependency is slow in a way that never quite trips the breaker.
Written by
Tech Talk News Editorial
Computer engineering background. Writes about software, AI, markets, and real estate, and the places where the three meet.
More about the author