Circuit Breaker

The big picture#

Everything below expands on these points. Read the core sections top to bottom for the full mental model; the collapsible "Go deeper" boxes hold the tuning math and edge cases you can skip on a first pass.

Start here: why retrying isn't enough#

A naive client just calls the dependency every time and waits for a response or a timeout. If the dependency is down, every caller now blocks for the full timeout on every request — and if timeouts are long (they often are, to tolerate normal slowness), that's a lot of held threads, connections, and queued work for a call that was never going to succeed.

Worse than plain errors is slowness. A dependency that returns an error in 1 ms lets the caller move on. A dependency that hangs until a 10-second timeout holds a thread (and a connection) for those 10 seconds doing nothing. At even modest request rates, the caller's fixed-size thread pool fills with threads parked on doomed calls, and there are none left to serve healthy requests.

The circuit breaker's insight: once you have strong evidence the dependency is unhealthy, the most useful thing you can do is stop calling it. Failing instantly returns the thread immediately, frees the connection, and gives the dependency room to recover instead of piling on. You trade a few requests that *might* have succeeded for the survival of the whole caller.

The three states#

A circuit breaker sits in front of every call to the dependency and is always in exactly one of three states.

1. CLOSED: every call counts toward a sliding window of the last N outcomes. Once the window has enough calls to be statistically meaningful and the failure rate within it reaches the threshold, the breaker trips OPEN.
2. OPEN: every call is rejected instantly with no network call at all — this is the 'fail fast' the pattern is named for. A cooldown timer starts the moment the breaker opens.
3. Once the cooldown elapses, the breaker doesn't proactively probe on its own — the very next call attempt is treated as a canary trial and is actually allowed to reach the dependency. This is the HALF_OPEN state.
4. If that canary call (or the configured number of consecutive canary calls) succeeds, the breaker assumes the dependency has recovered: it closes and resets its failure window from scratch. If any canary call fails, the dependency is still unhealthy — back to OPEN, and the cooldown timer restarts.

One subtlety worth internalizing: a breaker does not need to understand *why* the dependency is unhealthy. It only observes outcomes — success, failure, slow-call — at its own edge. That's what makes it composable: the same breaker logic wraps an HTTP client, a database driver, a gRPC stub, or a message publisher without knowing anything about them.

Why Half-Open exists at all#

Without Half-Open, a breaker would have only two options once cooled down: stay Open forever (the dependency could have recovered minutes ago and nothing would know), or jump straight back to Closed (letting the full, possibly-still-unhealthy traffic volume slam the dependency the instant the cooldown ends — the exact pile-up the breaker exists to prevent).

Half-Open is the compromise: a small, controlled number of canary requests test reality before committing either way. It's deliberately asymmetric — one failure is enough to reopen, but recovery requires multiple consecutive successes — because the cost of being wrong about 'recovered' (slamming a still-broken dependency) is higher than the cost of being wrong about 'still broken' (one more cooldown cycle).

How it tracks failures: counts, windows, and timers#

Under the hood a breaker is cheap — a small amount of state updated on every call. The interesting design choice is how it remembers recent outcomes, because that determines how sensitive and how fair the tripping decision is.

• Consecutive-count: keep a single counter of failures in a row; trip at, say, 5 consecutive failures; any success resets it to 0. Trivial (O(1) state) but blind to mixed traffic — 50% failures interleaved with successes never trip it.
• Count-based sliding window: keep the last N outcomes (e.g. a ring buffer of 100 booleans) and trip when the failure rate over those N crosses a threshold. O(N) memory, O(1) update. Reacts to rate, not just streaks.
• Time-based sliding window: bucket outcomes into the last T seconds (e.g. ten 1-second buckets) and trip on the failure rate within that window. Bounds how stale the evidence can be, independent of request rate — better for bursty or low-traffic services.

Most production breakers also count slow calls as failures: a call that returns successfully but took longer than a slow-call-duration threshold is recorded as a failure for tripping purposes. This is what catches the 'not erroring, just hanging' dependency — the most dangerous case — before its slowness exhausts the pool.

Variants and the patterns it pairs with#

A circuit breaker is rarely deployed alone. It's the decision-maker in a small stack of resilience patterns, each handling a different part of the problem:

• Timeout: bounds how long any single call may block. This is the prerequisite — a breaker can't protect you from a slow dependency if individual calls never time out. Set per-call timeouts first, then wrap a breaker around them.
• Retry with exponential backoff + jitter: handles transient blips (one dropped packet, one GC pause). Backoff spaces attempts out; jitter (randomized delay) prevents synchronized retry storms. Critically, retries belong inside the breaker — a tripped breaker should suppress retries, because retrying against a known-dead dependency is exactly the pile-on you're avoiding.
• Bulkhead: isolates resources (separate thread pools or concurrency limits per dependency) so a flood toward one dependency can't drain the pool shared with others. Named after a ship's watertight compartments. The breaker decides *whether* to call; the bulkhead bounds *how many* concurrent calls can exist.
• Fallback: what you return when the breaker is open or a call fails — a cached value, a default, a degraded response, or a fast error. The breaker creates the opportunity to fail gracefully; the fallback decides what 'gracefully' means.

On the tripping logic itself, the main split is consecutive-count vs rate-based windows (covered above). The other axis is where the breaker lives:

• In-process library: the breaker runs inside your application (resilience4j on the JVM, Polly on .NET, historically Hystrix). It sees real call outcomes and can run per-instance or per-dependency.
• Out-of-process / mesh: a sidecar proxy trips on observed outcomes (Envoy's outlier detection ejects unhealthy upstream hosts; Istio configures this declaratively). No code change in the app, and it works across languages — at the cost of seeing only what crosses the proxy.

The trade-offs#

A circuit breaker is fundamentally a bet that recent failures predict near-future failures. When that bet is right, it saves the system. When it's wrong, it rejects requests that would have succeeded. Every design choice is about managing that bet.

Where it goes wrong#

• Flapping: a breaker that closes too eagerly trips, recovers on one lucky trial, gets slammed, trips again — oscillating Open↔Closed instead of settling. Caused by too-short cooldowns or requiring too few Half-Open successes. Fix: require multiple consecutive trial successes and use exponential backoff on the cooldown.
• Thundering herd on Half-Open: if Half-Open admits all waiting traffic instead of a small trial budget, the first instant after cooldown re-floods a still-fragile dependency and re-breaks it. Fix: cap concurrent Half-Open trials to a handful.
• Mis-tuned thresholds: a threshold that's too low trips on normal variance (turning resilience into self-inflicted unavailability); too high never trips until the caller has already drowned. Both come from setting numbers without measuring real failure rates and latency.
• Tripping on transient blips: counting one-off network hiccups as breaker-worthy failures opens the circuit for problems that retries would have absorbed. Distinguish 'retryable transient' from 'breaker-worthy sustained' — let retries handle the former.
• Wrong failure classification: counting business errors (a 404, a validation 400) as breaker failures trips the circuit when the dependency is perfectly healthy. Only count failures that indicate the dependency itself is unhealthy (5xx, timeouts, connection errors).
• Shared vs per-instance state confusion: an in-process breaker is per-instance, so 10 caller instances each learn independently and trip at different times. That's usually fine, but don't assume a single global view unless the breaker state is genuinely shared.

Circuit breaker vs the other resilience patterns#

These are complementary, not alternatives — a robust client uses several together. The table shows what each one is actually for.

Where circuit breakers run in the wild#

The pattern is standard equipment in every mature microservice stack. The implementations differ mainly in where they run and how they track failures.

• Netflix Hystrix — the implementation that popularized the pattern at scale (each command ran on its own bulkheaded thread pool with a breaker). Netflix put it into maintenance mode in 2018: it stopped active development in favor of adaptive, lighter-weight approaches (concurrency-limiting like adaptive-concurrency/concurrency-limits) and pointed users toward resilience4j. Still everywhere in older codebases and interview questions.
• resilience4j — the de facto JVM successor: a lightweight, functional, modular library (separate CircuitBreaker, Retry, RateLimiter, Bulkhead, TimeLimiter modules) with count-based and time-based sliding windows and slow-call detection. The reference implementation most people learn from today.
• Envoy / Istio outlier detection — the service-mesh approach: the sidecar proxy observes upstream host health and ejects (temporarily removes) hosts that return too many consecutive 5xx or gateway errors, re-admitting them after a base-ejection time. Language-agnostic, no app code.
• Polly (.NET) — the standard resilience library for .NET, with circuit-breaker, retry, timeout, bulkhead, and fallback policies that compose into resilience pipelines; integrated into Microsoft.Extensions.Http resilience.
• AWS SDKs / cloud clients — retry behavior with backoff and (in newer SDKs) adaptive retry and circuit-breaker-style token buckets that stop retrying when a dependency is shedding load. The same idea baked into the client library.

Common misconceptions & gotchas#

In an interview#

Lead with the failure mode this prevents: resource exhaustion from cascading timeouts, not just 'the dependency returns errors.' A circuit breaker's value is in how fast it fails once it's open — it protects the caller's own resources, not just the user-facing error rate.

Be ready to name the three states and the specific trigger between each one — interviewers often probe whether you understand Half-Open is reached lazily (on the next call attempt after the cooldown), not via a background timer that proactively flips the state.

Score points by placing it in the resilience family: timeouts bound each call, retries with backoff+jitter absorb transient blips, bulkheads isolate pools, and the breaker decides whether to attempt at all — and suppresses retries while Open. Mention rate-based vs consecutive-count windows and slow-call detection, and name a real implementation (resilience4j, Polly, or Envoy outlier detection; Hystrix as the historical original, now in maintenance mode).

Then try it in the simulator: send a burst of healthy calls (stays Closed), then a burst of failures (trips Open), watch requests get rejected instantly, advance time past the cooldown, and send a failing canary followed by a healthy one to see the full Open → Half-Open → Open → Half-Open → Closed cycle.

References & further reading#