Round Robin Distributes Requests, Not Work. And Your Health Check Cuts Both Ways.
Two things decide whether a load balancer helps. The algorithm, where round robin quietly assumes every request costs the same. And the health check, which is either too shallow to notice a broken server or so deep that one shared dependency marks your entire fleet unhealthy at once.
Key takeaways
- Round robin distributes request counts rather than work, so when request costs vary it keeps sending traffic to a server that is already busy with expensive requests.
- Least connections and least response time react to live load and are better defaults for mixed workloads, while round robin suits stateless services with uniform request cost and equal server capacity.
- A health check that only verifies the port is open will keep routing traffic to a server that is running but broken, such as one whose database connection has died or that is still warming up.
- A health check that tests a shared dependency can mark every server unhealthy at the same moment when that dependency has a brief problem, removing the entire fleet in response to something no individual server caused.
- The resolution is to separate the two questions: liveness should be shallow and local, checking only whether this process is functioning, while readiness may be deeper but must not fail for reasons common to every instance.
- In container environments the liveness and readiness distinction matters more, because without it half-initialised pods are placed into rotation before they can serve requests.
A load balancer is two decisions wearing one name. How to pick a server, and how to decide a server should be picked at all. The first gets all the attention and the second causes the outages.
Round Robin Assumes Something Untrue
Round robin sends request one to server A, two to B, three to C, and round again. Perfectly even, and the evenness is in the wrong unit.
It distributes counts, not work.[1] If requests vary in cost, and in a real service they vary enormously, then a server that happened to receive three report generations gets the next request anyway, while a peer that received three cache hits sits idle. Round robin has no way to know, because it never asked.
Least connections is the usual upgrade: send each new request to whichever server currently holds the fewest open connections, which is a live proxy for who is least busy.[1] Least response time goes further and reacts to how fast each server is actually answering.[2]
The honest guidance: round robin for stateless services with uniform request cost and matched server capacity, least connections when processing times vary.[2] Most services are the second case and are configured as the first, because round robin is the default.
“Round robin is fair in the sense that it gives every server the same number of requests, which is not the sense anybody wanted.”
The Health Check Is the Feature That Matters
Now the part that actually breaks things. A load balancer's real job is not distributing traffic, it is not distributing traffic to servers that cannot serve it. That is entirely a function of your health check, and health checks fail in two opposite directions.
Too shallow, and you route into a broken server
A check that verifies the port is open, or that / returns 200, proves the process is listening.[3] It proves nothing about whether a request will succeed. A server whose database connection has died, or that is still warming caches, passes happily and receives its share of traffic, which it then fails.[3]
From the outside this looks like intermittent errors affecting roughly one in N requests, which is a genuinely confusing symptom to debug, because every retry has a decent chance of landing on a healthy instance and working. Diagnosing it needs per-instance attribution on your telemetry, which is the argument for high-cardinality observability: an aggregate error rate of 12% tells you nothing, and the same number broken down by instance tells you everything.
Too deep, and you remove the entire fleet
So make the check meaningful. Have it query the database, confirming the server can really do work.
Now the database has a three-second blip. Every server runs its health check against that same database. Every check fails. The load balancer, behaving exactly as designed, removes every server from rotation.
A dependency that was briefly slow has become total unavailability, because the health check converted a shared problem into a unanimous verdict. You built an outage amplifier and called it a safety feature.
Why this matters
Liveness and Readiness Are Different Questions
This is what the two-probe model exists for, and it is worth having clearly even outside Kubernetes.[4]
Liveness: is this process still functioning, or should it be restarted? Shallow, local, cheap. It should almost never fail, and when it does the answer is to kill the process.
Readiness: can this instance serve traffic right now? May be deeper. Failing it removes the instance from rotation without killing it, which is exactly right for a server that is warming up, draining, or temporarily overloaded.
Conflating them produces two specific bugs. Use one shallow check for both and half-initialised instances get traffic before they can serve it, which is the problem the distinction was introduced to prevent.[4] Use one deep check for both and a transient dependency issue does not merely drain your fleet, it triggers a restart of every process in it, which is the same outage plus a cold start.
The Load Balancer You Added Is Also a Failure Point
Worth saying plainly, since the balancer is usually introduced as an availability improvement. You added a component that every request must pass through. That is a new single point of failure in service of removing others.
It is normally a good trade, because a balancer is far simpler than the application behind it and fails less often. But it is a trade, and it only holds if the balancer itself is redundant. A single load balancer in front of six application servers has moved your availability ceiling to that one box. This is the same accounting as any other network hop you introduce: each one buys something and costs a thing that can be down.
What I Would Configure
Least connections unless you know costs are uniform. The default is round robin and the default is usually wrong.
Liveness shallow, readiness specific.Liveness answers “is this process alive”. Readiness may check local state, an in-process pool, a warmed cache, but not shared infrastructure.
Let readiness fail fast and recover fast. Short intervals and a low failure threshold, so a genuinely broken instance leaves rotation in seconds, paired with a low success threshold to return.
Never let health checks drain everything. Many balancers support a minimum healthy count, below which they route to all instances regardless. Sending traffic to possibly-broken servers beats sending it nowhere, and this single setting is the difference between a degraded service and a dark one.
Handle the dependency separately. A failing dependency is a job for a circuit breaker in the application, which degrades one call path, not for the health check, which can only make a whole-instance decision. Using the balancer to respond to dependency failure is using an instrument with one blunt setting.
Takeaway
Round robin distributes counts, not work, so prefer least connections whenever request costs vary. Then spend your attention on the health check, which is where the outages are: a port-open check routes traffic into broken servers, and a check that queries shared infrastructure marks the whole fleet unhealthy the moment that infrastructure hiccups. Keep liveness shallow and local, keep readiness free of shared dependencies, set a minimum healthy count, and handle failing dependencies with a circuit breaker instead.
Sources and further reading
- 1.Reporting"Load balancing strategies: round robin is just the beginning". Source for round robin distributing counts rather than work, and least connections and least response time reacting to live load as better defaults for mixed workloads.
- 2.Reporting"Load balancing strategies: round robin, least connections and more". Source for round robin suiting stateless services with similar capacity and least connections suiting variable processing times.
- 3.Reporting"How to implement load balancer health checks". Source for port-only checks keeping traffic flowing to a server that is up but broken, including a dead database connection or an instance still warming up.
- 4.PrimaryKubernetes documentation, "Configure liveness, readiness and startup probes". The authoritative description of the probe types and why readiness prevents half-initialised instances receiving traffic.
Frequently asked questions
- Is round robin a good load balancing algorithm?
- It is fine when every request costs about the same and every server has the same capacity, and misleading otherwise. Round robin distributes counts rather than work, so a server that received three expensive requests still gets the next one even though it is far busier than its peers.
- When should I use least connections instead of round robin?
- When request processing times vary, which in practice is most real services. Least connections sends each new request to whichever server currently has the fewest open connections, which is a live proxy for who is least busy, so it adapts to uneven cost instead of assuming it away.
- What is wrong with a simple health check?
- A check that only confirms the port is open or that the root path returns 200 proves the process is listening, not that it can do work. A server whose database connection has died, or that is still warming caches, will pass that check and receive traffic it cannot serve.
- Can a health check cause an outage?
- Yes, and it is one of the nastier self-inflicted failures. If your health check queries a shared database and that database has a brief problem, every server fails the check simultaneously, so the load balancer removes the entire fleet from rotation. A dependency blip becomes a total outage because your health check turned a shared problem into a unanimous verdict.
- What is the difference between liveness and readiness?
- Liveness asks whether this process is still functioning and should be restarted if not, while readiness asks whether it can serve traffic right now. Keeping them separate is what lets you take a warming instance out of rotation without killing it, and in container environments the distinction is what prevents half-initialised pods receiving requests.
- Does a load balancer improve availability?
- It improves availability against the failure of individual servers and introduces a new component that can fail on its own. That is usually a good trade because the balancer is simpler than what it fronts, but it is a trade rather than a free gain, and the balancer needs its own redundancy to be worth making.
Written by
Tech Talk News Editorial
Computer engineering background. Writes about software, AI, markets, and real estate, and the places where the three meet.
More about the author