Health checks: three different questions called by one name

What is actually being asked

The ladder usually runs like this:

1. "Why do you need health checks?" — the warm-up.
2. "How does liveness differ from readiness?" — where the substance starts.
3. "What do you check inside a probe?" — the central question.
4. "Why should liveness not touch the database?" — about cascades.
5. "Why a separate startup probe?" — about slow boots.
6. "What happens when readiness lies?" — about the opposite mistake.

This lesson rests on a model rather than on a measurement: there is no common standard for how probes behave, only the documentation of particular products. The model answers not "how much exactly" but "what is connected to what", and its assumptions are named.

Base: three questions, not one check

A service has several identical replicas, and from outside nothing about the inside of any of them is visible. There is exactly one way to ask: knock and look at the answer. That is where a health check comes from — a short question somebody puts to a replica at regular intervals, plus an action decided in advance for the case where the answer is no.

The whole topic rests on "are you healthy" not being one question. The observer can take several different actions, and different actions fix different things. So there are three checks, each with a question of its own:

The middle column for liveness is phrased unusually, and that is the main point of the lesson. The usual phrasing is "is the process alive" — but a probe is not needed for that question: a dead process does not answer at all, and that is visible without any check. The probe exists for the other case: the process is alive, it answers requests, and it has stopped working. What it decides is not "alive or not" but whether the one available cure — a restart — is worth applying.

The right-hand column answers the lesson's main question too: what goes inside each probe? It follows from the consequence — check what that consequence fixes, and nothing else.

That is already enough to answer the basic interview question. Everything below is about the price of getting that choice wrong: what missing readiness costs, why a probe that looks into a shared database adds failures to somebody else's outage, and why an over-diligent readiness is as harmful as a credulous one.

Mechanism 1: the consequence is the definition

There is no normative document for this split: below is a distinction by consequence, which this lesson uses as a definition. The confusion starts wherever all three questions are answered by one handler.

Liveness — "would restarting this process help". A negative answer means: the process is broken badly enough that only a restart can return it to working order. A deadlock, an exhausted thread pool, an infinite loop — internal, cumulative failures, all of them cured by putting the process back into a known state.

Readiness — "can traffic be sent here right now". A negative answer means: do not send me requests, but do not restart me either. A warming cache, a lost connection to a dependency, a temporary overload — states a replica leaves on its own if it is not being loaded.

Startup — "has it finished booting". A separate probe exists so that a slow boot does not look like a death. This lesson assumes liveness does not apply until the startup probe has passed — that is what the separate probe is for.

The difference is in the consequence: liveness restarts, readiness removes from rotation. Hence the rule everything else follows from: check what the consequence can actually fix. For liveness that reads literally: only what a restart of this process can fix belongs inside it.

Mechanism 2: what each probe buys

The model: three replicas, constant traffic, one replica starting to answer slowly — but answering — from the tenth second.

REQUESTS SERVED SLOWLY OR NOT AT ALL
------------------------------------
                  mode       slow     failed      total
                  none       1600       1000       6000
              liveness        520       1000       6000
    liveness+readiness         80       1000       6000
        liveness-on-db        640       1200       6000

No probes — 1600 slow answers. The balancer does not know the replica is unwell and keeps sending it a third of the traffic the whole time.

Liveness only — 520. The probe notices the trouble and restarts the replica; the restart takes it out for five seconds, then it comes back — and degrades again. It helps, but badly: the replica shuttles between "unwell" and "restarting".

Liveness plus readiness — 80. The replica is taken out of rotation without a restart, and the load on it stops. 6.5 times fewer slow answers than with liveness alone.

Hence the answer to the second rung: readiness is not "the same thing but gentler". It is a different consequence, and for degradation it is the apt one: a replica that is not being loaded may recover on its own, while a restarted one begins from a cold start.

Mechanism 3: why liveness does not touch the database

The fourth row of the run above is about the mistake that adds failures to somebody else's outage. Its slow answers (640) sit close to the liveness-only mode's 520; the whole difference went into the failed column, and that is what this section is about. Here the probe checks not only itself but a shared dependency: if the database is unavailable, liveness is considered failed.

WHAT THE THREE MODES ACTUALLY CHANGE
------------------------------------
  no probes: slow answers                      1600
  liveness only: slow answers                  520
  liveness + readiness: slow answers           80
  liveness tied to the database: failures      1200
  the same failures without that tie           1000

  failures added by tying liveness to the database 200

The database is down for ten seconds in both runs, and those ten seconds cost 1000 failed requests regardless of the probes — a probe cannot fix somebody else's outage.

The extra 200, however, belong entirely to the probe. The mechanism: the database goes down → liveness fails on every replica at once → every replica restarts → while they come up the service answers nothing at all, and that continues after the database is already back.

Hence the rule worth answering with: liveness checks only what a restart of this process can fix. A database, a network and a neighbouring service are not fixed by restarts; their place is readiness, where the consequence is milder — the replica stops being loaded but stays alive and comes back on its own.

And the second half of the same rule: a shared dependency makes probes synchronous. If the check is identical on every replica and depends on one external object, they all fail together — and the redundancy that three replicas were meant to provide disappears exactly when it is needed.

Deeper: the boundaries of the rule, and both readiness mistakes

The opposite mistake works the other way round and costs the most slow answers in the model. A readiness handler that always answers "ready" — because it is a static handler returning 200, say — makes the balancer blind: it keeps sending requests where there is nobody to serve them.

The model has no separate mode for such a readiness; the nearest one is the mode with no probes at all: 1600 slow answers. The balancer is equally blind in both cases, so the number is the same. The replica is alive and formally ready, and the traffic keeps arriving.

Hence a practical check worth naming in an interview: readiness must reflect the ability to serve a request, not the fact that a process is running. The minimal honest implementation checks what a request cannot be completed without: a database connection, room in a pool, the state of a task queue.

And the other side of it: the more checks inside readiness, the higher the chance it turns falsely negative and a replica leaves rotation over a trifle. So what gets checked is not "everything there is" but what this service cannot serve its requests without.

This is where the boundary of the rule belongs — the one that mirrors the boundary for liveness. From "dependencies live in readiness" it is tempting to conclude "so list every dependency in it", and that is a mistake too, just a different one. Readiness does not answer "is everything around me fine", it answers whether this replica can serve any traffic at all. A service that still answers some of its requests without the second database has to stay in rotation while that database is down: taking it out turns a partial degradation into a total outage — by your own hand rather than by the outage.

A readiness that lists every dependency also brings back exactly the disease the previous section cured. A shared dependency makes probes synchronous no matter which probe it is written into: every replica leaves rotation at the same moment because they all look at the same external object. The consequence is milder than with liveness — at least the replicas are not restarting — but the redundancy disappears just the same.

How to answer in an interview

Short answer: there are three probes and they differ by consequence — liveness restarts, readiness removes from rotation, startup postpones liveness for the duration of the boot. So the liveness question is not "is the process alive" but "would restarting this process help": only what a restart fixes goes inside it, and dependencies go into readiness.

That is enough to answer correctly. Beyond it is what you add when the interviewer digs.

If the interviewer digs deeper

Three things separate a good answer. First, you give the cost of the mistake as a number: in the model a liveness probe tied to the database added 200 failed requests on top of the 1000 the outage itself cost, because every replica restarted at once. Second, you explain why readiness is needed when liveness exists: a restart returns the replica to service again and again, while removing it from rotation stops the load — 80 slow answers against 520 in the model. Third, you mention the opposite mistake: a readiness that always answers "ready" makes the balancer blind, and that is the worst case the model gives — 1600.

And one thing that is easy to overdo. "Dependencies go into readiness" does not mean readiness should be the sum of every dependency. The precise phrasing is: readiness answers whether the replica can serve any traffic at all — and a replica that serves some of its requests without one of its dependencies should not leave rotation. Otherwise the probe itself turns a partial degradation into a total outage, and the shared dependency makes every replica synchronous again.

Next they ask

Common misconceptions

Practice

Two exercises. Answer first, then check against the real output: in both, the correct answer comes from a run of the model rather than being written by hand.

none = simulate("none")
liveness = simulate("liveness")
both = simulate("liveness+readiness")
on_db = simulate("liveness-on-db")
print(none["slow"])
print(both["slow"])
print(on_db["failed"] - liveness["failed"])

1600
80
200

1600
80
0

80
1600
200

1600
520
200

Knowledge check