Two ways to build Valkey out of several nodes: Cluster and Sentinel

Who knows where to send the command

The whole difference follows from here. In a cluster every data node knows the map: come to the wrong one and it will not refuse, it will name the right one. Under Sentinel a data node knows nothing about the cluster topology — it knows only its own replication pair — and a separate service has to be asked.

cluster node:       SET foo 1  ->  MOVED 12182 127.0.0.1:48711
node under sentinel: CLUSTER INFO -> ERR This instance has cluster support disabled
sentinel:           SENTINEL get-master-addr-by-name cache -> 127.0.0.1:41979

Key, slot, node

Addressing takes two steps. The first is arithmetic: HASH_SLOT = CRC16(key) mod 16384. The second is cluster state: the slot belongs to someone. That is why moving data renames no key — it changes a slot's owner.

Braces in a key name are not decoration: only what is inside them is hashed. {u}:a and {u}:b land in one slot and end up on one node.

What breaks in a cluster

The refusal is not about "two writes at once" but about two different slots: a command runs entirely on one node. The set of keys you will ever touch with one command is decided before the first write.

Numbered databases: here the version decides

The claim "a cluster has exactly one database" is easiest to check with SELECT 1 → ERR DB index is out of range. The answer is real, the conclusion is not: that is the rule of Redis and of Valkey before 9.0.

cluster A: cluster-databases 1     SELECT 1 -> ERR DB index is out of range
cluster B: cluster-databases 16    SELECT 1 -> OK

Two clusters, one line of difference. The database number takes no part in addressing: the same key holds different values in db 0 and db 1, but its slot is the same one — 12182.

What databases in a cluster do NOT give you: SWAPDB stays forbidden, and DBSIZE, SCAN and FLUSHDB see only part of a database — the part on the node the command reached. Three keys in one database spread across three nodes, and each node counted one.

A primary fails

Comparing durations is pointless — they are configurable. Compare what the client received.

in the cluster, polling a live node:
  t= 0.00  MOVED 12182 127.0.0.1:36355     (the slot's old address)
  t= 3.39  CLUSTERDOWN The cluster is down (the slot is covered by nobody)
  t= 3.49  MOVED 12182 127.0.0.1:39129     (the new address)

under sentinel, polling the old address:
  t= 0.00  Could not connect to Valkey at 127.0.0.1:44393
  t= 2.30  the sentinel names 127.0.0.1:41979 as the primary

In the cluster the client kept getting meaningful answers from a live node. Under Sentinel nobody told it the new address — it has to ask.

The most dangerous place

The old primary came back — and came back as a primary: that is what it remembers being.

  t= 0.06  role master, write -> OK
  t=10.10  role slave,  write -> READONLY You can't write against a read only replica.

  right after demotion: GET who   old
  after the resync:     GET who   the key is gone

For ten seconds writes were accepted, then erased by the resync with the real primary. The client saw no error. In this experiment the cluster had no such window: the returned node answers MOVED to the new address immediately.

Hence the rule in the Sentinel client spec: re-resolve the address on every reconnection, not once at startup.

Which one to pick

one node is not enough memory or throughput   → cluster
fault tolerance only, data fits one node      → Sentinel
code leans on MSET over arbitrary keys        → Sentinel is simpler
you need numbered databases                   → Valkey 9+: either one
you need SWAPDB                               → Sentinel only
the client cannot speak the cluster protocol  → no cluster for you
the client cannot speak sentinel              → no Sentinel for you

The last two lines matter: both modes require their own support in the client, and the two abilities are independent. An ordinary client that knows nothing of clusters will, on receiving MOVED, simply hand the error text to the application — it keeps no slot map.

What measured this

Everything was taken on Valkey 9.1.2 built from source, on one host. The scripts and recorded runs are in bench/valkey-cluster/.

The seconds carry nowhere: a node counts as down after one second of silence instead of the default thirty. The verbatim server replies carry over completely — they are behaviour, not measurement.

And one thing carries worst of all: the version. Numbered databases in a cluster arrived in 9.0; atomic slot migration in Valkey 9.0 and in Redis 8.4. Rules about clusters taken from knowledge of Redis have to be tagged with a version.

One question, answered by two different parties

Nearly every comparison of Cluster and Sentinel starts from a table reading "sharding: yes / sharding: no". That is true, but it is a consequence rather than a cause, and none of the practical differences follow from it — not why MSET stops working, not why the client needs a special library, not where the window comes from in which writes go nowhere.

Four things separate the two modes at once, and conflating them is expensive:

DATA LAYOUT     a cluster shards across 16,384 slots; Sentinel does not,
                all the data sits on one primary
WHO DECIDES     cluster — the nodes themselves, by a majority of primaries;
                Sentinel — a separate service, by a majority of watchers
WHO ADDRESSES   cluster — the data node, with an ordinary command;
                Sentinel — that same service, with a command of its own
WHAT THE CLIENT each mode needs its own support in the library,
CAN DO          and those two abilities are independent

The first line gives rise to the set of usable commands, the second to the behaviour under failure, the third to the connection protocol, the fourth to whether you can take the mode at all with your library. They are taken in turn below, and the third is the place to start: it explains fastest why the two modes get confused in the first place.

So: the client has a question — where do I send this command. The two modes answer it from different sides.

The rest of the article is consequences of that fork, and each of them is checked by a run rather than argued. Arguing them would not work anyway: the data layout and the version add conditions of their own, and the single question "who answers with an address" does not explain every difference.

How a key becomes an address

Cluster addressing takes two steps, and it is worth holding both in mind because it is precisely these two that get conflated. The first step, from key to slot, is pure arithmetic:

The second step, from slot to node, is not arithmetic but cluster state: slots belong to someone, and that ownership changes. Hence a property that is otherwise hard to explain: moving data between nodes renames no key. The slot changes owner; the formula stays the same.

This is checked with one command sent to each of six nodes in turn. The client has no -c flag — it is deliberately "dumb" and follows nothing, so that what the server actually said is visible:

MOVED is a protocol error — a redirection error: the command was not executed, and the node returned not data but the address of whoever will execute it. It does not usually become an application failure, though: a cluster client repairs its slot map on such an answer and retries by address. Note that primaries and replicas answered alike: all of them know the map.

The difference between "not executed and redirected" and "not executed, outcome unknown" matters here, and it should be kept apart from a retry after a network failure. On MOVED a retry is safe: the node definitely did nothing. If the connection broke after the command was sent, the client does not know whether it ran — and there the retry is a question of idempotence, not of routing.

What a real client actually does

The block above shows the protocol but not the behaviour: a real library hides MOVED. What it does is counted by the nodes themselves — from their own processed-command counters, with a control window of equal length and no client (cluster nodes talk to each other constantly, and without a control the client's traffic cannot be told from the background).

The client was given one address out of six, and not the one holding the key:

One trip for the slot map (cluster|slots) — and the write went straight to the right node. Hence what people usually mean by "the cluster sorts itself out": the client need not know every node, one live address is enough, and there is no middleman between the client and the data.

Under Sentinel the order is reversed, and the same measurement shows it: first the question to the service, then the command to the data. The data node took no part in finding itself, and could not have:

One formulation should be narrowed right here, because it suggests itself in a form wider than what was measured. "A data node knows nothing about the topology" is wrong: it knows its own replication pair perfectly well.

What it does not know is the authoritative answer to "who is this service's primary right now". It has an opinion, and during a failover that opinion is sometimes stale; that is exactly what happens in the section on the returning address.

The Sentinel client spec describes the required sequence in plain words — and it has three steps, not one:

The third step — verifying with ROLE — looks like belt and braces right up to the section about the returning address. There it becomes clear what it guards against.

And what it does not give: it is a check, not a proof. If both the service's answer and the node's own opinion are stale — and that is precisely the pair you get in the first seconds after a failover — ROLE will answer master, and the client will take the old address for the right one. The spec calls this case rare and relies on the sentinels tearing such connections down later; it promises no consistency.

What breaks in a cluster

Out of this grows the list of commands that stop working. The list is not arbitrary: a command runs entirely on one node, so a command needing two keys from different slots cannot run anywhere.

The refusal is not about "two writes at once" but about two different slots: the same two writes with a shared brace go through. Braces are not decoration in a name:

The practical consequence is costliest in hindsight: in a cluster the set of keys you will ever need to touch with one command is decided before the first write. Under Sentinel that decision need not be made at all.

The second item on the list is numbered databases — and this is where the article got it wrong. The mistake is instructive, so it is dealt with not by swapping a line but in a section of its own.

Numbered databases: here the answer depends on the Valkey version

The claim "a cluster has exactly one database" is easiest to check with one experiment:

— and drew from it the conclusion "a cluster has exactly one database". The server's answer is real, the conclusion is not. The only correct conclusion from that single experiment is: THIS cluster is configured with one database. Telling the two readings apart needed a second experiment, and there wasn't one.

Because in Valkey 9.0 numbered databases in a cluster appeared:

The number of databases is set by a separate option, cluster-databases, and it defaults to one — which is why a cluster brought up the ordinary way behaves the old way. Two clusters differing by exactly that line:

One config line — and SELECT 1 turns from a refusal into OK. The option is immutable (IMMUTABLE_CONFIG in src/config.c), so it is chosen when the node starts and cannot be changed on the fly.

The database number takes no part in addressing

That is the second question, and it is the more interesting one: does the new feature break the two-step addressing the whole cluster rests on. It does not.

One key, two values, one slot. The database is added as a third dimension inside the node, and the layout of data across nodes does not depend on it by a single key.

And what it does NOT give you

From "databases exist now" it is just as easy to build the opposite myth — that in a cluster they behave as they do on a single node. They do not:

SWAPDB was forbidden and stayed forbidden. And DBSIZE and SCAN count not the database as a whole but its part on the node the command reached: three keys were written into one database, spread across three nodes, and no node sees all three. The same goes for FLUSHDB. The documentation says so directly:

From the same page, on what numbered databases should not be taken for:

The upshot: a numbered database in a cluster is a namespace, not a separate store. Splitting environments across them is fine; expecting isolation, or a cluster-wide view of their contents, is not.

Hence the rule worth taking away from the article as a whole

The error here is not carelessness. It is that the knowledge about clusters came from where everyone takes it: shared memory of Redis Cluster, where there really is one database. On Valkey 9 that rule stopped being true — and still looks true, because the default value confirms it.

The rule is simple: any statement about clusters inherited from knowledge of Redis has to be tagged with a version — or checked. Checking it here cost one config line and two minutes.

A primary fails: one event, two client experiences

Here is the main practical difference, and it is not in the seconds. Comparing durations is pointless: both are configurable, and in the benchmark both were deliberately shortened so the run would fit in minutes. What should be compared is the text the client receives.

In the cluster a live node was polled — not the one that died:

Three different answers, all three meaningful: the slot's old address, an honest "the slot is covered by nobody right now", the new address. The client never had to guess — the node it reached knows the state of the whole cluster, because nodes exchange it continuously over the bus. And no external service was consulted: the replica was promoted by the nodes themselves, by a majority of primaries.

Under Sentinel the same event looks different:

The old address answered nothing — there is nobody there. No data node named the new address to the client and none could: the nodes do not know about their pair's service identity, and the service that does know does not call. Hence the rule in the client spec:

Not "preferably", but every time. Why, is the next section.

The old primary comes back

This is the most dangerous place in Sentinel mode, and it is about recovery rather than failure. The killed primary is brought back — and comes back as a primary: that is what it remembers being, and nobody has told it otherwise yet.

For ten seconds the node accepted writes and answered OK. The last two rows read together and in that order: right after demotion the write is still there, and after the resync the key is gone — the data was replaced with a copy of the real primary. The client got OK, and the write vanished, without seeing a single error.

In this experiment no such window appeared in the cluster, and that is checked by the same run — the last two rows of the cluster table above. The returned node learned from the bus that its slots belong to another before it took a single command.

"In this experiment" is not politeness. One scenario was checked — killed and brought back up; on start the node read the bus and got a fresh configuration before the client's first command. A network partition is a separate case in the cluster specification, and there a bounded loss of writes is admitted: a client with a stale map writes to a former primary that has not yet learned of its demotion. We did not measure that, and the article will not claim the cluster has no such window at all. What was measured is something else: in a cluster that window closes by itself, and under Sentinel only by the client's own actions.

This is what the ROLE check in the client spec guards against. A client that holds a data address directly and cannot re-ask the sentinel falls into that window whole.

Found along the way: ready to work ≠ ready to fail

This section came out of a bug in the benchmark. The script brought up a cluster, waited for cluster_state:ok on every node and killed a primary. The replica was never promoted — not in twenty seconds, not in a minute, never.

It turned out not to be a bug in the benchmark:

Inside those six seconds the cluster looks entirely healthy: slots are covered, keys are written and read. But the replicas have not completed a single sync — and a primary that dies here is never replaced:

Currently unable to failover: Disconnected from primary for longer than
allowed. Please check the 'cluster-replica-validity-factor' configuration
option.

Why "never" rather than "until some timeout expires" is visible in the source:

if (server.repl_state == REPL_STATE_CONNECTED) {
    data_age = (mstime_t)(server.unixtime - server.primary->last_interaction) * 1000;
} else {
    data_age = (mstime_t)(server.unixtime - server.repl_down_since) * 1000;
}

For a replica that has never connected to its primary, repl_down_since is zero. Its data age comes out measured from the Unix epoch, and no allowance covers an age like that.

That "never" needs narrowing, and it is narrowed by measurement rather than by a disclaimer. The freshness check on a replica's data is governed by cluster-replica-validity-factor; in the experiment above it holds its default. The same experiment with zero:

One config line — and "never" turns into four seconds to promotion. So the correct formulation is: at the default value, a replica that has completed no sync is never promoted; an option turns that off. No advice to do so follows: the check is there for a reason, and a replica that has never synced is promoted empty — a failover of that kind cures availability at the cost of data.

The conclusion is not about Valkey but about operations: "the cluster is up" and "the cluster survives a failure" are different checks. The first answers whether requests are served; the second requires looking at master_link_status on every replica. Automation that rolls out a cluster and immediately tears down the old one passes the first and fails the second.

Quorum and majority are different numbers

In sentinel monitor <name> <ip> <port> <quorum> the last number is read as "how many sentinels suffice to fail over". That is wrong, and the documentation says so directly:

The numbers were chosen so that one condition holds and the other does not: five sentinels, quorum 2, three killed. The quorum is satisfied — and there is no failover:

+sdown master cache 127.0.0.1 48147
+odown master cache 127.0.0.1 48147 #quorum 2/2
+new-epoch 1
+try-failover master cache 127.0.0.1 48147
+vote-for-leader c92cd70b21a9dd26ec40e67185371e78ee502f9a 1
-failover-abort-not-elected master cache 127.0.0.1 48147

#quorum 2/2 — the configured condition is met and the primary is declared down. And right after it -failover-abort-not-elected: two votes out of five were not enough to become leader. Bringing back one of the killed sentinels gave a majority, and the failover completed in two seconds — at the very same quorum.

None of this has to be discovered during the incident. One command answers in advance, and it checks both conditions:

The refusal names the second condition in so many words: there are not enough watchers to reach a majority and authorize a failover — while the configured quorum is met. That makes it a ready-made monitoring signal: it answers "will we fail over if the primary dies right now", not "is everything fine at this moment".

A practical rule that does not follow from one setting: the system will not survive losing a majority of its sentinels, however low the quorum. Hence the minimum in the documentation:

How this differs from Redis

A fair question: Valkey inherited both mechanisms from Redis whole, and up to a point this is literally the same code. The boundary is named inside Valkey itself, in the same file as the version number:

#define VALKEY_VERSION "9.1.2"
 
/* Redis OSS compatibility version, should never
 * exceed 7.2.x. */
#define REDIS_VERSION "7.2.4"

So Valkey declares compatibility up to 7.2.4 — the version at which the projects parted.

That line cannot be used as proof, though, and it is worth saying so plainly. It means exactly one thing: a shared past up to 7.2.4. It does not follow that everything before that point still matches today, nor that everything after it diverged. Both sides have since changed things that used to be common. The only honest way to compare is to take a capability and read the current documentation of both projects; that is what is done below, and the result was not what the reasoning "it arrived in Valkey, so Redis has not got it" leads to.

The first example is atomic slot migration, introduced in Valkey 9.0.

What matters here for the reader: the old way of moving slots is the very one that keeps ASK redirection alive in the specification, along with the warning that multi-key operations may answer TRYAGAIN while a slot is on the move. Atomic migration shortens that phase.

But the conclusion "and Redis has nothing of the kind", which suggests itself here, is wrong. Redis Open Source 8.4 gained atomic migration too, under a command of its own:

So as of today both have the capability, while the commands and the protocol differ: in Valkey it is CLUSTER MIGRATESLOTS plus the internal CLUSTER SYNCSLOTS, in Redis CLUSTER MIGRATION. Operationally that is a more practical difference than "present or absent": migration procedures and the tooling around them do not carry between the projects, even where the capability goes by the same name.

Everything else in this article — slots, MOVED, CROSSSLOT, the shape of Sentinel, quorum against majority — came to both projects from a shared past. None of it was re-checked here on Redis, and the article will not claim "Redis does exactly the same": not measured, not stated. That is precisely what undoes that kind of claim.

Which one to pick

The analysis is short, and it is not about throughput.

you need more memory or throughput than one node gives
    → native cluster: it is the only one of the two that shards

the data fits one node and you need fault tolerance only
    → Sentinel stays a candidate

the code leans on multi-key commands over arbitrary keys
    → Sentinel is simpler; in a cluster the keys must be pulled
      into one slot with braces

you need numbered databases
    → Valkey before 9: Sentinel only
    → Valkey 9+: a cluster too, via cluster-databases

you need SWAPDB
    → Sentinel only: in a cluster it is forbidden in 9.x as well

does the client speak the cluster protocol?
    → no: a cluster is out of reach without replacing the client or a proxy

does the client speak sentinel?
    → no: Sentinel is out of reach without replacing the client or a proxy

The last two lines deserve a careful reading: the received "the client can't speak sentinel → take a cluster" leads up to the idea that a cluster forgives a naive client. It does not. An ordinary client that knows nothing of clusters, on receiving MOVED, will simply hand the error text to the application: it keeps no slot map, cannot retry by address, does not know ASK and ASKING, does not handle TRYAGAIN. Both modes require their own support in the client, and the two abilities are independent: a library may have one, the other, both or neither. With neither, nothing works, and the question moves from choosing a mode to choosing a library or a proxy.

What is true and stayed true: a cluster client needs one live address to get the map. In production several are listed anyway — in case that particular node is unreachable at startup.

And one more caveat about multi-key commands that was missing from the first version. The CROSSSLOT refusal is a server contract. Some client libraries offer a convenient multi-key call on top of it, splitting the request into several commands by node. That works as a wrapper, not as the same command: it has no atomicity, its failure model is different (some sub-requests may succeed and others not), and the latency is the sum of several round trips.

What should reproduce, and what should not

The numbers were taken on Valkey 9.1.2 (built from source — the README in bench/valkey-cluster/ says how), on one host, processes on random ports. The scripts and the recorded runs open from here: bench/valkey-cluster/.

The seconds carry nowhere. A node counts as down after one second of silence instead of the default thirty, and cluster-node-timeout is 2000 ms instead of 15,000. That was done so the set would fit in minutes. The ten seconds of the recovery block and the 6.07 seconds of the readiness block will differ on your settings — but that both windows exist does not depend on settings.

The verbatim server replies carry over completely. MOVED, CROSSSLOT, ERR DB index is out of range, READONLY, -failover-abort-not-elected are behaviour rather than measurement, and on the same version they will be the same.

Across versions, not necessarily. Everything was taken on 9.1.2; these runs were repeated neither on 8.x nor on Redis. And the numbered-databases section above is the article's own demonstration of why that caveat is not boilerplate.