Idempotency: exactly once is about the effect, not the delivery

What this question is really about

The ladder looks like this:

1. "What is idempotency?" — a warm-up on the definition.
2. "How do you make a retry safe?" — the substance starts here.
3. "What if the answer is lost after the effect has been applied?" — the central question of the topic.
4. "Is exactly-once delivery possible?" — a trap.
5. "Two retries arrived at once — what happens?" — a question about the order of working with the key.
6. "Who generates the key?" — the question that shows whether the client was thought about.

The numbers come from running bench/idempotency/keys.py and bench/idempotency/practice.py over loopback. An effect here is one increment of a counter inside the service: the thing that must not happen twice. That is what is counted — not the number of requests and not what the client saw.

Base: the server charged the money and the answer never arrived

Start with the story the whole topic grew out of.

A client asks a service to charge a hundred. The service charges it — the money is gone, everything worked — and sends back an answer. The answer never reaches the client: the connection broke, a timeout expired, a packet was lost. The client sees one thing: there is no answer.

What should it do? There are two options and both are bad. Not retrying: if the request never reached the service at all, the charge never happened and nobody will find out. Retrying: if the charge did happen, it happens a second time, and a hundred becomes two hundred.

The client cannot tell the two cases apart, and that is the crux of the whole topic. "The request never arrived" and "the request arrived, the effect was applied, the answer was lost" reach it in exactly the same shape — as the absence of an answer.

Since the fork cannot be resolved, it is stepped around: retrying is made safe. An operation is called idempotent when it can be repeated with no additional effect once it has succeeded a first time: the first application does the work, each later one does nothing and reports the same result. Then the client does not need to tell anything apart — it simply retries until it gets an answer.

Some operations are idempotent by themselves: "set the status to done" can be repeated as often as you like, the result is the same. Charging a hundred has no such property — two charges are two hundred. So the property has to be added from outside, and it is added with an idempotency key: the client invents a unique value for the operation and sends it with every attempt, and the service uses that value to recognise a retry and return the earlier answer instead of doing the work again.

And here is the question this lesson is about: what does the service have to do for that to actually work? "The service stores keys" is not yet an answer: what matters is what exactly it does with a key, and when.

That is already enough to answer the basic interview question. Everything below is about how many effects come out with a key and without one, about the window between the effect and the answer that a key does not close, and about who generates the key and when.

Mechanism 1: a retry is a second effect

Now the same thing in numbers, starting with what happens without any protection. Four identical requests — exactly what a client with retries from the retries-and-jitter lesson does:

1. A RETRY WITHOUT A KEY APPLIES THE EFFECT AGAIN
-------------------------------------------------
  requests sent                                  4
  times the effect was applied                   4
  what the client got each time                  ok #1 | ok #2 | ok #3 | ok #4

Four requests, four effects. And note the last line: the answers are different. That is an honest picture of what happened — from the server's point of view these were four separate operations, not one operation retried four times.

This is the root of the topic. A client that retries believes it is doing one thing; a server that does not distinguish retries does four. There is no agreement between them that this is one operation — it has to be created.

Mechanism 2: a key creates that agreement

The mechanism that fixes this we have already named: the idempotency key. There is no standard for it: what follows is a draft of the IETF httpapi working group whose term has expired, and it is quoted for its wording rather than as a norm. The wording is short and explains the whole mechanism:

Two phrases carry everything: generated by the client and recognize subsequent retries. Test it on the same four requests:

2. THE SAME RETRIES WITH AN IDEMPOTENCY KEY
-------------------------------------------
  requests sent                                  4
  times the effect was applied                   1
  what the client got each time                  ok #1 | ok #1 (replayed) | ok #1 (replayed) | ok #1 (replayed)

Four requests, one effect. And all four got the same answer — the one issued the first time.

The service did not become cleverer and guesses nothing. It does exactly what the definition says: on seeing a familiar key it returns the stored answer instead of doing the work a second time. The draft states that part too:

Note the "or an error". A retry returns the same result rather than trying to do better: if the operation failed the first time, it fails the same way the second. Otherwise the key would stop meaning "the same operation".

Mechanism 3: the window between the effect and the answer

Now the central question of the topic — the one the Base section opened with, only now visible in a measurement. The effect is applied on the server, the answer travels to the client — and between those two events there is a gap. What if the answer is lost inside it?

3. THE WINDOW: THE EFFECT HAPPENED, THE ANSWER DID NOT ARRIVE
-------------------------------------------------------------
  what the client saw on the first try           empty answer
  times the effect was applied by then           1
  what the client saw on the retry               ok #1 (replayed)
  times the effect was applied in total          1

Read it line by line. The line empty answer means zero bytes: the connection closed without bringing anything. For the client that is a failure. The effect had already been applied: the counter shows one.

And the client was right to retry. The run shows the second half of the picture: the effect happened, the answer is gone. The first half — "the request never arrived at all" — is not in the measurement, and that is exactly why the client is powerless: both halves reach it as the same empty result.

Hence the main conclusion of the lesson — and that is already reasoning rather than measurement. It is worth stating precisely: a side effect on the remote side and the client's knowledge that the answer was delivered cannot be tied together atomically — at least not without an additional protocol on top of a plain request and answer. However many acknowledgements you add to such an exchange, the last of them is itself a message that can go missing, so the gap between "done" and "the other side knows it is done" never disappears; it only moves. A key does not remove that gap: it makes a retry after it safe. The retry lands on the same key and gets the same answer, and the effect stays at one.

That is the exact meaning of "exactly once": what can be guaranteed is not the delivery but the effect. The delivery will be repeated by anyone, any number of times.

Mechanism 4: the client generates the key — once per operation

The fourth level is about who is responsible for the key and when it is born. The draft answers the first half directly:

The key is generated by the client, and that is the whole point. A server cannot derive the key from the request itself: by the time it has the request, there is nothing left to tell it apart from a retry. The quotation gives a definition rather than a prohibition: schemes where the server hands out a token in advance do exist — but there too the value reaches the server before the operation rather than together with it.

Here is what happens when a client makes a new key per attempt:

5. A NEW KEY PER ATTEMPT PROTECTS NOTHING
-----------------------------------------
  three attempts, one key: effects               1
  three attempts, a new key each: effects        3
  total effects applied                          4

Three attempts under one key: one effect. Three attempts under three keys: three. The service did not change between the halves at all; the third line of the block is their sum — the counter is shared by both halves: 1 + 3.

Hence the practical point this level gets asked about: a key is born with the intent, not with the attempt. A client that generates the key inside its retry loop has implemented the mechanism and gained nothing from it.

And hence why this cannot be solved on the server side. What counts as "the same operation" is known only to whoever intended it.

Deeper: the order of working with the key decides everything

Everything above describes what a key is for and where it comes from. The last level is about how the service handles it — and this is where an implementation most often turns out to be fake. A key by itself guarantees nothing; what matters is when it is written. Test it on two retries that arrive at the same moment:

4. TWO RETRIES AT THE SAME MOMENT, ONE KEY
------------------------------------------
  key claimed before the work                    1 effect from 2 requests
    answers                                      ok #1 | ok #1 (replayed)
  key written after the work                     2 effects from 2 requests
    answers                                      ok #1 | ok #2

The same key, the same two simultaneous requests, the same store. The only difference is the moment of writing — and it gives one effect against two.

Why. If the key is written after the work, there is a gap between the check "have we seen this key" and the write. Two requests both get through the check while neither has written yet — and the work is done twice.

Hence the rule worth answering with: the store has to claim the key, not remember it. The check and the write have to be one indivisible operation: an insert conditioned on "if this key does not exist yet", not "read, think, write".

In the run the indivisibility comes from a lock inside the process — which is enough, because both ends live in one. The rule about a conditional insert is the consequence: across several processes a lock would not help, and the store has to take the lock's role.

This is exactly where an implementation of idempotency most often turns out to be fake: the key is there, the store is there, and so is the race.

How to answer in an interview

Short answer: exactly-once delivery cannot be guaranteed, but the effect can. The client generates a key per operation, the server claims a place under that key before doing the work and stores the answer, and any retry gets the same answer instead of a second effect. Measured: four retries without a key, four effects; with a key, one.

That is enough for a correct answer. What follows is what you add when the interviewer digs.

If the interviewer digs deeper

Three things separate a good answer. First, you name the window between the effect and the answer and say precisely what is impossible in it: tying the effect on the remote side to the client's knowledge that the answer was delivered, atomically — that cannot be done without an additional protocol on top of the exchange. Idempotency does not remove that uncertainty; it makes a retry after it safe. Measured: the client sees a failure where the effect has already been applied. Second, you talk about order: the key has to be claimed before the work rather than written after it; measured, the second gives two effects on two simultaneous retries. Third, you say the key is generated by the client, and explain why: by the time a request reaches the server, nothing distinguishes it from a retry.

What not to say: "we did exactly-once". That is a promise about delivery that nobody can make; the promise to make is about the effect — and that one can be checked.

Next they ask

Common misconceptions

Practice

Two exercises. Answer first, then check against the real output: in both, the correct answer is what the measurement script prints.

without_key = retries(use_keys=False)
with_key = retries(use_keys=True)
late_key = simultaneous(claim_key=False)
print(without_key)
print(with_key)
print(late_key)

4
1
2

4
1
1

4
4
2

1
1
1

Knowledge check