Page cache and fsync: why "written" and "will not be lost" are different claims

What is actually being asked

The ladder usually runs like this:

1. "What does write do?" — the warm-up, where it becomes clear whether the candidate believes the data went to the disk.
2. "Is the data lost if the process crashes right after write?" — where the substance begins.
3. "And if the machine goes down?" — the same question with a different answer, and the difference explains the whole mechanism.
4. "How does fsync differ from fdatasync?" — whether you know they differ over metadata.
5. "Why does a database not fsync on every write?" — a question about cost.
6. "Is fsync on the file enough for the file to exist?" — the question where a file's content and its name come apart.

The numbers come from running bench/durability/fsync.py and bench/durability/practice.py. Timings depend heavily on the disk: yours will differ. The observations do not depend on the machine: what survives a process crash, and what each call promises.

Base: where the data is once the write returned success

Before talking about caches and flushes it is worth naming, in ordinary words, the path the data travels. It has four stretches:

1. the application called write — it handed the kernel a chunk of memory and said which file to put it in;
2. the kernel's memory — the data is accepted and sits in an area the kernel owns; the call returns at this point;
3. the filesystem — it decides where on the device those bytes will land and which bookkeeping records have to change with them;
4. the storage device — the place where bytes stay when there is no power.

The key question is which stretch the write call ends on. It ends on the second, not the fourth: the application gets success when the kernel took the data, not when the disk did. From there the kernel carries it onward — by itself, at its own pace — and the application neither waits for that nor hears about it.

And here is the question this lesson is about: the write returned success — where is the data really? Everything else follows from the answer: what happens to it if the process dies right now, what happens if the power goes, and what it costs to demand that the data already be on the device.

The short answer: the data is in the kernel's memory. That memory is shared — it does not belong to the process that wrote it and outlives that process's death. But it is also the memory that goes away with the power, so "the data was accepted" and "the data is durable" are two different statements, and the second one has to be asked for by a separate call.

That is already enough to answer the basic interview question. Everything below is about what that separate call costs, how its two forms differ, and why databases do not demand an acknowledgement per write.

Mechanism 1: write carries data to the cache, not to the disk

Start with what write(2) says about itself:

Note the second half: an error may arrive later, and it may arrive in close. That is the first practical consequence: code that ignores the result of close loses write errors silently.

What write actually does is move the data into the page cache, the cache of file pages living in the kernel's memory. From there the kernel's writeback will put them on the disk at some point; when exactly, the application neither knows nor decides.

And since the cache is the kernel's memory, "closed the file, therefore saved it" rests on nothing either, which close(2) states separately:

Mechanism 2: a process crash and a power loss are different events

The most useful question in the topic: if the data is not on the disk yet, what loses it? Check it the crudest way available. A process writes to a file, never calls fsync — and receives SIGKILL, dying without a single line of cleanup:

2. WHOSE MEMORY THE PAGE CACHE IS
---------------------------------
  child wrote without fsync, then SIGKILL      exit -9
  file content after the kill                  survived the kill

exit -9 is how subprocess reports a death by signal: the negative number is the signal number itself. A shell in the same place would print 137, that is 128 + 9 (the lesson on signals).

The data is there. What is more, another process sees it: bench/durability/practice.py prints the file twice — once read back after the writer died, once from an unrelated process — and both lines are the same, because reads go through the same cache.

Hence the distinction worth answering with:

• A process crash loses nothing that made it into write. The cache outlives a crash, a kill and the OOM killer alike.
• A power loss is another matter: data that has not become durable yet may be lost there, because the memory the cache lived in goes off with the machine.

Note the "may be". There is no guarantee that unflushed data survives; there is no guarantee that it is certainly gone either — the kernel's writeback may have carried some of those pages to the device already, and the application has no idea which ones. Nothing can be planned on that: from the application's point of view, everything unflushed is data whose fate after a power cut is simply unknown.

That is why "we are fine, the application just restarted" is a legitimate sentence and "we are fine, the machine just rebooted" is not.

Mechanism 3: what fsync promises and what it costs

The promise is written in fsync(2):

The price of that promise is the lesson's main measurement:

1. THE PRICE OF DURABILITY, PER WRITE
-------------------------------------
  write only (4 KiB), median                   1.4 us
  write + fdatasync, median                    142.9 us
  write + fsync, median                        143.9 us
  fsync over plain write                       104x
  gap between fsync and fdatasync              0.9 us
  run-to-run spread of the same mode           51.1 us
  difference resolvable on this machine        no

This block comes from the bench/durability/fsync.py run. The exercises below quote a second run, bench/durability/practice.py, where the same pair came out at 1.8 and 164.7 us and the ratio at 90.1. Two runs on one machine diverge exactly as the run-to-run spread line promises: what carries meaning is the order of the ratio, not its digits.

Two things in that block matter in different ways.

The ratio carries meaning. A hundredfold: that is what separates "the data is in kernel memory" from "the data is on the device". It is not a Linux quirk or an oversight but physics: the second case has to wait for a real device to acknowledge the write.

The gap between fsync and fdatasync is not resolvable here, and the script says so itself. A gap of 0.9 us against a spread of 51.1 us between repeats of the same mode — the script measures each mode five times over, which is the run-to-run spread of the same mode line. No ratio may be claimed from that, and the block's last line, difference resolvable on this machine, says no. The difference between the calls is real and documented:

So fdatasync saves the metadata write — and the saving shows up where metadata is expensive: when a file grows, on filesystems that journal metadata, on slow devices. On this container's virtual disk it drowned in the spread, and the honest answer here is "not measured", not "the same".

Mechanism 4: why a database does not fsync on every write

The ratio from the previous block is the price of one promise. What follows is arithmetic: how many times to demand it.

3. ONE FSYNC FOR MANY RECORDS, OR ONE EACH
------------------------------------------
  100 records, fsync after each                17.2 ms
  100 records, one fsync at the end            1.0 ms
  ratio                                        17x

Same bytes, same disk, and after the last line the data is equally durable in both cases. What differs is the number of points at which acknowledgement was demanded.

Hence group commit: transactions arriving at almost the same time wait for a shared flush. Each waits longer than it would alone, but one latency serves the whole group rather than each transaction separately.

And hence the answer to "why not just fsync every request": you can, and it will be correct. The cost is that ratio multiplied by the number of requests, and it turns into a ceiling on throughput.

Deeper: a file's name does not live in the file

The last step is about the place where the durability of the content and of the name come apart. You created a file, wrote to it, called fsync, and the machine went down. Is the file guaranteed to exist?

fsync(2) answers without ambiguity:

4. WHAT FSYNC IS FOR: THE FILE, NOT THE DIRECTORY
-------------------------------------------------
  file synced                                  yes
  directory sync cost                          0.11 ms

The measurement here does not answer whether the directory needs its own flush — the man page does — but what it costs.

A file's content and the fact that it exists under that name are two different objects, flushed separately. Hence the sequence for an atomic file replacement, which is what gets asked: write to a temporary file, flush it, rename it over the target, flush the directory. Skipping the last step gives you code that works right up to the first power cut, and then loses precisely what it thought it had written.

How to answer in an interview

Short answer: write carries data only as far as the kernel's cache and returns; durability is promised by fsync, and it costs two orders of magnitude more. That is why a process crash loses nothing written — the cache belongs to the kernel — while a power loss may lose everything unflushed.

That is enough for a correct answer. What follows is what you add when the interviewer digs.

If the interviewer digs deeper

Three things separate a good answer. First, you distinguish the two failures, a dying process and a power cut, and say why the answers differ. Second, you give the cost as a number and know what follows from it: group commit exists not for elegance but because flushing on every write runs into the device. Third, you remember the directory: fsync on a file makes the content durable, not the name, and an atomic replacement has to flush both.

And one piece of care in phrasing that gets noticed. "A machine crash loses the data" is too strong a claim; the correct form is that data which has not become durable yet may be lost when the power goes. The kernel's writeback may have carried part of it to the device already, and which part is unknown to the application. The practical conclusion does not change, but the statement becomes true: unflushed data cannot be counted on in either direction.

Next they ask

Common misconceptions

Practice

Two exercises. Answer first, then check against the real output: in both, the correct answer comes from a script's committed output rather than being written by hand.

print(content_after_kill(path) or "(empty)")
print(seen_by_another_process(path) or "(empty)")

survived the kill
survived the kill

survived the kill
(empty)

(empty)
(empty)

(empty)
survived the kill

Knowledge check