cgroups and the OOM killer: who gets killed, why the log is empty, and what 137 really means

What is actually being asked

The ladder usually runs like this, and the first two rungs matter more than they look:

1. "What are cgroups?" — the warm-up: grouping, accounting, limits.
2. "What happens when a process hits its memory limit?" — where everyone expecting a MemoryError is filtered out.
3. "Why is there nothing in the application log?" — whether you connected this to SIGKILL from the previous lesson.
4. "Who gets killed when a container runs several processes?" — whether you know the badness score and what it is built on.
5. "How do you tell an OOM kill from an orchestrator's kill if the code is the same?" — a question about evidence, not guesswork.
6. "How do you choose a limit?" — whether you see that failcnt and peak usage answer different questions.

The numbers below come from running bench/cgroups/oom.py and bench/cgroups/practice.py on cgroup v1. In v2 the same quantities have other names (memory.max, memory.events, memory.peak) — the mechanism is the same, the spelling is not.

Base: a container has a memory budget, and it is not the application that counts it

Before taking apart who gets killed and what for, three things are worth naming in ordinary words.

A control group (cgroup) is a Linux kernel mechanism that accounts for the resources of a group of processes and limits them. Not one process but a group: the kernel keeps counters for it — how much memory is held right now, how many times the group has hit its ceiling — and holds the number that bounds that usage.

A container is exactly such a group. When its settings say "512 megabytes of memory", that number goes into the group's file, and from then on the kernel is what enforces it — not the application, not the runtime, not a library.

The processes inside share one budget. There may be one of them or several: the main one, a helper, a child started by accident. All of them take memory from the same number, which is why "my process is small" guarantees nothing — what is counted is not it but all of them together.

Hence the question the rest of the lesson answers: what happens when the group's processes need more memory than the group is allowed? The answer is short and uncomfortable: the kernel does not refuse, it either frees or kills. First it tries to take back, inside that same group, whatever can be taken, and hand the memory over after all; when there is nothing left to take, it picks one of the group's processes and ends it. No notice, no request to shut down, no last word — it ends it.

That is already enough to answer the basic interview question. What follows is about who exactly gets picked, why the application log is empty afterwards, and what evidence separates this death from an ordinary container stop.

Mechanism 1: the limit belongs to the group, not the process

Now the same thing in the documentation's words. A control group is a set of processes plus accounting of what they consume plus limits on that consumption. On memory, cgroups(7) is brief:

The key word is cgroups, plural: both the accounting and the limit belong to the group. Two consequences follow, and each gets asked about separately.

First, if a container runs several processes, they spend a shared limit. A helper process that "only takes a hundred megabytes" takes those hundred megabytes away from the main one.

Second, the kill is an event of the group rather than of a process. The kernel notices that the group has nowhere to grow and picks which of its members to kill. The one it picks need not have done anything wrong.

The group exposes three files: the limit, the peak usage, and a counter of refused charges (failcnt). The script creates a group, sets a 64 MiB limit and runs a program in it that asks for 1600 MiB and touches every page it gets:

1. THE KILL: the limit is 64 MiB, the program asks for 1600
-----------------------------------------------------------
  limit (memory.limit_in_bytes)            64 MiB
  exit code from waitpid                   -9
  as the shell reports it                  137
  stdout of the program                    ''
  stderr of the program                    ''
  peak usage before the kill               64.0 MiB
  times the limit was hit (failcnt)        36

The rest of the lesson takes those seven lines one at a time, because each answers a different rung of the ladder.

Mechanism 2: usage stops at the limit rather than growing through it

The most common wrong picture is a process that "swelled past its limit and burst". Look at the peak: 64.0 MiB against a 64 MiB limit. In this run usage climbed to the limit and did not go past it.

The boundary of that observation is worth naming right away, because the practice below rests on it. "The peak equals the limit" is what we measured, not something the kernel promises: cgroup v2 documents memory.max as a limit at which usage may temporarily go higher under some circumstances. So the rule is written from the cause rather than from the precision of a number: the kill comes not after a process outgrows the limit but because there was nothing left to free inside the group.

The program in the run does not merely ask for memory, it touches every page — and what reaches the group's counters is what is held, not what was asked for: 1600 MiB requested, a group peak of 64.0 MiB. When the next page would not fit inside the limit, the kernel simply does not hand it over. From there two paths exist. The ordinary one is to free something inside the same group and hand the memory over after all. The extreme one, when there is nothing left to free, is to kill.

The first path is what failcnt counts — that same line from the block above:

  times the limit was hit (failcnt)        36

Thirty-six times the group hit its ceiling, and up to the last of them it carried on working. The program never knew: from where it stood, memory was simply granted. Hence a practical conclusion worth saying out loud in an interview: a non-zero failcnt is not a crash report, it is a sign that the group lives at its boundary. It still works, but every further request is served through reclaiming, which is to say more slowly.

Now the two empty lines in the same block:

  stdout of the program                    ''
  stderr of the program                    ''

No MemoryError, no traceback, no final message. The reason is known from the previous lesson and written in signal(7): SIGKILL cannot be caught, blocked or ignored. The program was not asked to stop; it was removed.

So the third rung has its answer: an empty log is not lost messages, it is a sign of how the process died. Memory exhausted inside the process would leave an exception and a traceback; a SIGTERM would leave whatever the handler wrote. Empty is what SIGKILL leaves.

Mechanism 3: the victim is chosen by memory held

Now the question the ladder was built for: with several processes in a group, who gets killed? The answer is in proc(5), and it is not about ordering:

And separately, what counts as "allowed memory" in a container's case:

So the score is the share of the group's limit a process occupies. Check it: a group limited to 160 MiB with two processes in it. The small one takes 40 MiB and holds them; the big one asks for 600.

3. WHO GETS KILLED: the biggest, not the last one
-------------------------------------------------
  small process (40 MiB, holding it)       alive
  big process (asked for 600 MiB)          killed
  victim                                   big

That is good news and bad news at once. Good: a well-behaved neighbour usually survives someone else's mistake. Bad: "usually" is not "always". The score is computed from the share held, and when two processes hold about the same, small things decide the outcome.

The choice can be nudged by hand — oom_score_adj is added to the score, and its range is documented:

Use it carefully: a protected process does not stop consuming memory, it stops being a candidate. The memory still runs out and the kernel kills somebody else — possibly the one the protected process cannot work without.

Mechanism 4: the evidence that separates OOM from a stop

Exit code 137 is the same for a process killed by the OOM killer and for one finished off by somebody else's SIGKILL — the one that arrives at the end of a grace period when a container is stopped, as in the previous lesson. Both are 128 + 9. What separates them is not the code but the kernel's own record — the second block of the run:

2. THE KERNEL SAYS IT, THE APPLICATION DOES NOT
-----------------------------------------------
  constraint reported by the kernel        CONSTRAINT_MEMCG
  cgroup that ran out                      /de_oom_kill
  victim named by the kernel               python3

The constraint field is what says where memory ran out. In the run the kernel named a group: CONSTRAINT_MEMCG — so the group's limit is the cause. Had the line named something other than a group, the cure would be entirely different: the machine's memory rather than the container's limit.

So here is the check, and it is worth giving in this order: an OOM kill has a witness, and the witness is the kernel. Three questions in a row — is there an oom-kill:constraint= line near that moment; which group does it name; and does the timing coincide with an orchestrator event? If you could read dmesg and the line is not there, there are no grounds for calling it an OOM kill: the search moves to whoever may send signals.

Deeper: the same code, a different limit, a different outcome

The fourth block of the run is the same code on the same machine with the same request for 1600 MiB, under a 2048 MiB limit:

4. THE SAME PROGRAM AND THE SAME REQUEST, A LIMIT THAT FITS
-----------------------------------------------------------
  limit                                    2048 MiB
  exit code                                0
  stdout of the program                    allocated 1600 MiB
  times the limit was hit (failcnt)        0

Zero in failcnt and zero as the exit code. Nothing in the program changed — the permission did.

The fifth block answers the question that follows from it: is all memory equally dangerous? A 256 MiB file — four times the limit — was read through a group limited to 64 MiB:

5. PAGE CACHE COUNTS TOO, BUT IT CAN BE GIVEN BACK
--------------------------------------------------
  limit                                    64 MiB
  page cache dropped before the read       yes
  file read through the group              256 MiB
  exit code                                0
  stdout of the program                    read 256 MiB
  peak usage                               63.2 MiB
  times the limit was hit (failcnt)        95

Ninety-five hits against the ceiling, a peak right at it — and exit code zero. Page cache is charged to the group, but it can be given back, and the kernel gave it back. What cannot be given back is anonymous memory — the kind that killed the process in the first block.

Hence a distinction that usually goes unmade: a large failcnt in a program that reads a lot from disk more often means lost speed than an approaching death. What to watch is anonymous memory.

From this follows the practical way of setting limits that gets asked as "how do you choose a limit". Two different quantities have to be read:

• peak usage says how much the group needs in the worst case seen so far;
• failcnt says how often it has hit its current ceiling.

A peak close to the limit with a growing failcnt means the headroom is gone and the next spike ends in a kill. A failcnt of zero with a peak three times below the limit means the limit is too generous — and on a densely packed machine somebody else is paying for that headroom.

How to answer in an interview

Short answer: a memory limit is set on a group of processes, and when the group hits it the kernel first tries to free memory inside the group; when that fails, it kills one of the group's processes with SIGKILL. That is why the application log is empty and why the outside sees 137, which is 128 + 9. Usage does not run away past the limit either: in the measured run the group's peak climbed exactly to it and stopped there.

That is enough for a correct answer. What follows is what you add when the interviewer digs.

If the interviewer digs deeper

Three details separate a good answer. First, you say that hitting the limit is not the same as dying: failcnt grows in a perfectly healthy group and means life at the boundary. Second, you name what the victim is chosen by: the share of the limit a process holds, not the order of requests. Third, you know the evidence that separates an OOM kill from an orchestrator's stop — the kernel line with constraint=CONSTRAINT_MEMCG and a group name; the exit code alone is not enough.

And one thing that is easy to overdo. Saying "a process never exceeds the limit by a byte" passes a measurement off as a guarantee. Our peak did climb to the limit and stop there, but cgroup v2 documents memory.max as a limit at which usage may temporarily go higher under some circumstances. The precise phrasing is this: the kill is not for exceeding the limit but for having nothing left to free inside the group — and that is checked with the group's peak and failcnt files rather than with arithmetic.

Next they ask

Common misconceptions

Practice

Two exercises. Answer first, then check against the real output: in both, the correct answer comes from a script's committed output rather than being written by hand.

group = make_group("de_practice_kill", 64)
done = subprocess.run(
  [sys.executable, "-c", EATER, group, "1600"], capture_output=True, text=True
)
print(128 - done.returncode)
print(done.stderr.strip() or "none")
print(victim_of(make_group("de_practice_victim", 160)))

137
none
big

137
MemoryError
big

1
MemoryError
small

137
none
small

Knowledge check