Too many open files: whose limit it is, why it is not shared, and what counts as a descriptor

What is actually being asked

The ladder usually runs like this:

1. "What does 'Too many open files' mean?" — the warm-up.
2. "Whose limit is it — the process's, the user's, the machine's?" — where the substance starts.
3. "Soft and hard limit — what is the difference?" — whether you know that a process can change one of them itself.
4. "What counts as a descriptor?" — whether you name sockets and epoll rather than only files.
5. "Why did raising the limit in a shell not help?" — a question about inheritance.
6. "How do you tell a descriptor leak from an honest shortage?" — a question about what to watch.

The numbers come from running bench/rlimits/descriptors.py and bench/rlimits/practice.py. There is not a single timing here — only descriptor counts and error names — so everything reproduces on any Linux machine except the values of the limits themselves, which are each machine's own.

Base: a descriptor is a number, not a file

Before talking about limits it is worth naming, in ordinary words, what exactly is being counted.

A descriptor is a small whole number. When a process opens a file, connects to a database or creates a pipe between itself and a subprocess, the kernel sets up the object it needs and hands the process a number. From then on the process works with the number rather than with the object: it asks to read, to write and to close by that number.

Every process has its own numbering. The table of numbers belongs to the process: number 7 in one process and number 7 in another point at completely different things. That is why what is open is counted per process rather than per machine.

Three numbers are taken before the first line of your code. They are 0, 1 and 2 — input, output and the error stream; the process receives them at birth. You did not open them, but they are occupied exactly like everything else.

And here is the question the rest of the lesson answers: how many numbers is a process allowed to hold at once, and where does that allowance come from? There is always an allowance — no table is infinite. Every process has its own, and it reaches the process not from its own code but from whoever started it. When the numbers run out, the kernel refuses to open the next one, and that is where the familiar line about too many open files appears in the log.

That is already enough to answer the basic interview question. What follows is about why there are in fact two allowances, what other than files spends numbers, and why a limit raised in a shell so often never reaches the service.

Mechanism 1: two limits, and it is not duplication

getrlimit(2) defines the pair like this:

Three consequences, and all three get asked about.

The soft limit is the one checked. The kernel consults it at the moment a process opens another descriptor: not enough left, refusal. When someone says "the descriptor limit is a thousand", they almost always mean this one.

The hard limit bounds the raising, not the opening. How many descriptors a process opens is decided by the soft limit; how high the process may raise its own soft limit is decided by the hard one. And it may do that raising itself, without a restart. This is the most underused fix there is: a service that has hit its soft limit often needs neither a new config nor a restart, just one call at startup.

The hard limit only goes down. Lower it and it stays lowered unless the process holds CAP_SYS_RESOURCE; that is a property of the process, not of a setting.

The run shows all of it:

1. TWO LIMITS, AND ONLY ONE OF THEM IS ENFORCED
-----------------------------------------------
  soft limit (the one the kernel enforces)     20000
  hard limit (the ceiling for the soft one)    20000
  descriptors this process holds right now     3
  after lowering the soft limit to 256         256
  raised back to the hard limit, no restart    20000
  raising the hard limit                       refused: ValueError

Note the third line: the process already holds three descriptors — input, output and the error stream. They count too — and they are exactly why the next section arrives at 61 rather than 64.

Mechanism 2: EMFILE is about a process, not a machine

getrlimit(2) says plainly what happens when the limit is reached:

The key words are this process. There is a system-wide ceiling as well, but it raises a different error, and open(2) separates them explicitly: EMFILE is "the per-process limit on the number of open file descriptors has been reached", ENFILE is "the system-wide limit on the total number of open files has been reached".

In practice that is the first thing to establish from a log: which of the two letters. EMFILE means one process is at fault and the cure is at process level. ENFILE means a shared resource of the machine ran out: proc(5) describes it as /proc/sys/fs/file-max, the file that "defines a system-wide limit on the number of open files for all processes". Editing one service's limit will not help there.

Now the exhaustion itself. A child sets its soft limit to 64 and opens files until the kernel refuses:

2. WHAT HITTING THE LIMIT LOOKS LIKE
------------------------------------
  soft limit set by the child                  64
  descriptors the child already had            3
  files it managed to open                     61
  error name                                   EMFILE
  error message                                Too many open files

61, not 64. Three descriptors were already there, and they count against the limit. At 64 that looks like nitpicking; it stops looking like it when the subject is a connection pool, because the limit counts everything open, not everything you opened.

Mechanism 3: a descriptor is not a file

The word "file" in the name misleads. What counts is any object with a number in the process's descriptor table, and that can be checked by opening different things until refusal:

3. IT IS NOT ONLY FILES THAT COUNT
----------------------------------
  file: opened before the refusal              61 — EMFILE: Too many open files
  socket: opened before the refusal            61 — EMFILE: Too many open files
  pipe: opened before the refusal              60 — EMFILE: Too many open files
  epoll: opened before the refusal             61 — EMFILE: Too many open files

A socket, a pipe end, an epoll instance — each takes a number. Hence the practical list of what eats the limit in a real service: database connections, HTTP pool connections, listening sockets, accepted connections, inotify watches, timers, open log files.

And one separate line about pipes: 60 rather than 61. There are 61 free numbers, an odd count, and a pipe takes two at once: thirty pipes occupy sixty numbers, and the single number left is not enough for a thirty-first. A detail, but it explains why the count of open objects and the count of occupied numbers diverge: a process that spawns subprocesses with captured output spends numbers in pairs.

Mechanism 4: the limit is inherited, hence "I raised it and nothing changed"

The practical mistake that makes a limit "not work" is usually one: it was raised for the wrong process. The limit belongs to a process but passes to its children, and execve(2) states this the other way round — "All process attributes are preserved during an execve(), except the following" — with resource limits absent from the list of exceptions.

4. THE LIMIT IS PER PROCESS, AND IT IS INHERITED
------------------------------------------------
  soft limit set in the parent                 128
  soft limit the child starts with             128
  soft limit restored in the parent            20000

Hence the rule worth answering with: the limit is set on whoever starts the process. If a service manager starts the service, the limit belongs in the service unit; if a container does, in the container's settings. ulimit -n in your shell changes the limit only for what you start from that shell and means nothing for a process started another way.

Check it where the kernel keeps it: /proc/<pid>/limits shows the limits in force for that process. It is the only answer that does not depend on what the configs say.

Deeper: a leak or an honest shortage

Both look identical — EMFILE in the log — and their cures are opposite. The difference shows in one observation: does the number of open descriptors grow monotonically?

You count it where the rest of this lesson looks: /proc/<pid>/fd holds an entry per open descriptor, and their count is the current consumption. From there, two cases.

The count grows and does not fall when load drops: a leak. Something is not being closed — a response, a connection, a file. A higher limit only postpones it.

The count sits on a plateau near the limit and falls with the load: an honest shortage. That many connections are genuinely needed. Then the limit is raised — and raised for whoever starts the process.

There is a third case that gets mistaken for a leak: sockets in CLOSE_WAIT. The peer closed the connection and the application did not, so the descriptor stays taken until close is called. It will not free itself: this is a leak whose cause happens to be visible from outside.

How to answer in an interview

Short answer: EMFILE is a limit on the number of descriptors held by one process, and what it counts is not files but every open object — sockets, pipes, epoll — plus the three the process received at birth. There are two limits: the kernel checks the soft one when a descriptor is opened, while the hard one bounds how high the process may raise its own soft limit — and the process can do that raising itself, without a restart.

That is enough for a correct answer. What follows is what you add when the interviewer digs.

If the interviewer digs deeper

Three things separate a good answer. First, you distinguish EMFILE from ENFILE: one is about the process, the other about the machine, and they are fixed in different places. Second, you talk about inheritance and know why ulimit -n in somebody's shell changed nothing: the limit comes from whoever starts the process. Third, you name how to tell a leak from a shortage — watch the count in /proc/<pid>/fd over time, not the fact of the error.

Next they ask

Common misconceptions

Practice

Two exercises. Answer first, then check against the real output: in both, the correct answer comes from a script's committed output rather than being written by hand.

name, message, opened, held = probe(SOFT)
print(name)
print(message)
print(inherited_soft(128))

EMFILE
Too many open files
128

ENFILE
Too many open files
128

EMFILE
Too many open files
20000

ENOMEM
Cannot allocate memory
128

Knowledge check