Context managers: two methods, one returned truth, and one silent data loss

Base: what with was invented for

A program holds things it must not merely acquire but eventually give back: an open file has to be closed, a connection returned, a lock released. The whole difficulty is in the word "eventually". The line that releases the resource sits at the end of the work, and the end is not guaranteed to be reached: an error happens along the way and control leaves past that line. The resource stays held — the file unclosed, the lock never returned.

So the release is written not "after the work" but in a block that runs either way:

handle = open("data.txt")
try:
    process(handle)
finally:
    handle.close()      # runs on success and on failure alike

That is correct but wordy, and it has to be repeated everywhere the resource is taken. with is the same construction folded into one line:

with open("data.txt") as handle:
    process(handle)

Here open knows what to do on the way out and does it — on an ordinary end of the block, on a return from the middle, on an exception. For a reader, with means exactly one thing: leaving this block will be handled, however the block ends.

Hence the protocol. For an object to work with with, it has to do two things: something on entering the block and something on leaving it. In Python those two things are called __enter__ and __exit__; as receives what the first one returned, and the second one is called on the way out.

That is already enough to answer the basic interview question. Everything below is about where that guarantee stops, and about the exit method doing more than cleaning up: it also decides the fate of the exception that was on its way out. It is that second job that loses data.

Mechanism 1: two methods, and both on the type

The reference describes executing a with in seven steps. The first five are about entry: evaluate the expression, load __enter__, load __exit__, call __enter__, assign the result to the target after as.

The word "load" is not decoration. Both methods go through implicit special method lookup — that is, they are found on the type, not on the instance. You cannot attach them to an object:

class Plain:
    pass
 
p = Plain()
p.__enter__ = lambda: None
p.__exit__ = lambda *a: None
 
with p:          # TypeError: 'Plain' object does not
    pass         # support the context manager protocol

On 3.14 the message gains (missed __exit__ method): there __exit__ is loaded first, which is why its name is the one that reaches the error text.

The second thing visible from step 5: as receives the return value of __enter__, not the manager. A method that forgets to return hands back None:

class ReturnsNothing:
    def __enter__(self): pass
    def __exit__(self, *a): pass
 
with ReturnsNothing() as v:
    print(v)     # None

The object and the as variable being the same is the manager author's decision, not a rule of the language. open() returns the file itself from __enter__, so there they coincide; contextlib.suppress returns itself; @contextmanager returns whatever follows the yield.

Mechanism 2: the truthiness of one value decides the fate of the exception

PEP 343 writes down the translation of with into an explicit try. Read it in full: nothing in the rest of this lesson goes beyond what these lines already show.

mgr = (EXPR)
exit = type(mgr).__exit__
value = type(mgr).__enter__(mgr)
exc = True
try:
    try:
        VAR = value
        BLOCK
    except:
        exc = False
        if not exit(mgr, *sys.exc_info()):
            raise
finally:
    if exc:
        exit(mgr, None, None, None)

The line if not exit(...): raise is the entire suppression mechanism. The reference states the same normatively: "If the return value was true, the exception is suppressed, and execution continues with the statement following the with statement".

So __exit__ can end in three different ways. The first two are told apart by the truthiness of the returned value — that is what gets tested, not any particular object: any truthy value will do, True, a one, a non-empty string. The third way is the method raising something itself.

The rule has a boundary, and the PEP's transcription names it: exit is called after __enter__ has returned control. Before a successful entry there is nothing to decide — __exit__ is not called at all (its own section below).

Mechanism 3: the error that does not crash

As with generators, the interesting error is not the one that brings the program down. Here is code that parses rows, one of them broken:

ROWS = [
    {"id": 1, "amount": "100"},
    {"id": 2, "amount": "no data"},      # broken
    {"id": 3, "amount": "50"},
    {"id": 4, "amount": "25"},
]

The correct version puts suppress around one statement:

total = 0
for row in ROWS:
    with suppress(ValueError):
        total += int(row["amount"])

The wrong one differs by the position of two lines — suppress ends up around the loop:

total = 0
with suppress(ValueError):
    for row in ROWS:
        total += int(row["amount"])

Measured (identical on 3.13.7 and 3.14.7):

The broken record is genuinely skipped — a separate counter in the measurement gets exactly one. 75 of 175 lost. Not one exception, not one line in the log: suppress does not "skip the error", it exits the block entirely — exactly as written in PEP 343, where suppression means jumping to the statement after the with.

The same mistake written as your own manager is worse, because the call site contains no mention of suppress:

class Quiet:
    def __enter__(self): return self
    def __exit__(self, *a): return True     # "so it stops crashing"

With it, the same total: 100.

Mechanism 4: swallowing a group and cutting out a branch are different operations

In the previous section "suppress" meant one thing: __exit__ returned true and the block ended. With an exception group there are two operations, and they give different results (bench/ctxmgr-async/exceptiongroup_suppress.py).

The first is the old one. __exit__ -> True over an ExceptionGroup swallows the whole group, every branch of it (here and below the script's output is abridged to the relevant lines and its labels are translated):

1) __exit__ -> True over an ExceptionGroup
   __exit__ saw the nested: ['ValueError', 'TypeError']
   execution continued: the group was swallowed whole, both branches

The second arrived in 3.12 and works more finely: suppress cuts out its own branch and reassembles the rest. On 3.11 the same code suppressed nothing at all:

3.11.15
2) with suppress(ValueError): raise ExceptionGroup('eg', [ValueError])
   out came ExceptionGroup 'eg (1 sub-exception)' nested: ['ValueError']
3) ... ExceptionGroup('mix', [ValueError, TypeError])
   out came 'mix (2 sub-exceptions)' nested: ['ValueError', 'TypeError']
   the same object we raised: True

3.12.3 and newer
2) RESULT: suppressed, execution continued
3) out came 'mix (1 sub-exception)' nested: ['TypeError']
   the same object we raised: False

(Labels translated from the script's output; the values are as printed.)

The line saying that what came out is no longer the object that was raised is the whole mechanism. The documentation describes it verbatim:

It works downwards, too: a ValueError sitting inside a nested group is cut out of it on 3.12, and stays where it was on 3.11.

Here it is worth removing a common misconception about the cause. BaseExceptionGroup.split, which does the cutting, has existed since 3.11 — the measurement calls it directly on 3.11 and gets the same result. What changed is not split but suppress, which started calling it: before 3.12 its __exit__ fitted on one line — return exctype is not None and issubclass(exctype, self._exceptions) — and a group simply did not match that condition.

Next to suppress it helps to remember except*. It divides a group the same way — split returns the pair (match, rest) where match is subgroup(condition) and rest is the remaining non-matching part — and the reference describes it like this:

The practical consequence for code that goes anywhere near a TaskGroup, or any other structured concurrency: suppress(ValueError) there does not mean "silence everything" but "take the ValueError out of the group and let the rest through". From 3.12 onwards.

Mechanism 5: if __enter__ raised, there will be no cleanup

The reference's guarantee is conditional, and the condition is easy to read past: "if the __enter__() method returns without an error, then __exit__() will always be called".

Which is to say there is no guarantee at all before a successful entry:

class FailsOnEnter:
    def __enter__(self):
        self.acquire()          # first half succeeded
        raise RuntimeError      # second did not
    def __exit__(self, *a):
        self.release()          # WILL NOT BE CALLED

Verified by running it: __enter__ writes to the log before it raises, and after the failure that entry is all the log holds — __exit__ added nothing. The practical conclusion is that __enter__ must either acquire everything or nothing — its own try/except inside, rather than hope placed in __exit__.

The reference separately says that with A(), B(): is equivalent to nested with statements. Put that together with the rule above and you get: if B.__enter__ raised, A.__exit__ runs and B.__exit__ does not. Verified by running it, on classes:

enter A | enter B | exit A

A @contextmanager with try/finally prints a different line — there the cleanup for B does run. Why, in the next section.

Mechanism 6: @contextmanager — where the exception lives

The generator form is shorter, and it shows neither __exit__ nor its three arguments. The line after yield reads like ordinary cleanup — and any exception skips it:

@contextmanager
def naive():
    resource = acquire()
    yield resource
    release(resource)        # never reached when an exception occurs

The contextlib documentation explains why: "If an unhandled exception occurs in the block, it is reraised inside the generator at the point where the yield occurred". The exception does not "happen outside" — it is thrown into the yield, and from there behaves like any exception inside a function: it unwinds the frame, skipping the lines after yield.

Verified by running it:

The way to suppress an exception in the generator form follows from the same property: not return True, but an ordinary except that does not re-raise:

@contextmanager
def catches():
    try:
        yield
    except ValueError:
        pass            # the exception is suppressed

The documentation warns about this explicitly: "If an exception is trapped merely in order to log it… the generator must reraise that exception".

There is also one place where the generator form behaves better than a class. If the code before yield is wrapped in try/finally and the generator fails before reaching yield, that finally still runs: the exception unwinds the generator frame and passes through it. A class in the same situation gets nothing called at all — which is the difference behind the "on classes" label on enter A | enter B | exit A above.

Mechanism 7: ExitStack — nesting that is not in the code

When the number of managers is not known in advance — open every file in a list, every connection in a configuration — nested with statements cannot be written. That is what ExitStack is for, and it is obliged to behave exactly as nested with statements would:

The second sentence is not a footnote; it is the reason to read the paragraph. It can be checked directly (bench/ctxmgr-async/exitstack.py, identical output on 3.11–3.14):

4) inner returned True — the block ended without an exception:
   exit  inner (sees ValueError)
   exit  outer (sees None)
   outer saw None, because inner had already suppressed it

5) out came: KeyError 'substitute'
   exit  inner (sees ValueError)
   exit  outer (sees KeyError)
   outer saw a KeyError, not the original ValueError

(Labels translated from the script's output.)

So the outer __exit__ sees not the exception that happened but the one left over after the inner. Logging "what went wrong" at the outer level therefore lies whenever something inside substitutes.

The rest of the behaviour is what nested with statements would do, and is therefore predictable: a third __enter__ that raises closes the first two in reverse order and never opens the fourth; enter_context(object()) gives a TypeError immediately rather than later. pop_all stands apart: it moves the registered cleanup into another object, and then leaving the with closes nothing — whoever received the cleanup closes it. That is the supported way to hand out an already-open resource without closing it on the way out of the function that opened it.

Mechanism 8: async with — the same protocol under different names

The asynchronous variant adds not a single new rule. The data model describes both methods with one formulation:

Everything else follows: as gets the returned value, a true from __aexit__ suppresses the exception, the order is the same. Checked by running it (bench/ctxmgr-async/async_protocol.py), with identical output on 3.11–3.13.

Two things matter in practice that a synchronous with cannot have.

First: between entry and exit, the event loop hands control to other tasks. Not "may" — observable:

4) interleaving with a background task: ['bg-0', 'body started', 'bg-1', 'body finished', 'bg-2']

A lock taken on entry to such a manager is held across an await — and everything the async lesson says about locks begins here.

Second: the two protocols are not interchangeable. A synchronous manager does not work under async with, nor the reverse:

3a) async with over a synchronous manager -> TypeError: 'SyncOnly' object does not support the asynchronous context manager protocol
3b) with over an asynchronous manager     -> TypeError: 'AsyncOnly' object does not support the context manager protocol

On 3.14 those same two errors began pointing the way out — a direct consequence of that version looking the two methods up separately:

3a) ... (missed __aexit__ method) but it supports the context manager protocol. Did you mean to use 'with'?
3b) ... (missed __exit__ method) but it supports the asynchronous context manager protocol. Did you mean to use 'async with'?

For the mixed case there is AsyncExitStack: it holds both synchronous and asynchronous managers in one stack and closes them in reverse order of registration, without separating them by kind. It has no close, only aclose, and that is written in the documentation rather than inferred from behaviour.

Deeper: one object, one entry

The documentation calls the result of @contextmanager single-use and shows an example ending like this:

RuntimeError: generator didn't yield

All four versions produce something else. The documentation's own example, run verbatim on 3.11.15, 3.12.3, 3.13.7 and 3.14.7, gives:

AttributeError: '_GeneratorContextManager' object has no attribute 'args'

The reason is visible in the source. Lib/contextlib.py at tag v3.13.7, lines 118–125:

def __enter__(self):
    # do not keep args and kwds alive unnecessarily
    # they are only needed for recreation, which is not possible anymore
    del self.args, self.kwds, self.func
    try:
        return next(self.gen)
    except StopIteration:
        raise RuntimeError("generator didn't yield") from None

The attribute deletion comes before next(self.gen), so a second entry fails before the generator is ever touched. The documented RuntimeError lives in the except StopIteration branch — a second entry never reaches it.

None of this changes the practical conclusion: do not re-enter the object, call the function again — with careful(): every time. But it is worth knowing: an AttributeError: ... has no attribute 'args' in unfamiliar code is not a mystery, it is a re-entry into a spent manager.

Deeper: what changed in 3.14

The times of the two versions cannot be put side by side: the builds of 3.13.7 and 3.14.7 differ by more than the language version — 3.14 has --with-tail-call-interp enabled, the very flag What's New in 3.14 credits with "a geometric mean of 3-5% faster".

So in the table below, do not read the columns against each other. Read them against the first row — the one that involves no with at all: whatever it moves by is how far the build moved.

The control did not move at all — 12.5 against 12.7 ns, a difference inside the spread, so there is no general build speed-up in this pair. The generator form also goes through with, so it cannot serve as a second control: its 3.1% mixes the build's contribution with that of the bytecode change. The class form dropped 31.7%. There is nothing to credit those thirty-two per cent to but the change itself: the control, measured by the same process in the same rounds, stayed put. The cause is visible in the bytecode: on 3.13, entering a with is a single BEFORE_WITH instruction; on 3.14 that instruction is gone entirely, replaced by ordinary loads and a call:

3.13:  CALL 0 / BEFORE_WITH / STORE_NAME x

3.14:  CALL 0 / COPY 1 / LOAD_SPECIAL __exit__ / SWAP 2 / SWAP 3
       / LOAD_SPECIAL __enter__ / CALL 0 / STORE_NAME x

More instructions, less time. The motive for the change is recorded in CPython gh-120507, and it is not about speed: "they are bulky and won't optimize well in tier 2. Instead, we should lower them to attribute lookups and calls which can then be optimized". The speed-up is a consequence rather than a goal, and no number is promised either in the issue or in the documentation.

The reordering has a visible side effect. On 3.14 __exit__ is loaded first, and the error messages say so:

That last row is not a typo: when neither method exists, 3.14 names __exit__, because it never gets as far as __enter__.

Deeper: version history

What measured this

The observations in this article open from here, together with the record of the run:

• bench/context-managers/bytecode.py
• bench/context-managers/cost.py
• bench/context-managers/generator_cm.py
• bench/context-managers/protocol.py
• bench/context-managers/silent.py
• bench/context-managers/suppress.py
• bench/context-managers/traps.py
• bench/ctxmgr-async/exceptiongroup_suppress.py
• bench/ctxmgr-async/exitstack.py
• bench/ctxmgr-async/async_protocol.py
• bench/ctxmgr-async/what_changed_312.py

The last four measure no time at all: they print behaviour on four interpreters — 3.11.15, 3.12.3, 3.13.7 and 3.14.7 — and the differences between them.

How to answer in an interview

The short answer: a with is a guarantee that leaving the block will be handled, however the block ends. The protocol is two methods on the type: __enter__ is called on the way in, __exit__ on the way out, and as binds what __enter__ returned rather than the object itself.

That is enough to answer correctly. Beyond it is what you add if the interviewer digs.

If the interviewer digs deeper

First, name the boundary of the guarantee: __exit__ is called only if __enter__ returned control without an error. If __enter__ raised, there is no cleanup at all, and whatever it managed to acquire has to be released inside it.

Second, say that after a successful entry __exit__ decides more than "clean up": the fate of the exception in flight is settled by the truthiness of what it returned. So return True in __exit__ does not mean "we cleaned up successfully" but "there is no exception any more" — and it silences every exception, not only the expected one. Hence the silent data loss: suppress around a loop instead of around a line yields 100 instead of 175, with not a single exception in the log.

Next they ask

Common misconceptions

Practice

Two exercises. Answer first, then check against the real output: in both, the right answer comes from a recorded run rather than from an editor.

from contextlib import suppress

ROWS = [
  {"id": 1, "amount": "100"},
  {"id": 2, "amount": "no data"},
  {"id": 3, "amount": "50"},
  {"id": 4, "amount": "25"},
]

total = 0
added = []
with suppress(ValueError):
  for row in ROWS:
      total += int(row["amount"])
      added.append(row["id"])

print(total)
print(added)

100
[1]

175
[1, 3, 4]

100
[1, 3, 4]

175
[1, 2, 3, 4]

Knowledge check