Comprehensions: the scope boundary runs inside the square brackets, not around them

What is actually being asked here

The common knowledge about comprehensions is one line: the loop variable does not leak, unlike an ordinary for. That is true, checks out in three lines, and covers exactly half the topic.

The other half sits next to it in the reference, and usually gets read as far as the comma:

The boundary runs not around the square brackets but inside them: one part of the comprehension stays outside, all the rest goes in.

Three things follow, and they are usually learned by accident. Before taking them one at a time, though, it is worth agreeing on what a comprehension even is.

Base: a comprehension is a loop written as a single expression

A comprehension introduces no new kind of computation. Everything it does can be written as an ordinary loop, and that is how it is normally explained — as a shorter way of writing the same thing.

Here are the two forms of one calculation side by side:

raw = [2, 3, 5]
 
doubled = []                     # the long form
for x in raw:
    doubled.append(x * 10)       # [20, 30, 50]
 
doubled = [x * 10 for x in raw]  # the short form — the same thing, [20, 30, 50]

They read the same way if you read the comprehension right to left: first for x in raw — where the elements come from, then x * 10 — what goes into the result. The third part, optional, is an if condition deciding whether an element makes it into the result at all. Together they produce a list, and at this level there really is no difference between the two forms: the same elements, the same order, the same result.

There is one difference, and everyone knows it: the comprehension's variable does not leak out, while an ordinary loop's variable stays behind. That difference has a reason — a comprehension runs not where the surrounding code runs but in a scope of its own, separate from it.

And here is the question this lesson is about: if a comprehension has its own scope, then where does it look when it resolves a name? An ordinary loop looks for names exactly where it is written. A comprehension does not necessarily — and the line between "where it is written" and "somewhere else" turns out not to be the square brackets at all.

That is already enough to answer the basic interview question. Everything below is about where that boundary actually runs, what does cross it outwards, and why the one place the difference is visible to the naked eye is a class body.

Mechanism 1: the variable does not leak — and that is half the rule

Start with the half everyone knows:

y = "value from outside"
squares = [y for y in range(3)]
y                       # 'value from outside' — untouched
 
for y2 in range(3):
    pass
y2                      # 2 — an ordinary loop keeps its variable

This is the answer people give in interviews. It is correct. What follows does not follow from it.

Mechanism 2: the first iterable is evaluated outside

The reference says so in the very next sentence: "The iterable expression in the leftmost for clause is evaluated directly in the enclosing scope and then passed as an argument to the implicitly nested scope".

Inside a function you cannot see the difference: a function has a closure, and a comprehension reads its local names perfectly well. The place where the difference surfaces is a class body, and the rule for that is written separately:

Put the two quotes together and you get a prediction: in a class body a comprehension will see the first iterable and nothing else. Checked:

class Prices:
    raw = [2, 3, 5]
    multiplier = 10
    limit = 3
 
    ok_first   = [x for x in raw]                    # [2, 3, 5]
    bad_expr   = [x * multiplier for x in raw]       # NameError
    bad_cond   = [x for x in raw if x > limit]       # NameError
    bad_second = [x + y for x in raw for y in raw]   # NameError
 
    loop = []                                        # a loop sees everything
    for x in raw:
        loop.append(x * multiplier)                  # [20, 30, 50]

The fourth is the clearest. The name raw appears in it twice, and the first occurrence is visible while the second is not: the first stands in the leftmost for, the second is already inside the nested scope.

And the last row of the table matters as much as the first: an ordinary loop in the same class body, at the same indentation, sees both raw and multiplier. It creates no separate scope at all.

Mechanism 3: one thing does leak out

From "the variable does not leak" a reader builds "nothing leaks out of a comprehension". There is one exception, and it is not an accident of the implementation but a written decision:

So := inside a comprehension binds the name outside:

items = [4, 7, 2]
result = [(seen := v) for v in items]
result                  # [4, 7, 2]
v                       # NameError — the loop variable stayed inside
seen                    # 2 — the walrus came out

Two variables in one expression behave in opposite ways.

PEP names the motive outright, and it is not about brevity: "allows us to conveniently capture a 'witness' for an any() expression, or a counterexample for all()". Without the walrus you would need a second pass to find the witness:

data = [1, 3, 8, 5]
if any((witness := d) > 4 for d in data):
    print(witness)      # 8 — the first value the condition fired on

The motive has a flip side, visible in the same example. If the condition never fired, the name is still bound — to the last value checked:

small = [1, 2, 3]
if any((w := d) > 100 for d in small):
    ...
else:
    w                   # 3, not "nothing"

What you have to check is the result of any(), not whether the name exists.

Three prohibitions, all caught at compile time

The PEP spells out three cases, and all of them fail before the code ever runs. Verified by compiling on all four versions — a run of bench/comprehensions/class_scope_walrus.py:

The PEP puts the first this way: "an assignment expression target name cannot be the same as a for-target name appearing in any comprehension containing the assignment expression". The prohibition is narrow: it is about REBINDING the comprehension variable, and a walrus with a neighbouring name — [(k := i) for i in range(3)] — compiles without complaint.

The second is broader than you might guess: the walrus is banned in the iterable part of any for in the comprehension, not just the first.

The edge case: in a class body a walrus in a comprehension is banned outright

The third prohibition is why this section sits next to the one about a class body, and it deserves separate billing: it is the one place where the form itself is inadmissible rather than merely producing a wrong result.

A walrus in a comprehension binds a name in the containing scope — but in a class body there is nothing to bind it in: the comprehension cannot reach in there (that is the NameError above), and the language bans the form outright. The refusal arrives at compile time, before the first run, as assignment expression within a comprehension cannot be used in a class body. The same walrus in a function and at module level compiles and binds the name outside:

walrus in a class body:        SyntaxError
the same in a function body:   compiles
the same at module level:      compiles, and j = 4 outside

Mechanism 4: the error that does not crash

The NameError above is an honest error: it crashes, and you see it. It is worse when the name is found — but the wrong one.

A comprehension in a class body cannot see class attributes, but it sees module names perfectly. If the same name exists in both places, what diverges is not types but numbers:

SCALE = 1                       # module-level constant
 
class Prices:
    SCALE = 100                 # "overridden" at class level
    raw = [2, 3, 5]
 
    scaled = [x * SCALE for x in raw]   # takes the MODULE name
 
    loop = []                           # takes the CLASS one
    for x in raw:
        loop.append(x * SCALE)

Measured:

Two lines in one class body, one indentation level, a hundredfold difference. Not one exception.

The second way to get a wrong answer in silence is a leaked walrus overwriting a name used below:

def report(events):
    status = "ok"
    failures = [e for e in events if (status := e["status"]) != "ok"]
    return {"failures": len(failures), "overall": status}

The failure count comes out right; overall does not. After the comprehension status holds the status of the last event, not "ok".

Look at the second row: the same events in a different order, and the answer is suddenly correct. The same code on the same data answers differently depending on order, and no single run catches that reliably.

The right form has no walrus in it: [e for e in events if e["status"] != "ok"].

Deeper: what PEP 709 changed, and what it did not

In 3.12 comprehensions stopped being a separate function. The PEP says it in one sentence: "This PEP proposes to inline list, dictionary, and set comprehensions into the code where they are defined".

Visible without any timing:

PEP 709 does not touch generator expressions — their separate function and separate frame survive in all four versions.

And the scope did not change. That is worth checking rather than assuming: inlining looks exactly like the kind of change that could have shifted it. The check is the same measurement on four versions: three identical NameErrors everywhere, down to the text of the message. The PEP lists three visible changes — the behaviour of locals(), the absence of a dedicated frame in a stack trace, and sys.settrace no longer seeing a call and a return — and scope is not among them.

The claimed effect is stated as two numbers of different strength: "1.96x faster" on a microbenchmark of comprehensions alone, and "11% faster" on pyperformance. The second is the one to look at: the first measures comprehensions in isolation from the code around them.

Deeper: an asynchronous comprehension — the same rules plus a switching point

Inside an async def a comprehension can become asynchronous, and the definition of that is broader than people assume:

Two consequences of that definition are usually missed (bench/iteration/async_comprehension.py).

First: async for is not required. One await in the expression is enough:

4) [await slow(v) for v in (1, 2)] -> [10, 20]
   order: ['await-1', 'await-2']
   no async for, but there is an await — the comprehension is asynchronous anyway

Second: the outer comprehension is infected too. The reference says so outright, with a versionchanged note for 3.11: "Outer comprehensions implicitly become asynchronous".

5) [[y async for y in ...] for _ in range(2)] -> [[0, 1], [0, 1]]
   the outer comprehension looks ordinary, but inside it is asynchronous

The main practical difference is that "may suspend". A comprehension stops being an indivisible step (the script prints one event per line; below they are joined onto one line and cut after the last interesting one, and the labels are translated):

1) order of events:
   comprehension started / src-yielded-0 / bg-0 / src-yielded-1 / bg-1
   / src-yielded-2 / bg-2 / comprehension finished
   the background entries landed BETWEEN the comprehension's steps

(Labels translated from the script's output.) Another task gets to run between list elements. Everything the async lesson says about state that must not change across an await starts here: [await fetch(u) for u in urls] is not "build a list" but a long sequence of switching points, and the data underneath can change during it.

The scoping rule is unchanged by asynchrony:

3) after the comprehension the outer x = 'outer' | values: [0, 1]
   the target name lives in the implicit nested scope, as in the ordinary case

PEP 709, discussed above, inlined the asynchronous comprehension along with the rest: on 3.11 the body compiles into a separate <listcomp> code object, from 3.12 into the function itself. The asynchronous instructions changed with it: GET_AWAITABLE, SEND on 3.11 against END_ASYNC_FOR, END_SEND, SEND from 3.12 on. None of this affected the scoping rule — exactly as in the synchronous case.

And two things to remember. A generator expression with async for is an async_generator, and it can only be collected with async for, never list(). Outside an async def an asynchronous comprehension does not compile at all: SyntaxError: asynchronous comprehension outside of an asynchronous function.

Deeper: what the choice between comprehension, loop and map costs

The argument gets conducted as though the choice cost something. Measured (3.13.7, a list of 1000, best of seven runs):

The spread across all three is 1.17×, and it is resolvable: the run repeats the whole measurement five times and compares the distance between the ways against their own wobble (labels translated from the script's output):

  SPREAD OVER 5 REPEATS OF THE WHOLE MEASUREMENT, as ratios to the comprehension:
    [f(x) for x in src]              x1.00 .. x1.03   spread 0.03
    loop with out.append(...)        x1.13 .. x1.14   spread 0.01
    list(map(f, src))                x1.17 .. x1.18   spread 0.01
    distance between the extremes         0.17
    widest spread of any single way       0.03
    can the instrument resolve the choice yes

The order of the rows holds across all five repeats, and what follows from it is not what the argument is usually started for: the comprehension both reads better and measures faster than the other two. There is no speed reason to rewrite one as map — on these numbers map is the slowest of the three. And rewriting a loop as a comprehension for fifteen percent makes sense exactly when fifteen percent is visible to somebody — the next table shows where a multiplier four times larger lives.

What matters is whether there is a Python-level call per element:

Removing the function call is 1.7×. Replacing a Python condition with filter(None, …), which is entirely in C, is another 1.9×. Both multipliers are four times larger than the whole spread between the three ways of writing the loop.

And here is the boundary: 1.7 and 1.9 are numbers from one machine, one build and a list of a thousand elements; on another machine, another list length and another function they will differ. What travels is not the numbers but the orders of magnitude: the choice of syntax costs percent, a removed Python-level call costs multiples. If a decision hinges on the exact figure, measure it on your own data.

The trick that stopped helping

One line stands apart — the advice to hoist a method into a local variable:

out = []
append = out.append          # "so the attribute is not looked up every time"
for x in src:
    append(f(x))

The measurement comes out with the opposite sign: such a loop is about ten percent slower than the plain one, consistently on 3.13.7 and on 3.14.7.

The instruction counts in the loop body are equal — nine and nine:

out.append(f(x)) :  STORE_FAST LOAD_FAST LOAD_ATTR   LOAD_FAST PUSH_NULL LOAD_FAST CALL CALL POP_TOP
append(f(x))     :  STORE_FAST LOAD_FAST PUSH_NULL   LOAD_FAST PUSH_NULL LOAD_FAST CALL CALL POP_TOP

The difference is not in the count but in what stands where LOAD_ATTR is. The dis documentation describes that form: "if STACK[-1] has a method with the correct name, the bytecode pushes the unbound method and STACK[-1]. STACK[-1] will be used as the first argument (self) by CALL". So self goes onto the stack right there, and no bound-method object is created at all. A call through a local variable is an ordinary call of an ordinary object, which has to supply self on every call.

The advice comes from a time when attribute lookup was unpredictably expensive. What has become cheaper since is the lookup, not the alternative to it.

What to do about it

Inside a class body, a comprehension may only reach for its leftmost iterable. That one is evaluated outside and does see class-local names; everything else in the comprehension runs in the nested scope and does not. It is the one place where the scope boundary is visible, and it becomes visible either as a NameError or, worse, as somebody else's value. If you need more than one class-local name, an ordinary loop or a staticmethod behaves predictably there.

Use the walrus in a comprehension only for a witness. Capturing the value that made any() fire is what it was made for. Using it as a way to "also update a variable" means writing code whose answer depends on element order.

Choose between a comprehension, a loop and map by readability — here it coincides with speed. The difference is about fifteen percent, and the fastest on these builds is the comprehension, which is the variant that usually reads better too. There is nothing to argue about.

If it really is hot, remove the call rather than rewriting the loop. An expression instead of a function gives 1.7×; filter(None, …) instead of a condition gives another 1.9×. That is four times more than the whole argument about syntax.

Do not hoist a method into a local variable. On the builds measured here the trick costs six and nine percent, and it costs them in the wrong direction: with adaptive specialisation a plain obj.method(...) stopped being the slower one. That is an observation about CPython 3.13.7 and 3.14.7, not a rule of the language — on another implementation, or after the next round of interpreter work, the sign may flip back, and one run settles it.

Version history

How to answer in an interview

Short answer: a comprehension is a loop written as a single expression, but it runs in a scope of its own, and the boundary of that scope is inside the square brackets rather than around them. Everything except the iterable expression in the leftmost for runs in a separate, implicitly nested scope — which is why the comprehension's variable does not leak out, and why the comprehension does not always see the same names as the code around it.

That is enough for a correct answer. What follows is what you add when the interviewer digs.

If the interviewer digs deeper

Exactly one thing leaks out — the walrus: [(seen := v) for v in items] leaves seen bound outside, and PEP 572 prescribes that normatively.

What separates a good answer: showing the consequence that does not crash. A comprehension in a class body finds the module-level name rather than the class one and gives [2, 3, 5] where an ordinary loop one line below gives [200, 300, 500]. And not confusing this with inlining: PEP 709 in 3.12 removed MAKE_FUNCTION from comprehensions but did not change the scope — the same three NameErrors on 3.11, 3.12, 3.13 and 3.14.

And one place where it is easy to go wrong in the other direction: a walrus in a comprehension inside a class body does not "leak somewhere unexpected" — it does not compile at all, a SyntaxError before anything runs.

Next they ask

Common misconceptions

Practice

Two exercises. Answer first, then check against the real output: in both, the right answer comes from a recorded run rather than from an editor.

multiplier = 1
raw = [2, 3, 5]


class Config:
  multiplier = 100
  scaled = [x * multiplier for x in raw]

  loop = []
  for x in raw:
      loop.append(x * multiplier)


print(Config.scaled)
print(Config.loop)

[2, 3, 5]
[200, 300, 500]

[200, 300, 500]
[200, 300, 500]

[2, 3, 5]
[2, 3, 5]

[200, 300, 500]
[2, 3, 5]

Knowledge check

What measured this

The numbers in this article come from these scripts. Each one opens from here, together with the record of the run: what it was measured on, what came out, and with what spread.

• bench/comprehensions/bytecode.py
• bench/comprehensions/class_scope_walrus.py
• bench/comprehensions/cost.py
• bench/comprehensions/scope.py
• bench/comprehensions/silent.py
• bench/comprehensions/walrus.py
• bench/iteration/async_comprehension.py