Iterators and generators: one method apart, and one silent bug

Base: what the loop actually gets

Start with code everyone has written:

for x in [10, 20, 30]:
    print(x)

The list sits in memory whole, and the loop takes items out of it one at a time. The question this topic starts from sounds almost dull and turns out to be the central one: what exactly does the loop take from the list?

Not the items. A list has no way to give out "the next one": it has no notion of a current position and no memory of where the last pass stopped. The walk needs both — and neither lives in the list.

So there are two steps here rather than one:

1. the loop asks the list for something to walk along — and the list hands out a new one, separate from itself;
2. then the loop asks that thing for the next value, over and over, until the values run out.

Both sides have names. An object you can ask for a walk is an iterable: a list, a string, a dict, an open file. What it hands out, and what remembers the position, is an iterator. Which is exactly why the protocol has two methods: __iter__ — "give me something to walk along", __next__ — "give me the next one".

The main difference in behaviour follows immediately. A list is a source of walks: every new loop asks it for a new iterator and starts from the beginning. An iterator is the walk itself, and there is only one of it: having reached the end, it stays at the end. A second loop over it does not start again — it simply gives nothing.

That is already enough to answer the basic interview question. Everything below is about that "gives nothing" happening silently, with no exception; about the generator, which is an iterator written as a function; and about the trade of memory for time being measurable in both directions.

Mechanism 1: one method apart

The glossary definitions are short. An iterable is "an object capable of returning its members one at a time". An iterator is "an object representing a stream of data" whose repeated __next__ calls return successive items until the data runs out and StopIteration is raised.

What actually distinguishes them:

xs = [1, 2, 3]
it = iter(xs)
 
iter(xs) is xs          # False — a list hands out a NEW iterator
iter(it) is it          # True  — an iterator hands back itself
hasattr(xs, "__next__") # False — a list has no way to "give the next one"
hasattr(it, "__iter__") # True  — an iterator has both methods

Everything else follows. A list is a source of iterators, so you can walk it as many times as you like, from the start each time. An iterator is the stream itself, and there is only one of it:

list(it)    # [1, 2, 3]
list(it)    # []          — same object, already exhausted
list(xs)    # [1, 2, 3]
list(xs)    # [1, 2, 3]   — a fresh iterator every time

The glossary puts the second case better than anyone could:

Attempting this with an iterator will just return the same exhausted iterator object used in the previous iteration pass, making it appear like an empty container.

Appear like an empty container. Not "raise", not "warn" — appear empty. That is the root of everything below.

Mechanism 2: the bug that does not raise

Take a function that walks its argument twice. Nothing exotic — an ordinary report:

def report(rows):
    bad = [r for r in rows if r < 0]
    total = sum(rows)
    return {"total": total, "negative": len(bad)}
 
data = [10, -3, 5, -1, 8]
 
report(data)                  # {'total': 19, 'negative': 2}
report(r for r in data)       # {'total': 0,  'negative': 2}

No exception. The second pass walked an exhausted iterator, found nothing, and sum honestly returned zero. The function ran, the report was produced, the number is wrong.

Worth internalising as a class rather than a case: a function that accepts "a sequence" must either walk it once or pin it down at the start.

def report(rows):
    rows = list(rows)         # one line that removes the whole class
    ...

That line has a price — the whole sequence now sits in memory — and that price is what the rest of this lesson is about. But "works and is slow" beats "fast and wrong".

The same trap in miniature:

g = (i for i in range(5))
3 in g          # True
list(g)         # [4] — the check ate 0, 1, 2 and 3
len(g)          # TypeError: object of type 'generator' has no len()

Mechanism 3: what in the standard library is single-use

The iter(x) is x marker from the previous section is not academic. Half of what people use daily satisfies it (bench/iteration/one_shot_types.py, identical output on 3.11–3.14):

The documentation of each says so outright — map "Return an iterator", filter "construct an iterator", zip "returns an iterator of tuples". The word "iterator" in the description is the warning.

For files the same thing is recorded separately:

The check is the same for all of them: the second pass is empty. (Here and below the script's output is abridged to the relevant lines and its labels are translated.)

2) map over the list [1, 2, 3]:
   first  list(m): ['1', '2', '3']
   second list(m): []

For a file the single use is reversible — the cursor can be moved back:

5) first  pass: ['a', 'b', 'c']
   second pass: []
   fh.tell(): 9 — the cursor is at the end, hence empty
   after fh.seek(0): ['a', 'b', 'c']

(The three lines the script writes are Cyrillic letters, two bytes each — hence nine and not six.)

(Labels translated from the script's output.) map, filter and zip have no such lever: the object itself has to be rebuilt.

And here is the shape in which this costs money:

8) sum and len over the same map:
   sum(values) = 60
   len(list(values)) = 0 -> nothing left to average
   total / count -> ZeroDivisionError: division by zero

The one place in this whole section where being single-use raises an exception — and it does not name the cause. A ZeroDivisionError instead of "you walked the iterator twice". Fixed by one line: values = list(values) before the first pass.

Mechanism 4: changing a collection while iterating over it

A separate kind of single use is an iterator that became wrong while you were walking it. What matters here is that Python behaves differently for different types, and the guard is not everywhere (bench/iteration/invalidation.py, identical output on 3.11–3.14).

Dicts and sets have a guard:

1) adding to a dict during iteration -> RuntimeError: dictionary changed size during iteration
5) adding to a set during iteration  -> RuntimeError: Set changed size during iteration

But the guard compares the length, not the contents. In Objects/dictobject.c the check is written as di->di_used != d->ma_used, and hence the hole:

3) replacing values during iteration: no error, the walk is complete: ['a', 'b', 'c']
4) delete + insert (the size did not change): no error
   what was walked: ['a', 'b', 'z'] | dict: {'a': 1, 'b': 2, 'z': 0}

Delete one, add another — the length is the same, no exception is raised, and the deleted key was never visited. So a RuntimeError is a signal, not a guarantee.

The guard is sticky, too: right next to it in the source stands di_used = -1 with the comment Make this state sticky.

9) the first next after the change -> RuntimeError
   size restored; the next next -> RuntimeError

A list has no guard at all, and that is worse, because it is silent:

6) removing from a list inside a for over that same list:
   visited:   ['a', 'c', 'e']
   left over: ['b', 'd', 'e']
   not one exception; 'b' and 'd' were never visited

7) next(it) -> a | the iterator's index is now 1
   after xs.remove('a') the list became ['b', 'c']
   next(it) -> c — because index 1 already points at it

(Labels translated from the script's output.) A list iterator holds an index. Remove an element to its left and everything shifts while the index does not, so one element is skipped. The classic "filter in place" that deletes half of what it should.

There is exactly one official recommendation, and it comes from the tutorial rather than the reference:

Both routes were checked and both give the right answer: for x in list(xs) and rebuilding with a comprehension. About the "internal counter" rule people often quote as a rule of the language, it is worth being blunt: it is not in the reference — the behaviour in points 6 and 7 is a property of the list iterator's implementation, not a written rule.

Mechanism 5: a generator — an iterator written as a function

The language reference states the transformation in one sentence: "Using a yield expression in a function's body causes that function to be a generator function".

The consequence takes three lines to check:

log = []
 
def h():
    log.append("body started")
    yield 1
 
obj = h()
log            # [] — the body did NOT run
next(obj)
log            # ['body started'] — only now

The call built an object and returned. Everything written in the body — argument checks, opening a file, hitting a database — will not happen until somebody asks for the first item. A function that validates its arguments and contains a yield does not validate them when called.

Mechanism 6: four states, and all four are observable

A generator's state is not a metaphor but a string inspect returns:

import inspect
k = (i for i in range(2))
inspect.getgeneratorstate(k)            # 'GEN_CREATED'
next(k); inspect.getgeneratorstate(k)   # 'GEN_SUSPENDED'
list(k);  inspect.getgeneratorstate(k)  # 'GEN_CLOSED'

There are four states, and the order between them explains everything the rest of the lesson calls a technique. A run of bench/iterators/states.py walks them:

Three things read off the picture, each of which is usually explained separately.

A suspended generator is a live frame. GEN_SUSPENDED means the body stopped at a yield and is holding its locals, its open files and its references. Hence both the memory in the next section and the fact that send can pass a value back into that very place.

GEN_RUNNING cannot be caught from outside. While the generator runs it holds control; only it can ask for its own state, from inside the body. That is not a limitation of the library but a consequence of the caller standing still at that moment.

Two different roads lead into GEN_CLOSED, and nothing tells them apart afterwards. The body ran out on its own — or it was closed from outside with close(). One state covers both, so "exhausted" and "closed" are indistinguishable after the fact: if the difference matters, you have to record it yourself.

Three different things with similar names

This is also the place to separate what conversation calls by one word:

The first is about the declaration, the second and third about the object. Confusing the first with the second is what produces "why did my generator print nothing": there is nobody to print until somebody asks for an item.

Mechanism 7: memory against time — the measurement

Here is the trade generators exist for. Peak memory via tracemalloc, one million elements:

The list peak is 38.57 MiB on all four versions; the generator never goes past a kilobyte on any of them (0.5 KiB on 3.11–3.13, 0.4 on 3.14). The order of the difference is tens of thousands of times, and it does not depend on the version. The reason is obvious — the list holds a million objects, the generator holds one suspended function.

On that same million the generator does not lose on time either; it wins: 45.6 ms against 59.1 for the list on 3.13.7. Building the whole list first costs more than not building it.

And now the part usually left out. On small data it is the other way round:

Creating the generator is cheaper — there was no body to run. Walking it is dearer: every next resumes a suspended function, and ten resumptions cost more than one loop over a finished list.

So the rule is written from its limit rather than from a number: when the alternative materialises every result, a generator usually cuts the extra memory dramatically; it saves time only when there is a lot of data, or when not all of it is needed. Both "generators are faster" and "generators always save memory" claim more than was measured.

That "always" is a word too far is shown by the same tool. A run of bench/iterators/memory_when_not.py takes three cases in a row:

In the second row the generator saves nothing and costs 352 bytes more: the list is already occupied before the measurement, and laziness does not undo it. In the third there is no win at all — list(i * i for i in range(N)) holds the same million values a comprehension does. And the fourth block of that run shows the other side: a suspended generator is a live frame with every reference it holds, and a hundred such generators with a buffer inside took 76.32 MiB.

The win comes not from laziness itself but from the alternative occupying memory. No alternative, no win.

Deeper: why the small-data loss is structural

The reason is visible without a stopwatch. In 3.12 comprehensions were inlined into the calling code (PEP 709): "Dictionary, list, and set comprehensions are now inlined, rather than creating a new single-use function object for each execution". Generator expressions were not part of that change — and that can be checked directly:

The list comprehension stopped being a separate function; the generator expression stayed one, in all four versions. Hence the extra frame, and hence the extra nanoseconds on short sequences.

Timings are never compared across versions in this lesson, and the reason is in the builds. 3.11 and 3.12 are built with GCC 13.3.0; 3.13 and 3.14 use Clang 20.1.4, and 3.14 additionally has --with-tail-call-interp, which 3.13 does not and cannot have. The difference between builds swamps the difference between versions. Only generator-against-list within a single run is comparable — which is what the tables above do.

Deeper: StopIteration inside a generator

The classic mistake the language eventually outlawed:

def broken(it):
    while True:
        yield next(it)      # when it runs out, next raises StopIteration
 
list(broken(iter([1, 2])))

Before Python 3.7 this silently produced [1, 2]: a StopIteration escaping the body is indistinguishable from the generator finishing normally. A bug deep in a pipeline turned into "the data just ended early".

From 3.7 on it is an exception:

RuntimeError: generator raised StopIteration

and its __cause__ is the original StopIteration, so the real reason is not lost. Checked on 3.11, 3.12, 3.13 and 3.14 — identical in all four.

There are two boundaries here, and both are recorded in the standard library — you can read them without leaving the interpreter:

import __future__
__future__.generator_stop
# _Feature((3, 5, 0, 'beta', 1), (3, 7, 0, 'alpha', 0), 8388608)

Optional from 3.5.0b1, mandatory from 3.7.0a0.

The correct form catches and returns:

def fixed(it):
    while True:
        try:
            yield next(it)
        except StopIteration:
            return

Deeper: a generator as a consumer — send, throw, close

PEP 342 made yield an expression, and the generator stopped being only a source. send(value) resumes it with the value becoming the result of the yield; throw raises an exception at the point of suspension; close raises GeneratorExit there.

Of the three, close stands apart: it is the only one that is about cleanup rather than about pushing data in:

def g():
    try:
        yield 1
    except GeneratorExit:
        return "cleaned up"
 
gen = g(); next(gen)
gen.close()

The change is stated in the reference outright: "Changed in version 3.13: If a generator returns a value upon being closed, the value is returned by close()". Measured on all four versions.

Deeper: yield from is not sugar over a loop

PEP 380 added delegation. The difference from a hand-written loop is not brevity: send, throw and the subgenerator's return value all pass through yield from, and none of them passes through for x in inner(): yield x.

yield from is not always the faster of the two. Walking a thousand elements through one layer — read the table by rows: within a row the loop and the delegation were measured in one run of one interpreter, while across rows sit different builds, whose contribution cannot be subtracted.

On 3.11 delegation is not faster but slightly slower than the hand-written loop — the sign there is the opposite one. So "yield from is always faster" is simply untrue; choose it for the protocol it forwards, not for the nanoseconds.

Deeper: the same protocol through await

Asynchronous iteration is the same protocol under different names, with one addition: every step may yield control.

The correspondence is one to one: __aiter__ for __iter__, __anext__ for __next__, StopAsyncIteration for StopIteration. The "its own iterator" marker works the same way (bench/iteration/async_iteration.py, identical output on 3.11–3.14):

1) a.__aiter__() is a          : True
   AsyncRange has __iter__     : False
   AsyncRange has __next__     : False
3) await it.__anext__() -> 1
   await it.__anext__() -> StopAsyncIteration (not StopIteration)

One syntactic subtlety: __aiter__ is an ordinary def, not an async def. Returning an awaitable from it was allowed until 3.7, when that support was removed; since then it must return the asynchronous iterator itself, or TypeError.

Two bridges break, and it is worth knowing in advance:

4) an ordinary for over AsyncRange -> TypeError: 'AsyncRange' object is not iterable
   list(async generator) -> TypeError: 'async_generator' object is not iterable

So list, sum, sorted, min and everything else that takes an iterable will not accept an asynchronous source. Collecting one into a list is possible only with an asynchronous comprehension: [v async for v in ...].

Being single-use is inherited whole — a second pass over an asynchronous generator is empty, exactly as with an ordinary one.

And here is what the synchronous one does not have:

7) interleaving the consumer with a background task:
   ['bg-0', 'got-1', 'bg-1', 'got-2', 'bg-2', 'got-3', 'bg-3']

8) leaving an async for by break: finally has not run yet
   after await g2.aclose(): ['finally ran']

Point 7 is the whole reason for this: every __anext__ is a switching point. Point 8 is the price: leaving an asynchronous generator by break does not run its finally right away. The cleanup has to be awaited explicitly — await g.aclose(), or the event loop's asynchronous finaliser. Exactly the class of problem that "a task nobody references" is, in the async lesson.

Deeper: version history

How to answer in an interview

The short answer: an iterable can hand you an iterator (__iter__); an iterator can hand you the next item (__next__) and must hand back itself from __iter__. One method of difference, and one difference in behaviour: a list can be walked twice, an iterator cannot. A generator is an iterator written as a function, and calling a generator function does not run its body — it builds an object and returns.

That is enough to answer correctly. Beyond it is what you add if the interviewer digs.

If the interviewer digs deeper

What separates a good answer: not stopping at "generators save memory" but naming both sides of the trade and the limit of the win. In memory the win appears when the alternative materialises every result: on a million squares the list takes 38.57 MiB against 0.5 KiB for the generator — while summing over an already existing list saves nothing at all, 48 B against 400 B for the generator expression. In time the sign depends on the size: on ten elements creating and summing through a generator costs MORE than through a list comprehension — 375 ns against 238 — and the reason is structural: comprehensions have been inlined since 3.12, generator expressions have not.

Next they ask

Common misconceptions

Practice

Two exercises. Answer first, then check against the real output: in both, the right answer comes from a recorded run rather than from an editor.

nums = iter([1, 2, 3])
print(sum(nums))
print(sum(nums))
print(list(nums))

6
0
[]

6
6
[1, 2, 3]

6
0
[1, 2, 3]

6
Traceback (most recent call last):
StopIteration

Knowledge check

What measured this

The numbers in this article come from these scripts. Each one opens from here, together with the record of the run: what it was measured on, what came out, and with what spread.

• bench/iterators/inlining.py
• bench/iterators/memory_time.py
• bench/iterators/memory_when_not.py
• bench/iterators/protocol.py
• bench/iterators/states.py
• bench/iterators/silent.py
• bench/iterators/traps.py
• bench/iterators/versions.py
• bench/iteration/one_shot_types.py
• bench/iteration/invalidation.py
• bench/iteration/async_iteration.py