From source to bytecode: the six steps Python takes before the first instruction

Nobody executes your text

There is a common picture: Python reads the file line by line and does what is written. That is not Python — that is a very simple calculator.

In reality there is a translator between your file and the machine that runs it. It reads the text once, whole, takes it apart, and produces a list of short commands, each doing exactly one simple thing: "push the value of this variable", "get this attribute from this object", "add the top two values".

From then on it is the commands that run. Nobody reads the text again.

Six steps the translator takes

The translator does not work in one go — it works in steps, and each step turns the material into something else.

• Text. Just a string of characters.
• Words. The text is cut into pieces: def, f, (, a, ). For each one, where it stood — line and column — is remembered. That will matter right at the end.
• Tree. The list of words becomes a structure: here is a function, it has a name, a list of arguments and a body, and the body returns a sum.
• List of names. A separate pass decides, for every name, whether it belongs here or came from outside. Here, not at run time.
• Tidying up. Everything that can be worked out in advance is worked out in advance; everything unreachable is thrown away.
• Commands. Whatever is left is what runs.

Switch the tabs on the diagram below — it holds the real output of each step for one small function.

Tidying up is fairly generous

Some things can be computed without running the program at all. Python does compute them.

def g():
    return 2 + 3 * 4

There is no multiplication in the finished commands — there is the number 14 right away. The multiplication happened once, during translation.

The same happens to:

• code after return — it is unreachable anyway;
• an if False: branch, along with everything inside it;
• adjacent quoted strings: 'a' 'b' 'c' becomes the single string 'abc'.

Hence a practical conclusion: hoisting 2 + 3 * 4 into a variable "so it is not recomputed every time" is wasted effort. It was never being recomputed.

Six commands, sixteen cells

Here is the number that surprises people. For one line,

def f(a): return a.x + 1

you get six commands. And they take up as much room as sixteen.

Ten cells are empty. They are scratch space: while the program runs, Python watches what actually happens — what type the object is, where its attribute lives — and writes down what it saw in those cells. Next time the same work can be skipped.

The command that asks for the most scratch space is "get an attribute" — nine cells. No other instruction asks for that much: it is the largest reservation in the whole table.

Where the coordinates of words end up

Remember that at the second step, where every word stood was recorded? That is not bookkeeping for its own sake.

def boom(a, b, c, d):
    return a.x + b.y * c.z - d.w

If d turns out not to be what was expected, Python shows:

    return a.x + b.y * c.z - d.w
                             ^^^
AttributeError: 'NoneType' object has no attribute 'w'

The carets sit exactly under d.w — not under the whole line, even though there are four attribute accesses in it. The coordinates written down at the very beginning made it to the end and surfaced at the moment of the error.

It is not free: the files of ready-made commands for the standard library grew 22% because of this. Python's developers decided a precise caret was worth it.

Why a variable can be "already local"

That step with the list of names explains a famous puzzle:

x = 10
 
def f():
    print(x)   # UnboundLocalError
    x = 5

It looks as if x = 5 has not been reached yet, so print(x) should pick up the outer x. But the decision about local versus outer is made not at run time — it is made at the fourth step of the pipeline, during translation, when the whole function is visible at once.

If there is an assignment anywhere in the body, the name is local for the entire function, including the lines above the assignment. By run time there is nobody left to argue with.

Three things worth remembering

Translation happens once. When the file is loaded, not on every call.

Some of the work is done in advance. Arithmetic on constants, joining string literals, dropping unreachable code — all before the start.

The empty cells are not waste, they are a workbench. They are exactly where the interpreter puts what it noticed, so that the second time round it can work faster.

Why bother?

Almost everything that happens later in the runtime rests on the code object. Until you can see that there are more instructions than lines, and more cells than instructions, there is nothing to hang a conversation about specialisation or the attribute cache on: it is an instruction that specialises, and the cache lives in exactly those cells.

There is a practical side too. This is where it becomes clear why UnboundLocalError happens on a line where the variable "obviously exists", why a traceback in 3.11+ points at the precise sub-expression, and why code that looks identical sometimes compiles to a different number of instructions.

The honest place to start is the warning that opens the dis documentation:

Everything below is about CPython, and about specific versions. None of these numbers is part of the language.

The model to hold in your head

Python's compiler is a pipeline, not a function. The text passes through six stations, and at each one it stops being what it was.

1. Source text — a string of bytes.
2. Tokens — words with coordinates.
3. Tree — structure instead of sequence.
4. Symbol table — who is local here and who is global.
5. Graph and optimisations — whatever can be dropped is dropped.
6. Code object — the thing that runs.

Since 3.9, step 3 is done by a PEG parser rather than the old LL(1) one (PEP 617) — after that replacement the grammar stopped being a limit on the language.

The key property of the pipeline: the coordinates from step 2 survive to step 6. The line and column recorded during tokenisation end up in the code object and, years later, surface in a traceback as a caret under the right character.

Four of the six steps have a module that shows them: tokenize, ast, symtable, dis; the text is just a file. The fifth, the control-flow graph, is only observable indirectly from Python — and that is worth saying plainly rather than drawing an "approximate diagram".

The graph is the body of the function cut into stretches without branches, with arrows for the jumps between them. It is what the compiler reads to see what is unreachable and what can be folded. The graph itself is never exposed; the one crack is the ast.PyCF_OPTIMIZED_AST flag, added in 3.13, and it is discussed in the section on the two passes.

What is actually visible at each step

Tokens come from tokenize. Fourteen of them for one line, each with a pair of coordinates. Those coordinates are what PEP 657 is about.

The tree comes from ast.parse. Note the ctx=Load() on Name and Attribute: reading and writing are already different things here, long before any bytecode. Different opcodes will grow out of that difference later.

The symbol table comes from symtable — a module few people remember, though it shows the least obvious pass in the compiler. A separate walk over the tree, entirely before code generation, decides for every name whether it is local, global, or captured from an enclosing scope. For our function:

>>> st = symtable.symtable("def f(a): return a.x + 1", "<s>", "exec")
>>> f = st.get_children()[0]
>>> f.get_parameters()
('a',)
>>> [(s.get_name(), s.is_local(), s.is_parameter()) for s in f.get_symbols()]
[('a', True, True)]

This, not the runtime, is where LOAD_FAST instead of LOAD_GLOBAL comes from. And it is where UnboundLocalError comes from. That can be shown rather than told:

x = 10
 
def f():
    print(x)   # UnboundLocalError
    x = 5

>>> st = symtable.symtable(open("unbound.py").read(), "unbound.py", "exec")
>>> for s in st.get_children()[0].get_symbols():
...     print(s.get_name(), s.is_local(), s.is_global())
print False True
x     True  False

x is marked local at parse time, before a single line has run — and it is marked local for the whole function, print(x) above the assignment included. By runtime there is nobody left to argue with:

  File "unbound.py", line 4, in f
    print(x)
          ^
UnboundLocalError: cannot access local variable 'x' where it is not associated with a value

The same table also marks print as is_global. Hence the different opcodes — and the old advice to put a builtin in a local variable before a hot loop. The advice is checkable, so let us check it (3.13.7, minimum of seven runs of 2,000,000 calls):

1.36 ns, about 10%. The difference is real and reproducible — but this is one of those cases where the right conclusion from a number is the opposite of the expected one: a trick passed around as a notable optimisation buys one and a half nanoseconds per call. Not worth spoiling readability for.

The code object: six instructions, sixteen cells

Now the central number of this article.

>>> def f(a): return a.x + 1
>>> len(list(dis.get_instructions(f)))
6
>>> len(f.__code__.co_code) // 2
16

Six and sixteen. The difference is neither a bug nor padding: ten cells are the inline cache, room an instruction reserves for its own notes about what it has already seen.

LOAD_ATTR reserves nine cells, BINARY_OP one. The whole table is visible without reading any C:

>>> import opcode
>>> {k: v for k, v in sorted(opcode._inline_cache_entries.items()) if v}
{'BINARY_OP': 1, 'BINARY_SUBSCR': 1, 'CALL': 3, 'COMPARE_OP': 1, ...
 'LOAD_ATTR': 9, 'LOAD_GLOBAL': 4, 'STORE_ATTR': 4, 'TO_BOOL': 3, ...}

The leading underscore is a warning: private, changes between versions. Which is exactly what happens — on 3.14 BINARY_OP is already 5.

Why this matters beyond arithmetic. Specialisation — the subject of the next article — works by having an instruction rewrite itself into a specialised variant and stash its observations in these very cells. Until you know the cells exist, specialisation looks like magic.

The coordinates that made it to the end

PEP 657 added four numbers per instruction to the code object: start and end line, start and end column. Visible through co_positions(), and it works like this:

class O: pass
 
def boom(a, b, c, d):
    return a.x + b.y * c.z - d.w
 
o = O(); o.x = 1; o.y = 2; o.z = 3
boom(o, o, o, None)

On 3.12.3:

  File "pep657.py", line 4, in boom
    return a.x + b.y * c.z - d.w
                             ^^^
AttributeError: 'NoneType' object has no attribute 'w'

The caret sits under d.w, not under the whole line — even though the line has four attribute accesses and three operations. Those are the coordinates recorded during tokenisation.

On 3.13.7 the same program also gets carets on the call line:

    boom(o, o, o, None)
    ~~~~^^^^^^^^^^^^^^^

The price is named in the PEP itself and not hidden: the standard library's pyc files grew 22%, from 28.4 to 34.7 MB. Traceback precision is paid for in disk and memory, and it is a deliberate trade.

What the optimiser does

The control-flow graph itself is not visible from Python: it lives inside compile.c, between _PyAST_Compile (line 442 at tag v3.13.7) and optimize_and_assemble (line 7689), and is never exposed. The one crack appeared in 3.13 — see the section on the two passes below.

Its consequences, though, are fully measurable. Everything below is a real co_consts and a real opcode sequence on 3.13.7:

The NOP left where the if used to be is not junk. PEP 626 requires every executable instruction to carry a line number and requires lines that did execute not to vanish from the debugging information; the NOP keeps that bookkeeping straight.

Note what the table does not contain: not one optimisation that reorders computation or drops an attribute access. Python's compiler deliberately knows almost nothing about values — everything interesting happens later, in the interpreter.

What changed in 3.12, 3.13 and 3.14

The numbers come from one script run on one function.

The trap is in the second-to-last row. HAVE_ARGUMENT fell from 90 to 44, which reads easily as "fewer instructions take an argument". The last row says the opposite: there are more of them, 101 → 107. HAVE_ARGUMENT is simply a boundary in the opcode numbering, and its move means the numbering was rearranged, not that anything got simpler. A number that means nothing on its own is a good reason to distrust one-line version comparisons.

About co_consts on 3.14. It became (1,) instead of (None, 1), and the disassembler shows LOAD_SMALL_INT where LOAD_CONST used to be. For a long time this paragraph admitted the reason had not been read. It is taken apart now — in the section on the two passes below — and it turns out to have not one cause but two at once.

About the documentation. CPython's compiler does have internal docs — InternalDocs/compiler.md, code_objects.md, parser.md. Checked by fetching both tags: on v3.14.0 all three return 200, on v3.13.7 they return 404. In 3.13 the InternalDocs/ directory holds exactly one file, string_interning.md. So an article about the 3.13 compiler has to read the code rather than a document, and any citation of InternalDocs/compiler.md for 3.13 is a citation of something that does not exist.

There are two optimisers, and in 3.14 they swapped jobs

What was said above — that the fifth step is not visible from Python at all — stopped being accurate in 3.13, and the difference is worth knowing.

There are in fact two passes, and the compiler's internal documentation lists them as separate steps:

Before step 3 there is one more pass — over the tree itself. On 3.13 it lives in Python/ast_opt.c; on 3.14 there is no file by that name at all, and its place is taken by Python/ast_preprocess.c, of which the same documentation says briefly: Preprocesses the AST before compiling. The rename is not cosmetic.

The two passes can be told apart by observation in exactly one way — the ast.PyCF_OPTIMIZED_AST flag added in 3.13. It returns the tree after the AST pass and before instruction generation (bench/source-to-bytecode/optimizer_passes.py).

3.13.7   the tree after the AST pass:
   constant arithmetic   Return(value=Constant(value=14))

3.14.7  the same tree:
   constant arithmetic   Return(value=BinOp(left=Constant(value=2), op=Add(),
                            right=BinOp(left=Constant(value=3), op=Mult(), right=Constant(value=4))))

On 3.13, 2 + 3 * 4 is folded before instructions are generated. On 3.14 the AST pass folded nothing, yet the bytecode still holds LOAD_SMALL_INT 14: the folding happened later, on the graph. The code confirms it — in v3.14.0 the fold_binop in ast_preprocess.c has a single branch left (% with a string on the left), while flowgraph.c gained fold_const_binop and fold_const_unaryop, of which v3.13.7 has not one occurrence there. The limits moved across verbatim, with the same names and values: MAX_INT_SIZE 128, MAX_COLLECTION_SIZE 256, MAX_STR_SIZE 4096, MAX_TOTAL_ITEMS 1024.

What the AST pass still does on 3.14. It did not disappear; it narrowed to transformations of the tree's shape: '%s' % (x,) still becomes a JoinedStr, a list in in still becomes a tuple. The removal of an unreachable block was the graph's work on all three versions. With not not x → TO_BOOL be more careful: on 3.12 it was not folded at all — the bytecode holds two UNARY_NOT — and TO_BOOL itself does not exist there, it arrived in 3.13.

And what this move finally explains. The version section above notes that in 3.14 co_consts became (1,) instead of (None, 1), and the cause was left unexplained. Now it is visible: on 3.14 the constant table retains values no instruction looks at.

3.14.7, orphaned constants:
   constant arithmetic   co_consts=(2,)               orphan: (2,)
   unary minus           co_consts=(5, -5)            orphan: (5,)
   string multiplication co_consts=('a', 'aaaa…')     orphan: ('a',)
   dead branch           co_consts=(False, 'always')  orphan: (False,)

3.12.3 and 3.13.7: not one

(Labels translated from the script's output; it prints nine orphan rows, four are shown.) That is exactly what to expect if the constant table is filled during instruction generation while the folding happens after it, on the graph: the folded operand is left in the table as an orphan. The other half of the explanation is LOAD_SMALL_INT, which does not need co_consts at all — hence (1,) rather than (None, 1).

The limits of folding were not moved by the change: 'a' * 4096 folds and 'a' * 4097 does not; 2 ** 64 yes, 2 ** 65 no; (1, 2) * 128 yes, (1, 2) * 129 no — identically on 3.12, 3.13 and 3.14.

What we did not establish: why the folding was moved. The fact was taken by running it and checked against the code on three tags; we did not look for the discussion where it was decided, and there is no "why" here.

What to do about it

Do not count instructions from dis output. It shows six where memory holds sixteen cells. For "how much space the bytecode itself takes" there is len(co_code), and for "how much the code object struct itself takes, without what it points at" there is sys.getsizeof.

Do not compare versions by a single number. HAVE_ARGUMENT is a live example of a value that halved without changing meaning.

Do not optimise what the compiler already folded. 2 + 3 * 4, adjacent string literals, tuples of constants, not not — each is already one constant or one operation. Hoisting them into a variable "for speed" is wasted work.

Remember that names are decided before execution. An assignment anywhere in a function body makes the name local for the entire function — a decision of the symbol table, not of the runtime. Hence UnboundLocalError on a line that comes before the assignment.

And the part that matters most. Those ten empty cells are not overhead, they are a workbench. What fills them is the specializing interpreter — the very mechanism the cells are reserved for.

What measured this

The numbers in this article come from this script. It opens from here, together with the record of the run: what it was measured on, what came out, and with what spread.

• bench/bytecode_bench.py
• bench/source-to-bytecode/optimizer_passes.py