Without the GIL: the races it was hiding

One whiteboard, one marker

Picture doorkeepers at the four doors of one building. On the wall hangs a shared whiteboard, and on it a number: how many people have come in. When someone enters, a doorkeeper walks up to the board, reads the number, adds one in their head and writes the new number over the old one.

The doorkeeper here is a thread, and the board is the program's shared data.

In ordinary Python the doorkeepers have one marker between them. Only the one holding it can write. The marker does not change hands at just any moment, only during pauses — between two people coming in, or when a doorkeeper gets distracted, say, goes off to call someone. While a doorkeeper is reading the number, adding one and writing, there are no pauses, and nobody takes the marker away from them.

So for an action this short the count on the board comes out right. Not because that was promised, but because here, between "read" and "wrote", the doorkeeper never once gets distracted.

In Python the "one marker for everyone" rule is called the GIL. In the diagrams below, the places where the marker can change hands are marked as "checkpoints".

In the free-threaded build of Python everyone has a marker. Four doorkeepers walk up to the board at the same time. Two of them read the same number, both add one, both write — and one of the people who came in is lost.

This was measured

Four threads, each adding one a million times. The result should be 4,000,000.

import threading
 
class Counter:
    pass
 
c = Counter()
c.n = 0
 
def work():
    for _ in range(1_000_000):
        c.n += 1
 
threads = [threading.Thread(target=work) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()
print(c.n)   # measured: one marker for everyone: 4000000; a marker each: less

In the measurement for the article (the same loop, with the threads starting at the same moment) ordinary Python 3.14.10 gave exactly 4,000,000 in five runs out of five. The free-threaded 3.14.10t build lost 26 to 40% of the count, and a different amount each time. How much the next run will lose cannot be said in advance — and a single run with no losses proves nothing.

The bug was there before

The most important thing in this story: the ordinary build was not correct. It simply kept the bug from happening.

That is easy to check. Let the doorkeeper get distracted for a second between "read" and "wrote" — call someone over. In code that means calling a function in the middle of the addition: c.n = c.n + one(). Now the marker has time to pass to another doorkeeper, who writes their own numbers, and then the first one comes back and writes their long-outdated number over the top.

In this variant the ordinary build lost 50 to 69% of the count — about as much as the free-threaded one. So there was no protection. It was luck: in the shortest variant the doorkeeper never got distracted.

Two signs for one name

Things made only when first needed run into the same trouble. Say a sign with a guest's name has to be hung at the entrance if there is not one there yet. A doorkeeper looks: no sign, so they go and make one. While they are making it, the marker sometimes passes to another doorkeeper — who looks in, sees no sign, and also goes to make one. Two signs end up on the wall, and each doorkeeper is sure that theirs is the right one. With one marker this happens rarely; with a marker each, all the time.

In a program the place where the signs hang is called a cache: finished things are put there so they do not have to be made twice. Two different signs for one name is a bug nobody can see. If the cache holds locks (things that let only one thread at a time get at the shared data), two threads hold two different locks "for the same person", and the lock protects nothing.

The longer a sign takes to make, the more often this happens without a shared marker. With five thousand names and slow sign-making, the ordinary build got it wrong in 20–24 cases, the free-threaded one in several thousand.

One old library, and it is one marker again

The free-threaded build has a safety net against old libraries that are not ready for everyone having a marker. If you plug in such a library, Python brings back the "one marker for everyone" rule without any error — for the whole program at once — and prints a single line of warning about it, which is easy to miss.

The program keeps working; it just stops getting what the free-threaded build was installed for: two threads no longer count twice as fast as one, but at the same speed.

What follows from this

• Give each thread its own count and add them up at the end. That is both correct and the fastest option in the measurement.
• If something shared really is needed, agree that the doorkeepers come up to the board one at a time: cover "read — modify — write" with a lock (threading.Lock).
• Hand out work to threads through a queue (queue.Queue) rather than through one shared pile of tasks that everyone pulls from on their own: in the free-threaded build the same task goes to several threads at once.
• At program startup, check that "a marker each" really is in effect: sys._is_gil_enabled() should return False.

The detailed breakdown — with interpreter instructions, iterators that yield numbers outside the range, and two settings that change the result without any race at all — is in the "In depth" tab.

Why know this?

Since 3.14 the free-threaded build has been officially supported, and moving services onto it is no longer an experiment. The usual expectation of such a move is that the code will get faster or stay as it was. In practice, some code changes its result.

What breaks is not new code. It is code that has passed tests and production load for years — counters, lazy caches, work handed out through a shared iterator. It was wrong before, too, but the GIL made its bug rare or outright impossible, and nobody saw it. In our measurement without the GIL the same bug fired in a large share of cases, tens of percent.

Below are five measurements. Each is the same script run on three interpreters (where that makes sense): 3.14.10 with the GIL, 3.14.10t without it, and 3.15.2t, the next version, also without the GIL. The machine has two cores; the measurements use four threads (two in the extension measurement).

The mental model: the GIL never made code atomic

A common belief: as long as there is a GIL, a line of Python runs to completion with no interference from other threads. That is wrong. The GIL guarantees that only one thread executes bytecode at any given moment, not that a thread will finish the line it has started.

A thread does not give up the GIL just anywhere. It checks for a request to yield in specific instructions: in Python/bytecodes.c the check (_CHECK_PERIODIC, or in RESUME its variant _CHECK_PERIODIC_IF_NOT_YIELD_FROM) is part of RESUME (function entry), JUMP_BACKWARD (the jump back to the top of a loop) and CALL (after a call to a C function); for a Python function the check fires in RESUME on entry into it. Between two such points a thread runs without interruption, however many instructions lie between them — as long as none of them releases the GIL on its own (I/O, waiting on a lock) and none of them runs someone else's code. Even an innocent-looking instruction can: LOAD_ATTR on a property, BINARY_OP on a type with its own __add__, STORE_ATTR after which the old value dies and runs its __del__ — all of these execute Python code, with its own RESUME and CALL.

So under the GIL, the question "will an increment be lost" was really the question "is there a checkpoint on the execution path" — for specific code, specific types and a specific build. That is a property of the CPython implementation, not a language guarantee: Python does not promise that += is atomic, with the GIL or without it. Without the GIL the question is a different one: two threads execute bytecode at the same time on different cores, and any sequence of several instructions can interleave with another thread's.

The free-threading documentation puts this briefly. The page on the thread safety of built-in types says in its very first paragraph: When the GIL is enabled, most operations are implicitly serialized, and further down it says: Operations that involve multiple accesses, as well as iteration, are never atomic.

First: +=, a race nobody could see

Four threads, each adding one to the same attribute a million times. The invariant to preserve is simple: the total equals the number of increments made, that is, 4,000,000. The correct version is known in advance — read, add and write under a single lock:

lock = threading.Lock()
 
def work():
    for _ in range(1_000_000):
        with lock:
            c.n += 1

The lock changes nothing in += itself: it keeps a second thread out of the gap between the read and the write while the first one is in it — provided every thread takes the same lock. What it costs is below, in "The fix and what it costs". Now the version without it — and why under the GIL it lost nothing.

The conditions of the experiment: c is an instance of an empty class Box, n is a built-in int, with no properties and no custom __add__ or __del__; the interpreter is CPython 3.14.10, and the checkpoints are taken from Python/bytecodes.c at the same tag. Here is what lies between the read and the write:

1. INCREMENT WITH NO CALL BETWEEN READ AND WRITE
------------------------------------------------
  LOAD_ATTR n
  LOAD_SMALL_INT
  BINARY_OP +=
  SWAP
  STORE_ATTR n
checkpoints between the read and the write: 0

Five instructions, and under these conditions not one of them checks for a request to give up the GIL or runs Python code. So in this experiment another thread has no place to wedge itself in between LOAD_ATTR and STORE_ATTR in the GIL-enabled build. (The script marks checkpoints by instruction name in the body of this one function — an illustration of the experiment, not an analysis of all the code that executes.) And that is exactly what happens:

Share of increments lost out of 4,000,000 across five runs, bench/free-threading-races/lost_update.py.

The first two rows are what you see when you migrate: code that lost not a single increment in this measurement loses roughly a quarter to two thirds of them on 3.14.10t. The losses themselves vary from run to run: by half again for c.n += 1 on 3.14.10t, by more than double for d["n"] += 1 on 3.15.2t. That is a result too: a bug like this does not reproduce as a stable number, and a run with no losses does not prove that losses cannot happen.

The race was always there

The third row is the main point of this section. The form c.n = c.n + one() does the same thing, but with a call between the read and the write:

2. INCREMENT WITH A CALL BETWEEN READ AND WRITE
-----------------------------------------------
  LOAD_ATTR n
  LOAD_GLOBAL one + NULL
  CALL  <- checkpoint
  BINARY_OP +
  LOAD_FAST_BORROW c
  STORE_ATTR n
checkpoints between the read and the write: 1

Once there is a checkpoint, the GIL-enabled build loses 50 to 69% — comparable to the free-threaded one (51 to 57% on 3.14.10t). The mechanism is this: the thread read c.n, went into one(), gave up the GIL there at a waiting thread's request, and while it waited for its turn the other threads kept adding. Then the first thread came back and wrote its old value plus one. All the other threads' work in the meantime vanished in a single write.

So the GIL did not fix this race. It narrowed the window in which the race can happen down to the places where a thread gives up control — and c.n += 1 simply happened to contain no such place. Any change that adds a call between the read and the write (logging, a property instead of an attribute, __add__ on a type of your own) brings the race back into the GIL-enabled build too.

The fix and what it costs

Measured by bench/free-threading-races/lost_update.py, block 4: five runs of four threads each, the losses and the best time of the five.

In the 3.14.10t build the lock does eliminate the losses, but four threads queue for a single lock four million times: 1.005 s against 0.538 s for the same lock in the GIL-enabled build. A counter of its own in each thread, summed at the end, is not only correct but also the fastest of all the variants measured: 0.051 s without the GIL against 0.090 s with it. There is no shared mutable state, so there is nothing to contend for. (Part of the gain comes from the fact that in the measurement such a counter is a local variable, with no attribute access.)

Second: check, then act

Lazy caches have been written like this for years:

def get(key):
    if key not in cache:
        cache[key] = make(key)
    return cache[key]

Between the check and the write there is a call to make(), so there are checkpoints even in the GIL-enabled build — for the whole time make() runs. The window is the same in both builds. The difference is how often a second thread lands in it. With the GIL that takes a switch. In this measurement the threads neither wait nor block, so the GIL changes hands at a waiting thread's request, and that request is timed by the switch interval, 5 ms by default. The interval is a scheduling target, not a lower bound: a thread that blocks on I/O or waits on a lock gives up the GIL sooner, and the actual interval can also be longer. Without the GIL, threads inside the window run at the same time: two threads on different cores check the key, both miss, both build.

Four threads, 5,000 keys. With an empty make() there is almost no window in either build. With a make() of a thousand iterations there are 20–24 redundant make() calls with the GIL and 2,682–4,038 without it, on 3.14.10t. And nearly all of them are keys for which the threads got different objects — a split: 20–24 with the GIL and 2,646–4,035 without it.

A split is counted on the objects themselves: each thread keeps everything it got until the end of the run, and only then are the objects compared by id(). Keeping bare id values would not do: an id is unique only for the object's lifetime, and the address of an object that died after being pushed out of the cache can go to a new one — then two different objects would look like one.

A split is more dangerous than wasted work. If the cache holds per-key locks — locks[user_id] — a split means two threads are holding two different locks for the same user, which means there is no mutual exclusion. And no error shows up: each lock was acquired properly.

Two fixes, on the widest window:

bench/free-threading-races/check_then_act.py, five runs each, 5,000 keys.

setdefault is on the list of safe dictionary operations on the thread-safety page: it inserts only if the key is missing and returns whatever ended up in the dictionary. After it there is no split, but make() is still called needlessly — without the GIL that is thousands of redundant calls per 5,000 keys. If building the value is expensive or has side effects (it opens a connection, writes to a database), you need a lock: it removes both the split and the wasted work.

Third: one iterator for several threads

A simple way to hand out work to threads is to give them one iterator:

it = iter(tasks)
 
def worker():
    for task in it:
        handle(task)

With the GIL this works: next() on a built-in iterator is a single operation in C, and the threads take the elements without duplicates. The free-threading documentation warns about this case in a section of its own: threads may see duplicate or missing elements. The measurement: four threads, a million elements:

bench/free-threading-races/shared_iterator.py, five runs each.

On 3.14.10t the range iterator produces values that do not exist in the source at all: range(1_000_000) yielded numbers from a million upwards — up to 57,022 of them in a run, and not a single negative one. That is not the "duplicate or missing" the documentation warns about but a race inside the iterator itself: the next value and the number of remaining items are stored in two fields that are not updated together. In the CPython tracker this is cpython#129068; it is fixed in 3.15 and not backported to 3.14 — the measurement on 3.15.2t is clean.

The list iterator behaves the same way in 3.15 too — exactly as the documentation warns: a shared iterator without synchronization can hand out elements more than once. In the measurement each element was received by two or three of the four threads on average. Work should be handed out through a queue: the same measurement with queue.SimpleQueue gives exactly a million with no duplicates on all three builds.

What the free-threaded build guarantees, and what it does not

After three sections about breakage it is easy to decide that everything is dangerous without the GIL. It is not: the built-in containers are protected from the inside. According to the thread-safety page:

• single operations do not corrupt a container: lst.append(x), lst.pop(), d[key] = value, del d[key], d.pop(key), d.setdefault(key, v), s.add(x);
• reading a single element is atomic: lst[i], d[key], d.get(key), key in d;
• “Operations that involve multiple accesses” are never atomic — d[k] += 1, c.n += 1, check-then-act, iteration while another thread is making changes.

It matters what kind of page this is. In the documentation that matches our 3.14.10 build, the free-threading HOWTO points to it for the guarantees of the built-in types. These are documented guarantees for specific CPython operations, not a rule of the Python language, and they do not turn a check–compute–store sequence into one atomic operation — that is what the sections above are about. The guarantees come with caveats of their own: dict operations compare keys with __eq__, and if a key's __eq__ is written in Python, another thread may modify the dict during the comparison. The HOWTO's advice is to use threading.Lock and other synchronization primitives where possible rather than rely on the internal locks of the built-in types.

Fourth: one extension brings the GIL back for the whole process

The free-threaded build has a safety net for C extensions that are not ready to run without the GIL. An extension has to declare its readiness explicitly, and how depends on the way the module is initialized: with multi-phase initialization, the Py_mod_gil slot set to Py_MOD_GIL_NOT_USED; with single-phase initialization, a call to PyUnstable_Module_SetGIL. If it does not, then with normal GIL selection (no PYTHON_GIL=0 or -X gil=0) the import will raise a warning and enable the GIL at runtime.

To show this without third-party packages, the measurement uses two tiny C extensions, both using multi-phase initialization. They differ only in one entry of the slots table and in the module name. Speedup of pure-Python counting with two threads over one, before and after the import:

bench/free-threading-races/gil_reenable.py, 3.14.10t, best of five runs. In the first and third rows the GIL was not switched on, and the difference between "before" and "after" is within the spread of the measurement: ×1.97 and ×1.90, ×1.88 and ×1.93, in opposite directions. On 3.15.2t the ready extension gave ×1.82 before the import and ×1.89 after it.

What matters is that the GIL is turned on for the whole process, for the rest of its life: one dependency somewhere deep in the import tree, and the service you installed the free-threaded build for runs like an ordinary build, only roughly 5–10% slower on a single thread (that is how What's New in Python 3.14 estimates the price of the free-threaded build). The only notice of this is a single RuntimeWarning at import time, which is easy to lose in the startup logs.

PYTHON_GIL=0 keeps the GIL off. The warning itself points to this way out — with the caveat at your own risk. An extension that has not declared its readiness may have been relying on the GIL to protect its state — with PYTHON_GIL=0 that protection is gone. So the third row of the table is a diagnostic experiment, not a fix: an extension that is not ready gets fixed in the extension itself (declaring support once it is ready) or replaced.

Fifth: same code, different result

The last group of changes is not about races but about two flags whose defaults are different in the free-threaded build. What's New in Python 3.14 describes each of them in identical words: defaults to true for the free-threaded build and false for the GIL-enabled build.

bench/free-threading-races/context_flags.py.

The first two rows are thread_inherit_context: a thread starts with a copy of the context of whoever called start(). That explains both the request_id and the decimal precision: a setting made in the main thread travels into the worker. The third is context_aware_warnings: in the GIL-enabled build catch_warnings changes the filter list shared by the whole process, and one thread's filter applies to all the others.

Here the free-threaded build behaves more sensibly: the context is inherited, and filters do not leak between threads. But code that relied on the old behavior gets a different result without a single race. You can bring the old behavior back with two options: -X thread_inherit_context=0 -X context_aware_warnings=0 — with them 3.14.10t prints exactly what the GIL-enabled build prints.

What to do about it

1. Check that the GIL really is off. Log sys._is_gil_enabled() after the extensions have been loaded, or export it as a metric. With normal GIL selection a single extension that has not declared free-threading support (Py_mod_gil or PyUnstable_Module_SetGIL) turns the GIL on for the whole process, and the only notice is a warning.
2. Look for read-modify-write on shared state. += on an attribute, a dictionary item, a global variable. With the GIL such code lost nothing in the measurement as long as there was no call between the read and the write — but it was not correct even then; without the GIL it loses.
3. Lazy caches go through setdefault or a lock. setdefault if a redundant build is cheap and has no side effects; a lock with a second check if building is expensive or if the cache holds locks.
4. Hand out work through a queue, not a shared iterator. An iterator shared by several threads without synchronization hands out elements more than once — on 3.14 and on 3.15 alike.
5. Give each thread its own counter. That is not only correct but faster: there is no shared mutable state, so there is nothing to contend for.
6. Before migrating, run your tests on the free-threaded build with several threads. Some of the races in this article do not show up on the GIL-enabled build at all: c.n += 1 gives zero losses out of four million, five times in a row; others show up in fractions of a percent. And a clean run on the free-threaded build is no proof either: a race does not show up in every run.

What measured this

The numbers in this article come from these scripts. Each one opens from here, together with the record of the run: what it was measured on, what came out, and with what spread.

• bench/free-threading-races/bytecode.py
• bench/free-threading-races/check_then_act.py
• bench/free-threading-races/context_flags.py
• bench/free-threading-races/gil_reenable.py
• bench/free-threading-races/lost_update.py
• bench/free-threading-races/shared_iterator.py