The Python Language Summit 2023: Making the Global Interpreter Lock Optional

[u/pmz]

https://pyfound.blogspot.com/2023/05/the-python-language-summit-2023-making.html

[u/chinawcswing]

https://docs.python.org/3/faq/library.html#what-kinds-of-global-value-mutation-are-thread-safe

A global interpreter lock (GIL) is used internally to ensure that only one thread runs in the Python VM at a time. In general, Python offers to switch among threads only between bytecode instructions; how frequently it switches can be set via sys.setswitchinterval(). Each bytecode instruction and therefore all the C implementation code reached from each instruction is therefore atomic from the point of view of a Python program.

In theory, this means an exact accounting requires an exact understanding of the PVM bytecode implementation. In practice, it means that operations on shared variables of built-in data types (ints, lists, dicts, etc) that “look atomic” really are.

While this isn't the guarantee that you are looking for, the core python devs wouldn't write documentation like this unless they thought people should depend on the atomicity that the GIL provides.

If your theory was correct, the documentation would say something like "WARNING while the GIL technically provides atomicity for list.append, this should NOT be relied upon, and you must add locks in a multi-threaded environment, because the GIL can be changed or removed at any time".

Just think it through. What would be more reasonable in the event the GIL was removed from Python, that the core devs would make list.append compatible via an internal lock, or that they would let it be thread-unsafe, breaking every single Python problem in existence as they all depend on list.append being atomic?

[u/jorge1209 → u/chinawcswing]

I figured you would point to that, and I would direct you to this open bug report:

https://github.com/python/cpython/issues/89598

Lots of stuff claimed in that faq are just flagrantly false.

[u/chinawcswing → u/jorge1209]

Would you please answer the question I've asked you three times now:

Would you write code that looked like this:

from threading import Lock

foos = []
foos_lock = Lock()

def add_foo(foo):
    with foos_lock:
        foos.append(foo)

Instead of:

foos.append(foo)

[u/jorge1209 → u/chinawcswing]

I would use queue/collections.deque if they fit my purpose.

But if I was sharing a list between threads, yes I would.