Issue 40522: [subinterpreters] Get the current Python interpreter state from Thread Local Storage (autoTSSkey)

_PyThreadState_GET() gets the current Python thread state from _PyRuntime.gilstate.tstate_current atomic variable.

When I experimented per-interpreter GIL (bpo-40512), I got issues with _PyThreadState_GET() which didn't return the expected Python thread state.

I propose to modify _PyThreadState_GET() in the exprimental isolated subinterpreters mode to get and set the current Python thread state using a Thread Local Storage: autoTSSkey.

New changeset e838a9324c1719bb917ca81ede8d766b5cb551f4 by Victor Stinner in branch 'master':
bpo-40522: _PyThreadState_Swap() sets autoTSSkey (GH-19939)
https://github.com/python/cpython/commit/e838a9324c1719bb917ca81ede8d766b5cb551f4

See also bpo-15751: "Make the PyGILState API compatible with subinterpreters".

There are many ways to get the current interpreter (interp) and the current Python thread state (tstate).

Public C API, opaque function call:

* PyInterpreterState_Get() (new in Python 3.9)
* PyThreadState_Get()

Internal C API, static inline functions:

* _PyInterpreterState_GET()
* _PyThreadState_GET()

There are so many variants that I wrote notes for myself:
https://pythondev.readthedocs.io/pystate.html

This issue is about optimizing _PyInterpreterState_GET() and _PyThreadState_GET() which are supposed to be the most efficient implementations.

Currently, _PyInterpreterState_GET() is implemented as _PyThreadState_GET()->interp, and _PyThreadState_GET() is implemented as:

    _Py_atomic_load_relaxed(_PyRuntime.gilstate.tstate_current)

--

To find the _PyInterpreterState_GET() machine code, I read the PyInterpreterState_Get() assembly code (not optimized, it adds tstate==NULL test) and PyTuple_New() assembly code, since PyTuple_New() now needs to get the current interpreter:

static struct _Py_tuple_state *
get_tuple_state(void)
{
    PyInterpreterState *interp = _PyInterpreterState_GET();
    return &interp->tuple;
}

To find the _PyThreadState_GET() machine code, I read the PyThreadState_Get() assembly code.

I looked at the x86-64 machine code generated by GCC -O3 (no LTO, no PGO, it should not be relevant here), using GCC 10.2.1 on Fedora 33.

_PyThreadState_GET():

   mov    rax,QWORD PTR [rip+0x2292b1]   # 0x743118 <_PyRuntime+568>

_PyInterpreterState_GET():

   mov    rax,QWORD PTR [rip+0x22a7dd]        # 0x743118 <_PyRuntime+568>
   mov    rax,QWORD PTR [rax+0x10]

By default, Python is built with -fPIC: _PyRuntime variable does not have a fixed address.

$ objdump -t ./python|grep '\<_PyRuntime\>'
0000000000742ee0 g     O .bss	00000000000002a0              _PyRuntime

The "[rip+0x2292b1] # 0x743118 <_PyRuntime+568>" indirection is needed by PIC.

>_PyInterpreterState_GET():
>   mov    rax,QWORD PTR [rip+0x22a7dd]        # 0x743118 <_PyRuntime+568>
>   mov    rax,QWORD PTR [rax+0x10]

While working on bpo-39465, I wrote PR 20767 to optimize _PyInterpreterState_GET(): single instruction instead of two:

* Add _PyRuntimeState.interp_current member: atomic variable
* _PyThreadState_Swap() sets _PyRuntimeState.interp_current

But I failed to measure any performance difference.

PR 23976 stores the currrent interpreter and the current Python thread state into a Thread Local Storage (TLS) using GCC/clang __thread keyword.

On x86-64 using LTO and GCC -O3, _PyThreadState_GET() and _PyInterpreterState_GET() become a single MOV.

Assembly code using LTO and gcc -O3.

_PyThreadState_GET() in _PySys_GetObjectId():

   0x00000000004adabe <+14>: mov rbx,QWORD PTR fs:0xfffffffffffffff8

_PyThreadState_GET() in PyThreadState_Get():

   0x000000000046b660 <+0>: mov  rax,QWORD PTR fs:0xfffffffffffffff8

_PyInterpreterState_GET() in PyTuple_New():

   0x000000000048dfcc <+12>: mov  rax,QWORD PTR fs:0xfffffffffffffff0

_PyInterpreterState_GET() in PyState_FindModule():

   0x000000000044bf20 <+16>: mov  rax,QWORD PTR fs:0xfffffffffffffff0

---

Note: Without LTO, sometimes there is an indirection:

_PyThreadState_GET() in _PySys_GetObjectId(), 2 MOV (PIC indirection):

   mov    rax,QWORD PTR [rip+0x1eb270]        # 0x713fe0
   # rax = 0xfffffffffffffff0 (-16)
   mov    r13,QWORD PTR fs:[rax]

_PyInterpreterState_GET() in PyTuple_New(), 2 MOV (PIC indirection):

  mov    rax,QWORD PTR [rip+0x294d95]        # 0x713ff8
  mov    rax,QWORD PTR fs:[rax]

An optimized Python should always be built with LTO.

PR 23976 should make _PyInterpreterState_GET() more efficient, and it prepares the code base for one GIL per interpreter (which is purpose of this issue ;-)).

Mark Shannon experiment using __thread:

* https://mail.python.org/archives/list/python-dev@python.org/thread/RPSTDB6AEMIACJFZKCKIRFTVLAJQLAS2/

* https://github.com/python/cpython/compare/master...markshannon:threadstate_in_tls

He added " extern __thread struct _ts *_Py_tls_tstate;" to Include/cpython/pystate.h.

> An optimized Python should always be built with LTO.

In MacOS is quite challenging to activate LTO, so normally optimized builds are only done with PGO. Also in Windows I am not sure is possible to use LTO. Same for many other platforms

One GIL per interpreter requires to store the tstate per thread. I don't see any other option. We need to replace the global _PyRuntime atomic variable with a TLS variable. I'm trying to reduce the overhead, but it's heard to beat the performance of an atomic variable.

That's also we I modified many functions to pass explicitly tstate to subfunctions in internal C functions, to avoid any possible overhead of getting tstate.

https://vstinner.github.io/cpython-pass-tstate.html


Pablo:
> In MacOS is quite challenging to activate LTO, so normally optimized builds are only done with PGO.

Oh right, I forgot macOS. I should check how TLS is compiled on macOS. IMO wwo MOV instead of MOV is not a major performance bottleneck.

The best would be to be able to avoid pthread_getspecific() function which is less efficient than a TLS variable. The glibc implementation uses an array for a few variables (first 32 variables?) and then a slower hash table.


Pablo:
> Also in Windows I am not sure is possible to use LTO. Same for many other platforms.

I will check how it's implemented on Windows.

We cannot use TLS on all platforms, since it requires C11 features which are not available on all platforms. Also, the implementation depends on the architecture.

New changeset 62078101ea1be5d2fc472a3f0d9d135e0bd5cd38 by Victor Stinner in branch 'master':
bpo-40522: Replace PyThreadState_GET() with PyThreadState_Get() (GH-24575)
https://github.com/python/cpython/commit/62078101ea1be5d2fc472a3f0d9d135e0bd5cd38