bpo-40522: Store tstate in a Thread Local Storage by vstinner · Pull Request #23976 · python/cpython

the-knights-who-say-ni

If Python is built with GCC or clang, the current interpreter and the
current Python thread state are now stored in a Thread Local Storage.

Changes:

* configure checks for C11 _Thread_local keyword.
* Use _Thread_local keyword, GCC and clang __thread extension.
* Add set_current_tstate() sub-function which sets these two new TLS
  variables (if available).
* _PyThreadState_Swap() and _PyThreadState_DeleteCurrent() now call
  set_current_tstate().
* _PyThreadState_GET() and _PyInterpreterState_GET() now use the TLS
  variable if available.

ericsnowcurrently added a commit that referenced this pull request

Apr 24, 2023

gh-103324)

We replace _PyRuntime.tstate_current with a thread-local variable. As part of this change, we add a _Py_thread_local macro in pyport.h (only for the core runtime) to smooth out the compiler differences. The main motivation here is in support of a per-interpreter GIL, but this change also provides some performance improvement opportunities.

Note that we do not provide a fallback to the thread-local, either falling back to the old tstate_current or to thread-specific storage (PyThread_tss_*()). If that proves problematic then we can circle back. I consider it unlikely, but will run the buildbots to double-check.

Also note that this does not change any of the code related to the GILState API, where it uses a thread state stored in thread-specific storage. I suspect we can combine that with _Py_tss_tstate (from here). However, that can be addressed separately and is not urgent (nor critical).

(While this change was mostly done independently, I did take some inspiration from earlier (~2020) work by @markshannon (main...markshannon:threadstate_in_tls) and @vstinner (#23976).)