gh-117139: Convert the evaluation stack to stack refs by Fidget-Spinner · Pull Request #118450 · python/cpython

…` arguments in configuration command after #118450 (#121083)



Signed-off-by: Manjusaka <me@manjusaka.me>
Co-authored-by: Ken Jin <kenjin4096@gmail.com>

## Description

This PR adds support for Python 3.14 in the profiler by updating it to
handle CPython internal changes.

### Key CPython changes addressed 

**`_PyInterpreterFrame` Structure Changes**
1. Moved from `Include/internal/pycore_frame.h` to
`Include/internal/pycore_interpframe_structs.h`
2. `PyObject *f_executable` and `PyObject *f_funcobj` changed to
`_PyStackRef` type. Profilers like us now need to clear the LSB of these
fields to get the `PyObject*`. See
python/cpython#123923 for details
3. `int stacktop` field removed, replaced with `_PyStackRef
*stackpointer` pointer. See
python/cpython#121923 (GH-120024) for details
4. `PyObject *localsplus[1]` changed to `_PyStackRef localsplus[1]`. See
python/cpython#118450 (gh-117139) for details

**`FutureObj`/`TaskObj` Changes**
1. Added fields: `awaited_by`, `is_task`, `awaited_by_is_set` in
`FutureObj_HEAD` macro
2. Added `struct llist_node_task_node` field for linked-list storage

**Asyncio Task Storage Changes**

Prior to Python 3.14, 
- All tasks are stored in `_scheduled_tasks` WeakSet
([exported](https://github.com/python/cpython/blob/e96367da1fdc1e1cf17ca523e93a127b1961b443/Modules/_asynciomodule.c#L3738)
from C extension)
- Eager tasks are stored in `_eager_tasks` set
([exported](https://github.com/python/cpython/blob/e96367da1fdc1e1cf17ca523e93a127b1961b443/Modules/_asynciomodule.c#L3742)
from C extension)

From Python 3.14,
- Native `asyncio.Tasks` are stored in a linked-list (`struct
llist_node`) per thread and per interpreter
-
[Per-thread](https://github.com/python/cpython/blob/0114178911f8713bfcb935ff5542fe61b4a5d551/Include/internal/pycore_tstate.h#L46):
`tstate->asyncio_tasks_head` (in `_PyThreadStateImpl`)
-
[Per-interpreter](https://github.com/python/cpython/blob/0114178911f8713bfcb935ff5542fe61b4a5d551/Include/internal/pycore_interp_structs.h#L897):
`interp->asyncio_tasks_head` (for lingering tasks)
- Each `TaskObj` has a `task_node` field with `next` and `prev` pointers
- Third-party tasks: Still stored in Python-level `_scheduled_tasks`
WeakSet (now Python-only, not exported from C extension)
- Eager tasks:  Still stored in Python-level `_eager_tasks` set


### Implementation Summary

- **Frame reading** (`frame.h`, `frame.cc`): Updated header includes to
use `pycore_interpframe_structs.h` for Python 3.14+. Implemented tagged
pointer handling: clear LSB of `f_executable` to recover `PyObject*`
(per gh-123923). Replaced `stacktop` field access with `stackpointer`
pointer arithmetic for stack depth calculation. Updated `PyGen_yf()` to
use `_PyStackRef` and `stackpointer[-1]` instead of
`localsplus[stacktop-1]`. Added handling for
`FRAME_OWNED_BY_INTERPRETER` frame type (introduced in 3.14).

- **Task structures** (`cpython/tasks.h`): Added Python 3.14+
`FutureObj_HEAD` macro with new fields: `awaited_by`, `is_task`,
`awaited_by_is_set`. Added `struct llist_node task_node` field to
`TaskObj` for linked-list storage. Updated `PyGen_yf()` implementation
to handle `_PyStackRef` and `stackpointer` instead of `stacktop`.

- **Asyncio discovery** (`tasks.h`, `threads.h`): Implemented
`get_tasks_from_linked_list()` to safely iterate over circular
linked-lists with iteration limits (`MAX_ITERATIONS = 2 << 15`). Added
`get_tasks_from_thread_linked_list()` to read tasks from
`_PyThreadStateImpl.asyncio_tasks_head` (per-thread active tasks). Added
`get_tasks_from_interpreter_linked_list()` to read lingering tasks from
`PyInterpreterState.asyncio_tasks_head` (per-interpreter). Updated
`get_all_tasks()` to handle both linked-list (native `asyncio.Task`
instances) and WeakSet (third-party tasks).

- **Python integration** (`_asyncio.py`): Added compatibility handling
for `BaseDefaultEventLoopPolicy` → `_BaseDefaultEventLoopPolicy` rename
in 3.14. Updated `_scheduled_tasks` access to handle Python-only WeakSet
(no longer exported from C extension in 3.14+).

## Testing

All existing tests pass except for
tests/profiling/collector/test_memalloc.py which needed some edits.

## Risks

<!-- Note any risks associated with this change, or "None" if no risks
-->

## Additional Notes

---------

Co-authored-by: Brett Langdon <brett.langdon@datadoghq.com>

Problem 1:

    golang/_golang.cpp: In function ‘void XPyFrame_WhiteoutFastLocal(PyFrameObject*, PyObject*)’:
    golang/_golang.cpp:1572:43: error: invalid use of incomplete type ‘_PyInterpreterFrame’ {aka ‘struct _PyInterpreterFrame’}
     1572 |                                 f->f_frame->localsplus;
          |                                           ^~
    In file included from /home/kirr/local/py3.14/include/python3.14/pyframe.h:19,
                     from /home/kirr/local/py3.14/include/python3.14/Python.h:113,
                     from golang/_golang.cpp:150:
    /home/kirr/local/py3.14/include/python3.14/cpython/pyframe.h:25:8: note: forward declaration of ‘_PyInterpreterFrame’ {aka ‘struct _PyInterpreterFrame’}
       25 | struct _PyInterpreterFrame;
          |        ^~~~~~~~~~~~~~~~~~~

-> include internal/pycore_interpframe.h like we already do with e.g. internal/pycore_frame.h .

Problem 2:

    golang/_golang.cpp: In function ‘void XPyFrame_WhiteoutFastLocal(PyFrameObject*, PyObject*)’:
    golang/_golang.cpp:1575:45: error: cannot convert ‘_PyStackRef*’ to ‘PyObject**’ {aka ‘_object**’} in initialization
     1575 |                                 f->f_frame->localsplus;
          |                                 ~~~~~~~~~~~~^~~~~~~~~~
          |                                             |
          |                                             _PyStackRef*

It turns out _PyInterpreterFrame.localsplus was changed from PyObject to _PyStackRef:

    python/cpython@22b0de2755ee2
    python/cpython#118450
    python/cpython#117139

-> Adjust the code to work with localsplus as _PyStackRef on py3.14+ .

/reviewed-by @jerome
/reviewed-on https://lab.nexedi.com/nexedi/pygolang/-/merge_requests/39