Petr Viktorin:
> So, this will break Cython and gevent,
This change doesn't break Cython and gevent: they are already broken.
> but (unlike the optimization work that broke f_code/f_frame) it won't provide any value to users?
The problem is that the C API changed silently: existing code which gets directly PyFrameObject.f_back still compiles successfully, but it will no longer work in some cases.
See bpo-46356 "[C API] Enforce usage of PyFrame_GetBack()" for more details.
The intent of moving the structure to the internal C API is to clarify its status: we provide no backward compatibility warranty, you are on our own if you use it.
It's also a way to promote the usage of the new clean public C API: it is now reliable, whereas accessing directly PyFrameObject members break at each Python version.
The internal C API cannot be used easily on purpose: you have to opt-in for this API by defining the Py_BUILD_CORE_MODULE macro and you need to use different #include. It's a way to enforce the usage of the clean public C API.