[WIP, DO NOT MERGE] bpo-41188: Prepare CPython for opague PyObject structure.#21262
[WIP, DO NOT MERGE] bpo-41188: Prepare CPython for opague PyObject structure.#21262WildCard65 wants to merge 6 commits into
Conversation
This member is the companion member required for 'Py_TPFLAGS_USES_OPAQUE_OBJECT' flag.
vstinner
left a comment
There was a problem hiding this comment.
Add implementation defined "slib_python310" static library for end users to link against.
I don't understand this.
Change as many types as possible to use 'Py_TPFLAGS_USES_OPAQUE_OBJECT'
Please don't do that in the same PR.
Sorry, something went wrong.
|
I would prefer to open a separated issue on bugs.python.org for this work. |
Sorry, something went wrong.
|
@vstinner The static library will contain '.c' files that reference internal implementations FOR the runtime it's designed for. As it currently stands, the current Py_TYPE still requires direct access to internals for end users while this static library remove that requirement. example would be: This allows for implementation independent access while still allowing the C compiler to aggressively inline out the small functions. |
Sorry, something went wrong.
…thon's internals and external users. Note: This static library is incomplete.
|
@vstinner Before I proceed any further, I want you to review "slib_pythoncore" first. |
Sorry, something went wrong.
This is the purpose of the limited C API which provides a stable ABI. It's already implemented since Python 3.2. I don't see the point of adding a second flavor of the "libpython" library. |
Sorry, something went wrong.
|
As it stands, functions like _Py_TYPE, _Py_SIZE, _Py_INCREF, _Py_DECREF rely on "PyObject" being a complete type. Opaque objects on the other hand are classified as incomplete types. Incomplete types can't be used for anything but pointers (*) and references (&), these functions will fail when PyObject goes opaque in the limited API, infact, sizeof() also fails on incomplete types. The purpose of the static library is not to be a different flavour of Python, but to act as a bridge into Python's internals while respecting the limitations of incomplete types, while also eliminating function call overhead. As an added bonus, this static library can be "reskinned" for runtimes that take a different approach to PyObject (if any) and descendant built-in types while end users don't have to worry about what the runtime's schematic is. |
Sorry, something went wrong.
|
Added a basic implementation of the flag, but I do not know where all usages of "tp_basicsize" is, I'm also debating on whether or not I need to make GET_TYPE_TOTALSIZE into an inlinable function. |
Sorry, something went wrong.
vstinner
left a comment
There was a problem hiding this comment.
IMHO you gone too far in the implementation before discussing the design. You should first propose the design on python-ideas or python-dev list.
Sorry, something went wrong.
|
Py_TPFLAGS_USES_OPAQUE_OBJECT looks useful, but other changes like "slib" looks useless to me. |
Sorry, something went wrong.
vstinner
left a comment
There was a problem hiding this comment.
I don't see the point of tp_obj_offset, tp_obj_size and slib.
Sorry, something went wrong.
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
Sorry, something went wrong.
|
Consider the following: struct myobject;
void use_myobject(struct myobect *obj); // Valid, incomplete types can be used as pointers.
size_t get_object_value(struct my object *obj)
{
return obj->m_value; // ERROR: my_object is an incomplete type!
}Above demonstrates why functions like _Py_Type will fail in limited api. 'Py_TPFLAGS_OMIT_OBJECT_SIZE' was intended as the stepping stone for making types like PyTypeObject opaque as well, this makes knowing the true offset off the base time a runtime dependant scenario, one with minimal impact to performance, probably less than having _Py_TYPE, etc, exported from python binary. |
Sorry, something went wrong.
|
@vstinner One way or another, PY_TYPE and co need to have their implementations PURGED from the Python header files for a truly opaque PyObject. Another thing that needs to be purged is "ob_base" usage of objects like PyTupleObject, PyTypeObject, PyListObject to name a few. I'm willing to replace slib with a precompiled header trio, but that's a can of worms on its own (at least on MSVC). I'm not budging in regards to "tp_obj_offset" and "tp_obj_size", see above mentioned purging. |
Sorry, something went wrong.
The end goal of this pull request is to make the structure "PyObject" opaque to EVERYTHING except Python core.
To accomplish this, the following must be done:
https://bugs.python.org/issue41188