◐ Shell
reader mode source ↗
Skip to content

GH-59313: Do not allow incomplete tuples, or tuples with NULLs.#117747

Closed
markshannon wants to merge 19 commits into
python:mainfrom
faster-cpython:safer-tuples
Closed

GH-59313: Do not allow incomplete tuples, or tuples with NULLs.#117747
markshannon wants to merge 19 commits into
python:mainfrom
faster-cpython:safer-tuples

Conversation

@markshannon

@markshannon markshannon commented Apr 11, 2024

Copy link
Copy Markdown
Member

Prevents gc.get_referrers from returning invalid tuples, and should make the cycle GC a bit more robust.

Not using NULL allows some efficiency improvements, like changing XDECREF to DECREF.
I've also streamlined tuple_alloc, since I was fiddling with tuple creation and destruction anyway.
PySequence_Tuple takes 0.02% of the runtime of the benchmark suite, so any change to its performance will be undetectable.

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Can you please restrict changes to the bare minimum fix for gh-59313, to ease backport? It would be better to move other changes (ex: reject NULL) in a separated PR.

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

You should also update _PyTuple_Resize():

    /* Zero out items added by growing */
    if (newsize > oldsize)
        memset(&sv->ob_item[oldsize], 0,
               sizeof(*sv->ob_item) * (newsize - oldsize));

@vstinner

Copy link
Copy Markdown
Member

Overall, the change looks good to me, but I made a second review with more suggestions.

@erlend-aasland erlend-aasland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Some nits

@rhettinger rhettinger removed their request for review April 14, 2024 03:42
@markshannon

Copy link
Copy Markdown
Member Author

This change seems to break weakrefs.
I suspect that None and NULL are semantically different, so changing NULL to None in the tuples used in the weakref code changes the behavior.

Closing until I have time to investigate and fix it.

@markshannon markshannon reopened this Nov 8, 2024
@markshannon markshannon marked this pull request as draft November 8, 2024 14:17
@markshannon

Copy link
Copy Markdown
Member Author

It is possible that changing from NULL to None in PyTuple_New will break existing extensions.
The change to PySequence_Tuple has been moved to #127758

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants