◐ Shell
reader mode source ↗
Skip to content

gh-104615: don't make unsafe swaps in apply_static_swaps#104620

Merged
carljm merged 4 commits into
python:mainfrom
carljm:staticswaps
May 18, 2023
Merged

gh-104615: don't make unsafe swaps in apply_static_swaps#104620
carljm merged 4 commits into
python:mainfrom
carljm:staticswaps

Conversation

@carljm

@carljm carljm commented May 18, 2023 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member

It's not safe to apply SWAP statically if it would reorder two instructions that store to the same location.

@bedevere-bot bedevere-bot mentioned this pull request May 18, 2023
Closed
@carljm

carljm commented May 18, 2023

Copy link
Copy Markdown
Member Author

One thing I'm not clear on is whether we want to bump bytecode magic number for a fix like this. The old bytecode still works as well it ever did (no incompatible changes in the interpreter), so maybe no? But OTOH, nobody will see this bugfix until their pycs are invalidated.

Basically: do we bump bytecode magic number on any change to the compiler that results in different compiler output, or only when we change the interpreter in a way that makes it incompatible with previous bytecode?

@JelleZijlstra

JelleZijlstra commented May 18, 2023

Copy link
Copy Markdown
Member

One thing I'm not clear on is whether we want to bump bytecode magic number for a fix like this. The old bytecode still works as well it ever did (no incompatible changes in the interpreter), so maybe no? But OTOH, nobody will see this bugfix until their pycs are invalidated.

Basically: do we bump bytecode magic number on any change to the compiler that results in different compiler output, or only when we change the interpreter in a way that makes it incompatible with previous bytecode?

I was wondering a bit about that too. Several other recent fixes around comprehensions would also change the compiler output, so theoretically we should have bumped the magic number a few times. On the other hand, the only people who are really affected would be people who built from main directly after the PEP 695 change was merged (which did bump the magic number), so realistically there's not much of an issue. It's probably enough if we bump the magic number one last time before the beta is finalized.

@carljm carljm marked this pull request as ready for review May 18, 2023 20:35
@brandtbucher

brandtbucher commented May 18, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

My thought has mostly been that magic numbers should only be bumped if the bytecode itself is incompatible (meaning opcodes or the meanings of opargs have changed). I don't think we've bumped it for miscompilations before (but I could be wrong).

(We also bump it when changing how code objects are unmarshalled, which is arguably an abuse of the magic number and should be a marshal version bump instead.)

brandtbucher
brandtbucher approved these changes May 18, 2023
View reviewed changes

@brandtbucher brandtbucher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Thanks for the quick fix!

@carljm carljm enabled auto-merge (squash) May 18, 2023 21:00
@carljm carljm merged commit 0589c6a into python:main May 18, 2023
@carljm carljm deleted the staticswaps branch May 18, 2023 21:31
@carljm

carljm commented May 18, 2023

Copy link
Copy Markdown
Member Author

Oh, I forgot: this should be backported to 3.11, since it's a bug that exists there as well.

@carljm carljm added the needs backport to 3.11 only security fixes label May 18, 2023
@miss-islington

miss-islington commented May 18, 2023

Copy link
Copy Markdown
Contributor

Thanks @carljm for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

@miss-islington

miss-islington commented May 18, 2023

Copy link
Copy Markdown
Contributor

Sorry @carljm, I had trouble checking out the 3.11 backport branch.
Please retry by removing and re-adding the "needs backport to 3.11" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 0589c6a4d3d822cace42050198cb9a5e99c879ad 3.11

@carljm carljm added needs backport to 3.11 only security fixes and removed needs backport to 3.11 only security fixes labels May 18, 2023
@miss-islington

miss-islington commented May 18, 2023

Copy link
Copy Markdown
Contributor

Thanks @carljm for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

@miss-islington

miss-islington commented May 18, 2023

Copy link
Copy Markdown
Contributor

Sorry, @carljm, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 0589c6a4d3d822cace42050198cb9a5e99c879ad 3.11

carljm added a commit to carljm/cpython that referenced this pull request May 18, 2023
@carljm
Merge branch 'main' into nestedcomps …
54095f0 
* main:
  pythongh-74690: Don't set special protocol attributes on non-protocol subclasses of protocols (python#104622)
  pythongh-104623: Update Windows installer to use SQLite 3.42.0 (python#104625)
  pythongh-104050: Add more type annotations to Argument Clinic (python#104628)
  pythongh-104629: Don't skip test_clinic if _testclinic is missing (python#104630)
  pythongh-104549: Set __module__ on TypeAliasType (python#104550)
  pythongh-104050: Improve some typing around `default`s and sentinel values (python#104626)
  pythongh-104146: Remove unused vars from Argument Clinic (python#104627)
  pythongh-104615: don't make unsafe swaps in apply_static_swaps (python#104620)
  pythonGH-104484: Add case_sensitive argument to `pathlib.PurePath.match()` (pythonGH-104565)
  pythonGH-96803: Document and test new unstable internal frame API functions (pythonGH-104211)
  pythonGH-104580: Don't cache eval breaker in interpreter (pythonGH-104581)
carljm added a commit to carljm/cpython that referenced this pull request May 19, 2023
@carljm
[3.11] pythongh-104615: don't make unsafe swaps in apply_static_swaps (… …
eb83a26 
…pythonGH-104620).

(cherry picked from commit 0589c6a)

Co-authored-by: Carl Meyer <carl@oddbird.net>
@bedevere-bot

bedevere-bot commented May 19, 2023

Copy link
Copy Markdown

GH-104636 is a backport of this pull request to the 3.11 branch.

@bedevere-bot bedevere-bot removed the needs backport to 3.11 only security fixes label May 19, 2023
carljm added a commit that referenced this pull request May 19, 2023
@carljm
[3.11] gh-104615: don't make unsafe swaps in apply_static_swaps (GH-1… …
667e4ec 
…04620). (#104636)

(cherry picked from commit 0589c6a)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@iritkatriel iritkatriel iritkatriel approved these changes

@brandtbucher brandtbucher brandtbucher approved these changes

@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

Assignees

@carljm carljm

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@carljm @JelleZijlstra @brandtbucher @miss-islington @bedevere-bot @iritkatriel