◐ Shell
reader mode source ↗
Skip to content

bpo-31650: PEP 552 (Deterministic pycs) implementation#4575

Merged
benjaminp merged 14 commits into
masterfrom
benjamin-pep-552
Dec 9, 2017
Merged

bpo-31650: PEP 552 (Deterministic pycs) implementation#4575
benjaminp merged 14 commits into
masterfrom
benjamin-pep-552

Conversation

@benjaminp

@benjaminp benjaminp commented Nov 26, 2017

Copy link
Copy Markdown
Contributor

Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

  • The core changes to importlib to understand how to read, validate, and
    regenerate hash-based pycs.

  • Support for generating hash-based pycs in py_compile and compileall.

  • Modifications to our siphash implementation to support passing a custom
    key. We then expose it to importlib through _imp.

  • Updates to all places in the interpreter, standard library, and tests that
    manually generate or parse pyc files to grok the new format.

  • Support in the interpreter command line code for long options like
    --check-hash-based-pycs.

  • Tests and documentation for all of the above.

https://bugs.python.org/issue31650

@benjaminp benjaminp requested a review from a team November 26, 2017 22:39
@benjaminp benjaminp force-pushed the benjamin-pep-552 branch 4 times, most recently from 780bf48 to cd28db3 Compare November 27, 2017 07:31
1 hidden conversation Load more…

@benjaminp benjaminp left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Thanks for the comments! Update incoming.

@brettcannon brettcannon left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

Pretty much all of my comments are minor, so no need to have to double-check any changes made based on my feedback.

@gpshead gpshead left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hide comment

some minor comments to address but overall i believe this is good to go in when you are ready.

thanks!

Python now supports checking bytecode cache up-to-dateness with a hash of the
source contents rather than volatile source metadata. See the PEP for details.

While a fairly straightforward idea, quite a lot of code had to be modified due
to the pervasiveness of pyc implementation details in the codebase. Changes in
this commit include:

- The core changes to importlib to understand how to read, validate, and
  regenerate hash-based pycs.

- Support for generating hash-based pycs in py_compile and compileall.

- Modifications to our siphash implementation to support passing a custom
  key. We then expose it to importlib through _imp.

- Updates to all places in the interpreter, standard library, and tests that
  manually generate or parse pyc files to grok the new format.

- Support in the interpreter command line code for long options like
  --check-hash-based-pycs.

- Tests and documentation for all of the above.
@benjaminp benjaminp merged commit 42aa93b into master Dec 9, 2017
@benjaminp benjaminp deleted the benjamin-pep-552 branch December 9, 2017 18:26
@benjaminp

Copy link
Copy Markdown
Contributor Author

Thanks again for the reviews, Brett and Greg!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants