◐ Shell
reader mode source ↗
Skip to content

[WIP] bpo-42671: Make Python finalization deterministic#23826

Closed
vstinner wants to merge 1 commit into
python:mainfrom
vstinner:finalize_modules
Closed

[WIP] bpo-42671: Make Python finalization deterministic#23826
vstinner wants to merge 1 commit into
python:mainfrom
vstinner:finalize_modules

Conversation

@vstinner

@vstinner vstinner commented Dec 17, 2020

Copy link
Copy Markdown
Member

Make the Python finalization more deterministic:

  • First, clear the main module.
  • Then, clear modules from the most recently imported to the least
    recently imported: reversed(sys.modules).
  • builtins and sys modumes are always cleared last.
  • Module attributes are set to None from the most recently defined to the
    least recently defined: reversed(module.dict).

Changes:

  • finalize_modules() no longer uses a list of weak references to
    modules while clearing sys.modules dict.
  • When -vv command line option is used, the module name is now also
    logged, not only the attribute name.
  • test_module.test_module_finalization_at_shutdown(): final_a.x is
    now None when final_a.c is cleared.
  • test_sys.test_sys_ignores_cleaning_up_user_data(): the exception is
    no longer silently ignored. Rename the test to
    test_sys_cleaning_up_user_data().
  • test_threading.test_main_thread_during_shutdown() keeps a reference
    to threading functions since threading module variables are cleared
    before RefCycle object is deleted by the garbage collector.

https://bugs.python.org/issue42671

Make the Python finalization more deterministic:

* First, clear the __main__ module.
* Then, clear modules from the most recently imported to the least
  recently imported: reversed(sys.modules).
* builtins and sys modumes are always cleared last.
* Module attributes are set to None from the most recently defined to the
  least recently defined: reversed(module.__dict__).

Changes:

* finalize_modules() no longer uses a list of weak references to
  modules while clearing sys.modules dict.
* When -vv command line option is used, the module name is now also
  logged, not only the attribute name.
* test_module.test_module_finalization_at_shutdown(): final_a.x is
  now None when final_a.c is cleared.
* test_sys.test_sys_ignores_cleaning_up_user_data(): the exception is
  no longer silently ignored. Rename the test to
  test_sys_cleaning_up_user_data().
* test_threading.test_main_thread_during_shutdown() keeps a reference
  to threading functions since threading module variables are cleared
  before RefCycle object is deleted by the garbage collector.
@vstinner

Copy link
Copy Markdown
Member Author

I will update my PR later to take Serhiy's comment in account.

@github-actions

Copy link
Copy Markdown

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions Bot added the stale label Jan 18, 2021
@vstinner vstinner closed this Sep 21, 2021
@vstinner vstinner deleted the finalize_modules branch September 21, 2021 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

awaiting core review stale Stale PR or inactive for long period of time.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants