gh-109700: Improve stress tests for interpreter creation#109946

serhiy-storchaka

Ensure that destructors are called in the test that created interpreters, not after finishing it.
Try to create interpreters simultaneously in test_create_many_threaded.
Mark test_create_many_threaded that requires over 6GB of memory as bigmemtest.

Issue: test_interpreters: test_create_many_threaded() failed on FreeBSD: log: RuntimeError: interpreter creation failed #109700

…stressful

bedevere-bot

🤖 New build scheduled with the buildbot fleet by @serhiy-storchaka for commit 6aecbf9 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

serhiy-storchaka

While leaks are not yet fixed, we want to minimize effect of these tests on other tests.

serhiy-storchaka

Ensure that destructors are called in the test that created interpreters, not after finishing it.
Try to create/run interpreters in threads simultaneously.
Mark tests that requires over 6GB of memory with bigmemtest.

miss-islington-app

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

…nGH-109946) * Ensure that destructors are called in the test that created interpreters, not after finishing it. * Try to create/run interpreters in threads simultaneously. * Mark tests that requires over 6GB of memory with bigmemtest. (cherry picked from commit 61b50a9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

bedevere-app

GH-133391 is a backport of this pull request to the 3.13 branch.

…09946) (GH-133391) * Ensure that destructors are called in the test that created interpreters, not after finishing it. * Try to create/run interpreters in threads simultaneously. * Mark tests that requires over 6GB of memory with bigmemtest. (cherry picked from commit 61b50a9) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>

* origin/main: (111 commits) pythongh-91048: Add filename and line number to external inspection routines (pythonGH-133385) pythongh-131178: Add tests for `ast` command-line interface (python#133329) Regenerate pcbuild.sln in Visual Studio 2022 (python#133394) pythongh-133042: disable HACL* HMAC on Emscripten (python#133064) pythongh-133351: Fix remote PDB's multi-line block tab completion (python#133387) pythongh-109700: Improve stress tests for interpreter creation (pythonGH-109946) pythongh-81793: Skip tests for os.link() to symlink on Android (pythonGH-133388) pythongh-126835: Rename `ast_opt.c` to `ast_preprocess.c` and related stuff after moving const folding to the peephole optimizier (python#131830) pythongh-91048: Relax test_async_global_awaited_by to fix flakyness (python#133368) pythongh-132457: make staticmethod and classmethod generic (python#132460) pythongh-132805: annotationlib: Fix handling of non-constant values in FORWARDREF (python#132812) pythongh-132426: Add get_annotate_from_class_namespace replacing get_annotate_function (python#132490) pythongh-81793: Always call linkat() from os.link(), if available (pythonGH-132517) pythongh-122559: Synchronize C and Python implementation of the io module about pickling (pythonGH-122628) pythongh-69605: Add PyREPL import autocomplete feature to 'What's New' (python#133358) bpo-44172: Keep reference to original window in curses subwindow objects (pythonGH-26226) pythonGH-133231: Changes to executor management to support proposed `sys._jit` module (pythonGH-133287) pythongh-133363: Fix Cmd completion for lines beginning with `! ` (python#133364) pythongh-132983: Introduce `_zstd` bindings module (pythonGH-133027) pythonGH-91048: Add utils for printing the call stack for asyncio tasks (python#133284) ...

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Windows10 3.13 (tier-1) has failed when building commit 5daeebb.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1501/builds/1058) and take a look at the build logs.
Check if the failure is related to this commit (5daeebb) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1501/builds/1058

Failed tests:

test_venv

Failed subtests:

test_paths_depend_on_site_initialization - test.test_sysconfig.TestSysConfig.test_paths_depend_on_site_initialization

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line 203, in _force_run
    return func(*args)
PermissionError: [WinError 5] Access is denied: 'D:\\temp\\test_python_6l6mytan\\tmp2458yp30\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line 203, in _force_run
    return func(*args)
PermissionError: [WinError 5] Access is denied: 'D:\\temp\\test_python_s9fayd3i\\tmphcwb_78x\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\shutil.py", line 625, in _rmtree_unsafe
    os.unlink(fullname)
    ~~~~~~~~~^^^^^^^^^^
PermissionError: [WinError 5] Access is denied: 'd:\\temp\\test_python_5wmfyhpx\\test.test_sysconfig.TestSysConfig.test_paths_depend_on_site_initialization-venv-hp6qk7yr\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\test_venv.py", line 92, in tearDown
    rmtree(self.env_dir)
    ~~~~~~^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 470, in rmtree
    _rmtree(path)
    ~~~~~~~^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 413, in _rmtree
    _waitfor(_rmtree_inner, path, waitall=True)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 354, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 409, in _rmtree_inner
    _waitfor(_rmtree_inner, fullname, waitall=True)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 354, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 412, in _rmtree_inner
    _force_run(fullname, os.unlink, fullname)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line 214, in _force_run
    return func(*args)
PermissionError: [WinError 5] Access is denied: 'D:\\temp\\test_python_s9fayd3i\\tmphcwb_78x\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\test_sysconfig.py", line 653, in test_paths_depend_on_site_initialization
    with self.venv() as venv:
         ~~~~~~~~~^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\contextlib.py", line 148, in __exit__
    next(self.gen)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\venv.py", line 27, in from_tmpdir
    with tempfile.TemporaryDirectory(prefix=prefix, dir=dir, delete=delete) as tmpdir:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line 950, in __exit__
    self.cleanup()
    ~~~~~~~~~~~~^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line 954, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line 934, in _rmtree
    _shutil.rmtree(name, onexc=onexc)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\shutil.py", line 790, in rmtree
    return _rmtree_unsafe(path, onexc)
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\shutil.py", line 629, in _rmtree_unsafe
    onexc(os.unlink, fullname, err)
    ~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\tempfile.py", line 909, in onexc
    _os.unlink(path)
    ~~~~~~~~~~^^^^^^
PermissionError: [WinError 5] Access is denied: 'd:\\temp\\test_python_5wmfyhpx\\test.test_sysconfig.TestSysConfig.test_paths_depend_on_site_initialization-venv-hp6qk7yr\\Scripts\\python_d.exe'


Traceback (most recent call last):
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\test_venv.py", line 92, in tearDown
    rmtree(self.env_dir)
    ~~~~~~^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 470, in rmtree
    _rmtree(path)
    ~~~~~~~^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 413, in _rmtree
    _waitfor(_rmtree_inner, path, waitall=True)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 354, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 409, in _rmtree_inner
    _waitfor(_rmtree_inner, fullname, waitall=True)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 354, in _waitfor
    func(pathname)
    ~~~~^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\os_helper.py", line 412, in _rmtree_inner
    _force_run(fullname, os.unlink, fullname)
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\buildarea\3.13.bolen-windows10\build\Lib\test\support\__init__.py", line 214, in _force_run
    return func(*args)
PermissionError: [WinError 5] Access is denied: 'D:\\temp\\test_python_6l6mytan\\tmp2458yp30\\Scripts\\python_d.exe'

…nGH-109946) * Ensure that destructors are called in the test that created interpreters, not after finishing it. * Try to create/run interpreters in threads simultaneously. * Mark tests that requires over 6GB of memory with bigmemtest.

WIP: pythongh-109700: Make stress tests on interpreter creation more … …

91af60c

…stressful

bedevere-app Bot added the tests Tests in the Lib/test dir label Sep 27, 2023

Use less threads.

18031c9

bedevere-app Bot mentioned this pull request Sep 27, 2023

test_interpreters: test_create_many_threaded() failed on FreeBSD: log: RuntimeError: interpreter creation failed #109700

Closed

serhiy-storchaka added 2 commits November 8, 2023 22:25

Merge branch 'main' into test_interpreters-test_create_many_threaded

ef226df

Mark test_create_many_threaded with bigmemtest.

6aecbf9

serhiy-storchaka changed the title ~~WIP: gh-109700: Make stress tests on interpreter creation more stressful~~ Nov 8, 2023

serhiy-storchaka added skip news needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes 🔨 test-with-buildbots Test PR w/ buildbots; report in status section and removed only security fixes labels Nov 8, 2023

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 8, 2023

serhiy-storchaka marked this pull request as ready for review November 8, 2023 21:35

bedevere-app Bot added the awaiting core review label Nov 8, 2023

serhiy-storchaka changed the title ~~gh-109700: Improve stress tests on interpreter creation~~ Nov 8, 2023

serhiy-storchaka added the needs backport to 3.13 bugs and security fixes label May 9, 2024

Yhg1s removed the needs backport to 3.12 only security fixes label Apr 8, 2025

serhiy-storchaka added 2 commits May 4, 2025 20:02

Merge branch 'main' into test_interpreters-test_create_many_threaded

fd7cf3c

Add guards also for other stress tests.

5dd48d6

serhiy-storchaka requested a review from ericsnowcurrently as a code owner May 4, 2025 17:49

Polishing.

b3cf0b3

bedevere-app Bot removed the awaiting core review label May 4, 2025

serhiy-storchaka deleted the test_interpreters-test_create_many_threaded branch May 4, 2025 18:53

bedevere-app Bot removed the needs backport to 3.13 label May 4, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

gh-109700: Improve stress tests for interpreter creation#109946

gh-109700: Improve stress tests for interpreter creation#109946
serhiy-storchaka merged 7 commits into
python:mainfrom
serhiy-storchaka:test_interpreters-test_create_many_threaded

serhiy-storchaka commented Sep 27, 2023 •

edited

Loading

Uh oh!

bedevere-bot commented Nov 8, 2023

Uh oh!

serhiy-storchaka commented Nov 8, 2023

Uh oh!

serhiy-storchaka commented May 4, 2025

Uh oh!

Uh oh!

miss-islington-app Bot commented May 4, 2025

Uh oh!

bedevere-app Bot commented May 4, 2025

Uh oh!

bedevere-bot commented May 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

serhiy-storchaka commented Sep 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Nov 8, 2023

Uh oh!

serhiy-storchaka commented Nov 8, 2023

Uh oh!

serhiy-storchaka commented May 4, 2025

Uh oh!

Uh oh!

miss-islington-app Bot commented May 4, 2025

Uh oh!

bedevere-app Bot commented May 4, 2025

Uh oh!

bedevere-bot commented May 4, 2025

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

serhiy-storchaka commented Sep 27, 2023 •

edited

Loading