bpo-43517: Fix false positive in detection of circular imports#24895

pitrou

https://bugs.python.org/issue43517

miss-islington

Thanks @pitrou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

bedevere-bot

@pitrou: Please replace # with GH- in the commit message next time. Thanks!

miss-islington

Sorry, @pitrou, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 2fd16ef406bba239b1334057fb499496a84b3aa2 3.9

bedevere-bot

GH-24948 is a backport of this pull request to the 3.9 branch.

…H-24895) (GH-24948) (cherry picked from commit 2fd16ef) Co-authored-by: Antoine Pitrou <antoine@python.org> Automerge-Triggered-By: GH:pitrou

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86-64 macOS 3.x has failed when building commit 2fd16ef.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/366/builds/958) and take a look at the build logs.
Check if the failure is related to this commit (2fd16ef) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/366/builds/958

Failed tests:

test_httplib
test_importlib

Failed subtests:

testHTTPConnectionSourceAddress - test.test_httplib.SourceAddressTest

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/test/test_httplib.py", line 1539, in testHTTPConnectionSourceAddress
    self.conn.connect()
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/http/client.py", line 934, in connect
    self.sock = self._create_connection(
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/socket.py", line 843, in create_connection
    raise err
  File "/Users/buildbot/buildarea/3.x.billenstein-macos/build/Lib/socket.py", line 830, in create_connection
    sock.bind(source_address)
OSError: [Errno 48] Address already in use

* master: (129 commits) bpo-43452: Micro-optimizations to PyType_Lookup (pythonGH-24804) bpo-43517: Fix false positive in detection of circular imports (python#24895) bpo-43494: Make some minor changes to lnotab notes (pythonGH-24861) Mention that code.co_lnotab is deprecated in what's new for 3.10. (python#24902) bpo-43244: Remove symtable.h header file (pythonGH-24910) bpo-43466: Add --with-openssl-rpath configure option (pythonGH-24820) Fix a typo in c-analyzer (pythonGH-24468) bpo-41561: Add workaround for Ubuntu's custom security level (pythonGH-24915) bpo-43521: Allow ast.unparse with empty sets and NaN (pythonGH-24897) bpo-43244: Remove the PyAST_Validate() function (pythonGH-24911) bpo-43541: Fix PyEval_EvalCodeEx() regression (pythonGH-24918) bpo-43244: Fix test_peg_generators on Windows (pythonGH-24913) bpo-39342: Expose X509_V_FLAG_ALLOW_PROXY_CERTS in ssl module (pythonGH-18011) bpo-43244: Fix test_peg_generator for PyAST_Validate() (pythonGH-24912) bpo-42128: Add 'missing :' syntax error message to match statements (pythonGH-24733) bpo-43244: Add pycore_ast.h header file (pythonGH-24908) bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (pythonGH-24907) Remove unnecessary imports in the grammar parser (pythonGH-24904) bpo-35883: Py_DecodeLocale() escapes invalid Unicode characters (pythonGH-24843) Add PEP 626 to what's new in 3.10. (python#24892) ...

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora Clang Installed 3.9 has failed when building commit ac17ed6.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/570/builds/306) and take a look at the build logs.
Check if the failure is related to this commit (ac17ed6) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/570/builds/306

Failed tests:

test_importlib

Failed subtests:

test_multiprocessing_pool_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests
test_concurrent_futures_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

405 tests OK.

1 test failed:
test_importlib

19 tests skipped:
test_asdl_parser test_check_c_globals test_clinic test_devpoll
test_gdb test_ioctl test_kqueue test_msilib test_nis
test_ossaudiodev test_readline test_startfile test_tix test_tk
test_ttk_guionly test_winconsoleio test_winreg test_winsound
test_zipfile64

1 re-run test:
test_importlib

Total duration: 4 min 39 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/test_importlib/test_threaded_import.py", line 257, in test_multiprocessing_pool_circular_import
    script_helper.assert_python_ok(fn)
  File "/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 2
command line: ['/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/bin/python3.9', '-X', 'faulthandler', '-I', '/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/test_importlib/partial/pool_in_threads.py']


Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/test_importlib/test_threaded_import.py", line 251, in test_concurrent_futures_circular_import
    script_helper.assert_python_ok(fn)
  File "/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 2
command line: ['/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/bin/python3.9', '-X', 'faulthandler', '-I', '/home/dje/cpython-buildarea/3.9.edelsohn-fedora-z.clang-installed/build/target/lib/python3.9/test/test_importlib/partial/cfimport.py']

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86 Gentoo Installed with X 3.9 has failed when building commit ac17ed6.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/527/builds/315) and take a look at the build logs.
Check if the failure is related to this commit (ac17ed6) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/527/builds/315

Failed tests:

test_importlib

Failed subtests:

test_multiprocessing_pool_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests
test_concurrent_futures_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

411 tests OK.

1 test failed:
test_importlib

13 tests skipped:
test_asdl_parser test_check_c_globals test_clinic test_devpoll
test_gdb test_ioctl test_kqueue test_msilib test_startfile
test_winconsoleio test_winreg test_winsound test_zipfile64

1 re-run test:
test_importlib

Total duration: 29 min 25 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/test_importlib/test_threaded_import.py", line 257, in test_multiprocessing_pool_circular_import
    script_helper.assert_python_ok(fn)
  File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 2
command line: ['/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/bin/python3.9', '-X', 'faulthandler', '-I', '/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/test_importlib/partial/pool_in_threads.py']


Traceback (most recent call last):
  File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/test_importlib/test_threaded_import.py", line 251, in test_concurrent_futures_circular_import
    script_helper.assert_python_ok(fn)
  File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 2
command line: ['/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/bin/python3.9', '-X', 'faulthandler', '-I', '/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/test_importlib/partial/cfimport.py']

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable Clang Installed 3.9 has failed when building commit ac17ed6.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/17/builds/290) and take a look at the build logs.
Check if the failure is related to this commit (ac17ed6) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/17/builds/290

Failed tests:

test_importlib

Failed subtests:

test_multiprocessing_pool_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests
test_concurrent_futures_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

407 tests OK.

1 test failed:
test_importlib

17 tests skipped:
test_asdl_parser test_check_c_globals test_clinic test_devpoll
test_gdb test_ioctl test_kqueue test_msilib test_ossaudiodev
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_importlib

Total duration: 3 min 43 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/test_importlib/test_threaded_import.py", line 251, in test_concurrent_futures_circular_import
    script_helper.assert_python_ok(fn)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 2
command line: ['/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/bin/python3.9', '-X', 'faulthandler', '-I', '/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/test_importlib/partial/cfimport.py']


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/test_importlib/test_threaded_import.py", line 257, in test_multiprocessing_pool_circular_import
    script_helper.assert_python_ok(fn)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 2
command line: ['/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/bin/python3.9', '-X', 'faulthandler', '-I', '/home/buildbot/buildarea/3.9.cstratak-fedora-stable-x86_64.clang-installed/build/target/lib/python3.9/test/test_importlib/partial/pool_in_threads.py']

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable Clang Installed 3.9 has failed when building commit ac17ed6.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/283/builds/286) and take a look at the build logs.
Check if the failure is related to this commit (ac17ed6) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/283/builds/286

Failed tests:

test_importlib

Failed subtests:

test_multiprocessing_pool_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests
test_concurrent_futures_circular_import - test.test_importlib.test_threaded_import.ThreadedImportTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

407 tests OK.

1 test failed:
test_importlib

17 tests skipped:
test_asdl_parser test_check_c_globals test_clinic test_devpoll
test_gdb test_ioctl test_kqueue test_msilib test_ossaudiodev
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_importlib

Total duration: 4 min 21 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/test_importlib/test_threaded_import.py", line 257, in test_multiprocessing_pool_circular_import
    script_helper.assert_python_ok(fn)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 2
command line: ['/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/bin/python3.9', '-X', 'faulthandler', '-I', '/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/test_importlib/partial/pool_in_threads.py']


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/test_importlib/test_threaded_import.py", line 251, in test_concurrent_futures_circular_import
    script_helper.assert_python_ok(fn)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 156, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 142, in _assert_python
    res.fail(cmd_line)
  File "/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/support/script_helper.py", line 70, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 2
command line: ['/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/bin/python3.9', '-X', 'faulthandler', '-I', '/home/buildbot/buildarea/3.9.cstratak-fedora-stable-ppc64le.clang-installed/build/target/lib/python3.9/test/test_importlib/partial/cfimport.py']

pitrou requested review from brettcannon, encukou, ericsnowcurrently, ncoghlan and warsaw as code owners March 16, 2021 16:15

the-knights-who-say-ni added the CLA signed label Mar 16, 2021

bedevere-bot added the awaiting core review label Mar 16, 2021

bpo-43517: Fix false positive in detection of circular imports

59ee5e2

pitrou force-pushed the bpo43517-circular-import-misdetection branch from b138925 to 59ee5e2 Compare March 16, 2021 16:15

pitrou requested a review from methane March 16, 2021 16:19

pitrou added needs backport to 3.9 An unexpected behavior, bug, or error labels Mar 16, 2021

brettcannon approved these changes Mar 16, 2021

View reviewed changes

bedevere-bot added awaiting merge and removed awaiting core review labels Mar 16, 2021

methane approved these changes Mar 18, 2021

View reviewed changes

pitrou merged commit 2fd16ef into python:master Mar 20, 2021

pitrou deleted the bpo43517-circular-import-misdetection branch March 20, 2021 19:07

bedevere-bot removed the awaiting merge label Mar 20, 2021

miss-islington assigned pitrou Mar 20, 2021

bedevere-bot removed the needs backport to 3.9 label Mar 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-43517: Fix false positive in detection of circular imports#24895