bpo-41710: PyThread_acquire_lock_timed() clamps the timout#28643

vstinner

The pthread implementation of PyThread_acquire_lock_timed() using
semaphore now clamps the timeout if it is too large, rather than
calling Py_FatalError() which aborts the process.

PyThread_acquire_lock_timed() no longer uses
MICROSECONDS_TO_TIMESPEC() to compute sem_timedwait() argument, but
_PyTime_GetSystemClock() and _PyTime_AsTimespec_truncate().

https://bugs.python.org/issue41710

PyThread_acquire_lock_timed() now clamps the timeout into the [_PyTime_MIN; _PyTime_MAX] range (_PyTime_t type) if it is too large, rather than calling Py_FatalError() which aborts the process. PyThread_acquire_lock_timed() no longer uses MICROSECONDS_TO_TIMESPEC() to compute sem_timedwait() argument, but _PyTime_GetSystemClock() and _PyTime_AsTimespec_truncate().

Fix _thread.TIMEOUT_MAX value on Windows: the maximum timeout is 0x7FFFFFFF milliseconds (around 24.9 days), not 0xFFFFFFFF milliseconds (around 49.7 days). Set PY_TIMEOUT_MAX to 0x7FFFFFFF milliseconds, rather than 0xFFFFFFFF milliseconds. Fix PY_TIMEOUT_MAX overflow test: replace (us >= PY_TIMEOUT_MAX) with (us > PY_TIMEOUT_MAX).

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL8 LTO 3.x has failed when building commit 37b8294.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/361/builds/762) and take a look at the build logs.
Check if the failure is related to this commit (37b8294) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/361/builds/762

Failed tests:

test_unicodedata

Failed subtests:

test_normalization - test.test_unicodedata.NormalizationTest

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

413 tests OK.

10 slowest tests:

test_peg_generator: 7 min 5 sec
test_hashlib: 6 min 22 sec
test_codecmaps_jp: 6 min 12 sec
test_codecmaps_cn: 3 min 5 sec
test_gdb: 2 min 55 sec
test_concurrent_futures: 2 min 50 sec
test_unicodedata: 2 min 48 sec
test_multiprocessing_spawn: 1 min 45 sec
test_multiprocessing_forkserver: 1 min 19 sec
test_socket: 1 min 17 sec

1 test failed:
test_unicodedata

12 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_startfile
test_tix test_tk test_ttk_guionly test_winconsoleio test_winreg
test_winsound test_zipfile64

2 re-run tests:
test_unicodedata test_urllib2net

1 test run no tests:
test_urllib2net

Total duration: 18 min 30 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_ccac20f8'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_4830bf8b'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/test/test_unicodedata.py", line 342, in test_normalization
    self.fail(f"Could not retrieve {TESTDATAURL}")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Could not retrieve http://www.pythontest.net/unicode/14.0.0/NormalizationTest.txt


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/test/test_unicodedata.py", line 336, in test_normalization
    testdata = open_urlresource(TESTDATAURL, encoding="utf-8",
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL8-ppc64le.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_6a94ca42'

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE RHEL7 3.x has failed when building commit 37b8294.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/446/builds/748) and take a look at the build logs.
Check if the failure is related to this commit (37b8294) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/446/builds/748

Failed tests:

test_unicodedata

Failed subtests:

test_normalization - test.test_unicodedata.NormalizationTest

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

412 tests OK.

10 slowest tests:

test_hashlib: 5 min 49 sec
test_unicodedata: 4 min 48 sec
test_urllib2net: 4 min 26 sec
test_codecmaps_jp: 4 min 9 sec
test_codecmaps_cn: 3 min 51 sec
test_concurrent_futures: 3 min 7 sec
test_multiprocessing_spawn: 2 min 47 sec
test_multiprocessing_forkserver: 2 min 42 sec
test_unparse: 2 min 38 sec
test_tokenize: 2 min 23 sec

1 test failed:
test_unicodedata

14 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_smtpnet
test_ssl test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

1 re-run test:
test_unicodedata

Total duration: 10 min 34 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le/build/Lib/test/test_unicodedata.py", line 342, in test_normalization
    self.fail(f"Could not retrieve {TESTDATAURL}")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Could not retrieve http://www.pythontest.net/unicode/14.0.0/NormalizationTest.txt


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-ppc64le/build/Lib/test/test_unicodedata.py", line 336, in test_normalization
    testdata = open_urlresource(TESTDATAURL, encoding="utf-8",
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable LTO + PGO 3.x has failed when building commit 37b8294.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/101/builds/672) and take a look at the build logs.
Check if the failure is related to this commit (37b8294) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/101/builds/672

Failed tests:

test_unicodedata

Failed subtests:

test_normalization - test.test_unicodedata.NormalizationTest

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

413 tests OK.

10 slowest tests:

test_hashlib: 8 min 41 sec
test_codecmaps_jp: 4 min 22 sec
test_concurrent_futures: 2 min 49 sec
test_unicodedata: 2 min 43 sec
test_codecmaps_cn: 2 min 20 sec
test_codecmaps_tw: 1 min 51 sec
test_multiprocessing_spawn: 1 min 46 sec
test_urllib2net: 1 min 34 sec
test_multiprocessing_forkserver: 1 min 10 sec
test_multiprocessing_fork: 1 min 2 sec

1 test failed:
test_unicodedata

13 tests skipped:
test_devpoll test_gdb test_ioctl test_kqueue test_msilib
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_unicodedata

Total duration: 12 min 29 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_97df2eec'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_unicodedata.py", line 342, in test_normalization
    self.fail(f"Could not retrieve {TESTDATAURL}")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Could not retrieve http://www.pythontest.net/unicode/14.0.0/NormalizationTest.txt


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_82320373'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/test/test_unicodedata.py", line 336, in test_normalization
    testdata = open_urlresource(TESTDATAURL, encoding="utf-8",
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto-pgo/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_be034f7b'

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot PPC64LE Fedora Stable LTO 3.x has failed when building commit 37b8294.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/541/builds/637) and take a look at the build logs.
Check if the failure is related to this commit (37b8294) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/541/builds/637

Failed tests:

test_unicodedata

Failed subtests:

test_normalization - test.test_unicodedata.NormalizationTest

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

414 tests OK.

10 slowest tests:

test_peg_generator: 6 min 38 sec
test_hashlib: 5 min 20 sec
test_codecmaps_kr: 4 min 28 sec
test_codecmaps_jp: 4 min 25 sec
test_urllib2net: 4 min 11 sec
test_codecmaps_cn: 4 min 1 sec
test_gdb: 2 min 52 sec
test_concurrent_futures: 2 min 40 sec
test_unicodedata: 2 min 5 sec
test_multiprocessing_spawn: 1 min 48 sec

1 test failed:
test_unicodedata

12 tests skipped:
test_devpoll test_ioctl test_kqueue test_msilib test_startfile
test_tix test_tk test_ttk_guionly test_winconsoleio test_winreg
test_winsound test_zipfile64

1 re-run test:
test_unicodedata

Total duration: 10 min 11 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto/build/Lib/test/test_unicodedata.py", line 336, in test_normalization
    testdata = open_urlresource(TESTDATAURL, encoding="utf-8",
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
http.client.IncompleteRead: IncompleteRead(0 bytes read)


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto/build/Lib/test/test_unicodedata.py", line 342, in test_normalization
    self.fail(f"Could not retrieve {TESTDATAURL}")
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: Could not retrieve http://www.pythontest.net/unicode/14.0.0/NormalizationTest.txt


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto/build/Lib/http/client.py", line 586, in _read_chunked
    value.append(self._safe_read(amt))
                 ^^^^^^^^^^^^^^^^^^^^
http.client.IncompleteRead: IncompleteRead(3263 bytes read, 4929 more expected)


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_4d4def68'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_9f619789'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main
    cache[rtype].remove(name)
    ^^^^^^^^^^^^^^^^^^^^^^^^^
KeyError: '/psm_be848612'


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.lto/build/Lib/test/test_unicodedata.py", line 336, in test_normalization
    testdata = open_urlresource(TESTDATAURL, encoding="utf-8",
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TimeoutError: timed out

vstinner added the skip news label Sep 30, 2021

the-knights-who-say-ni added the CLA signed label Sep 30, 2021

bedevere-bot added the awaiting core review label Sep 30, 2021

vstinner added 3 commits September 30, 2021 03:50

Fix test

d8e1989

vstinner requested a review from rhettinger as a code owner September 30, 2021 07:23

vstinner added 2 commits September 30, 2021 09:27

Fix typo

01f5238

Fix signed/unsigned comparison

60bc581

vstinner merged commit 37b8294 into python:main Sep 30, 2021

bedevere-bot removed the awaiting core review label Sep 30, 2021

vstinner deleted the thread_timeout branch September 30, 2021 08:16

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bpo-41710: PyThread_acquire_lock_timed() clamps the timout#28643