Some of these actually get misreported on the website, which may indicate a bug either in extra_tests/custom_text_test_runner.py or in the website code, particularly when class names are identical between files. So far, I only got the ones on Linux, as they're the most visible on the website.
Lib/test/_test_eintr.py
- class
SelectEINTRTest(EINTRBaseTest)
-
test_poll: timed out at the 10 minute mark
Lib/test/test_asyncio/test_events.py
- class
EPollEventLoopTests(EventLoopTestsMixin), class PollEventLoopTests(EventLoopTestsMixin), and class SelectEventLoopTests(EventLoopTestsMixin):
-
test_create_connection_local_addr_skip_different_family: (Linux only) Flaky on CI
Lib/test/test_asyncio/test_free_threading.py
- class
TestEagerPyFreeThreading(TestPyFreeThreading) and class TestPyFreeThreading
-
test_all_tasks_race: hangs - Python _current_tasks dict not thread-safe
Lib/test/test_asyncio/test_subprocess.py
- class
SubprocessWatcherMixin(SubprocessMixin)
-
test_close_dont_kill_finished (Linux only) Flaky on CI
Lib/test/test_pydoc/test_pydoc.py
- class
PydocUrlHandlerTest
-
test_content_type_err: Panic
Lib/test/test_array.py
- class
LargeArrayTest
-
test_setitem_use_after_clear_with_float_data: Hangs
-
test_setitem_use_after_clear_with_int_data: Hangs
-
test_setitem_use_after_shrink_with_int_data: Hangs
Lib/test/test_atexit.py
- class
FunctionalTest
-
test_atexit_thread_safety: Flakey on CI
Lib/test/test_builtin.py
- class
BuiltinTest
-
test_bytearray_join_with_misbehaving_iterator: Hangs
-
test_filter_dealloc: Segfault
- class
PtyTests
-
test_input_no_stdout_fileno: FAILURE, WORKER BUG
- class
TestType
-
test_namespace_order: random failure
Lib/test/test_bytes.py
- class
ByteArrayAsStringTest(FixedStringTest(test.string_tests.BaseTest))
-
test_replace_overflow: may only apply to 32-bit platforms
N.B. this skip may seem redundant on its face, but when I put it in there in 7ab8ab1, the test was running when it wasn't supposed to. Perhaps we have better detection now?
- class
BytesAsStringTest(FixedStringTest(test.string_tests.BaseTest))
-
test_replace_overflow: may only apply to 32-bit platforms
N.B. this skip may seem redundant on its face, but when I put it in there in 7ab8ab1, the test was running when it wasn't supposed to. Perhaps we have better detection now?
Lib/test/test_class.py
- class
ClassTests
-
testSFBug532646: segmentation fault
- class
TestInlineValues
-
test_store_attr_type_cache: unexpectedly long runtime
Lib/test/test_cmd_line_script.py
- class
CmdLineTest
-
test_repl_stderr_flush: test hang in middle
-
test_repl_stderr_flush_separate_stderr: test hang in middle
-
test_repl_stdout_flush: test hang in middle
-
test_repl_stdout_flush_separate_stderr: test hang in middle
Lib/test/test_codecencodings_cn.py
- class
Test_GB18030(test.multibytecodec_support.TestBase): unknown encoding: gb18030
N.B. 19 methods are skipped here
- class
Test_GB2312(test.multibytecodec_support.TestBase): unknown encoding: gb2312
N.B. 19 methods are skipped here
- class
Test_GBK(test.multibytecodec_support.TestBase): unknown encoding: gbk
N.B. 19 methods are skipped here
- class
Test_HZ(test.multibytecodec_support.TestBase): unknown encoding: hz
N.B. 19 methods are skipped here
Lib/test/test_codecencodings_hk.py
- class
Test_Big5HKSCS(test.multibytecodec_support.TestBase): unknown encoding: big5hkscs
N.B. 19 methods are skipped here
Lib/test/test_codecencodings_iso2022.py
- class
Test_ISO2022_JP(test.multibytecodec_support.TestBase): unknown encoding: iso2022_jp
N.B. 19 methods are skipped here
- class
Test_ISO2022_JP2(test.multibytecodec_support.TestBase): unknown encoding: iso2022_jp_2
N.B. 19 methods are skipped here
- class
Test_ISO2022_JP2004(test.multibytecodec_support.TestBase): unknown encoding: iso2022_jp_2004
N.B. 19 methods are skipped here
- class
Test_ISO2022_JP3(test.multibytecodec_support.TestBase): unknown encoding: iso2022_jp_3
N.B. 19 methods are skipped here
- class
Test_ISO2022_KR(test.multibytecodec_support.TestBase): unknown encoding: iso2022_kr
N.B. 19 methods are skipped here
Lib/test/test_codecencodings_jp.py
- class
Test_CP932(test.multibytecodec_support.TestBase): unknown encoding: cp932
N.B. 19 methods are skipped here
- class
Test_EUC_JISX0213(test.multibytecodec_support.TestBase): unknown encoding: euc_jisx0213
N.B. 19 methods are skipped here
- class
Test_EUC_JIS_2004(test.multibytecodec_support.TestBase): unknown encoding: euc_jis_2004
N.B. 19 methods are skipped here
- class
Test_EUC_JP_COMPAT(test.multibytecodec_support.TestBase): unknown encoding: euc_jp
N.B. 19 methods are skipped here
- class
Test_SJISX0213(test.multibytecodec_support.TestBase): unknown encoding: shift_jisx0213
N.B. 19 methods are skipped here
- class
Test_SJIS_2004(test.multibytecodec_support.TestBase): unknown encoding: shift_jis_2004
N.B. 19 methods are skipped here
- class
Test_SJIS_COMPAT(test.multibytecodec_support.TestBase): unknown encoding: shift_jis
N.B. 19 methods are skipped here
Lib/test/test_codecencodings_kr.py
- class
Test_CP949(test.multibytecodec_support.TestBase): unknown encoding: cp949
N.B. 19 methods are skipped here
- class
Test_EUCKR(test.multibytecodec_support.TestBase): unknown encoding: euc_kr
N.B. 19 methods are skipped here
- class
Test_JOHAB(test.multibytecodec_support.TestBase): unknown encoding: johab
N.B. 19 methods are skipped here
Lib/test/test_codecencodings_tw.py
- class
Test_Big5(test.multibytecodec_support.TestBase): unknown encoding: big5
N.B. 19 methods are skipped here
Lib/test/test_concurrent_futures/test_shutdown.py
- class
ProcessPoolForkProcessPoolShutdownTest(ProcessPoolShutdownTest(ExecutorShutdownTest))
-
test_cancel_futures: - flaky race condition on macOS
N.B. should be @unittest.skipIf(sys.platform == 'darwin', 'TODO: RUSTPYTHON, flaky race condition)
-
test_interpreter_shutdown: hangs
-
test_submit_after_interpreter_shutdown: Hangs
- class
ProcessPoolForkserverProcessPoolShutdownTest(ProcessPoolShutdownTest(ExecutorShutdownTest))
-
test_cancel_futures: - flaky race condition on macOS
N.B. should be @unittest.skipIf(sys.platform == 'darwin', 'TODO: RUSTPYTHON, flaky race condition)
-
test_interpreter_shutdown: hangs
-
test_submit_after_interpreter_shutdown: Hangs
- class
ProcessPoolSpawnProcessPoolShutdownTest(ProcessPoolShutdownTest(ExecutorShutdownTest))
-
test_cancel_futures: - flaky race condition on macOS
N.B. should be @unittest.skipIf(sys.platform == 'darwin', 'TODO: RUSTPYTHON, flaky race condition)
-
test_interpreter_shutdown: hangs
-
test_submit_after_interpreter_shutdown: Hangs
- class
ThreadPoolShutdownTest(ExecutorShutdownTest)
-
test_interpreter_shutdown: hangs
-
test_submit_after_interpreter_shutdown: Hangs
Lib/test/test_csv.py
- class
Test_Csv
test_read_skipinitialspace: slice index starts at 1 but ends at 0
Lib/test/test_ctypes/test_dllist.py
- class
ListSharedLibraries
-
test_lists_system:
N.B. no reason given other than "TODO: RUSTPYTHON"
Lib/test/datetimetester.py (invoked from Lib/test/test_datetime.py)
N.B. the *_Fast variant is not yet supported
Lib/test/test_descr.py
- class
ClassPropertiesAndMethods
-
test_gh55664: crash. "dict has non-string keys: [PyObject PyInt { value: 1 }]"
-
test_recursive_call: rustpython segmentation fault
-
test_slots_trash: rustpython segmentation fault
-
test_wrapper_segfault: rustpython segmentation fault
- class
MiscTests
-
test_type_lookup_mro_reference: rustpython panicked at 'dict has non-string keys: [PyObject PyBaseObject]'
- class
PicklingTests
-
test_issue24097:
N.B. no reason given other than "TODO: RUSTPYTHON"
Lib/test/test_dictviews.py
- class
DictSetTest
-
test_deeply_nested_repr: segfault
Lib/test/test_dict.py
Lib/test/test_exception_group.py
- class
DeepRecursionInSplitAndSubgroup
-
test_deep_split: Segfault
-
test_deep_subgroup: Segfault
Lib/test/test_enum.py
- class
OldTestFlag
-
test_unique_composite: flaky test
- class
OldTestIntFlag
-
test_unique_composite: flaky test
Lib/test/test_email/test__encoded_words.py
- class
TestDecode
-
test_invalid_character_in_charset: str has surrogates
Lib/test/test_email/test_email.py
- class
TestMessageAPI
-
test_get_param_linear_complexity: Takes a long time to the point of timeouting
N.B. "timeouting" should be "timing out" or just "timeout"
Lib/test/test_email/test_headerregistry.py
- class
TestUnstructuredHeader
-
test_value_rfc2047_gb2312_base64 via string_as_value: RustPython currently does not support non-utf8 encoding
N.B. The stated reason may be misleading.
Lib/test/test_fileio.py
- class
CAutoFileTests
-
test_syscalls_read: extra ioctl(TCGETS) syscall for isatty check
- class
PyAutoFileTests
-
test_syscalls_read: extra ioctl(TCGETS) syscall for isatty check
Lib/test/test_httplib.py
- class
HTTPSTest
-
test_local_bad_hostname: Flaky on CI
Lib/test/test_httpservers.py
- class
CGIHTTPServerTestCase
-
test_large_content_length_truncated: flaky test
Lib/test/test_import/__init__.py
- class
ImportTests
-
test_create_dynamic_null: _imp.create_dynamic not implemented
Lib/test/test_importlib/test_locks.py
- class
Frozen_DeadlockAvoidanceTests(DeadlockAvoidanceTests)
-
test_deadlock: sometimes hangs
-
test_no_deadlock: flaky test
- class
Frozen_LifetimeTests(LifetimeTests)
-
test_all_locks:
N.B. no reason given other than "TODO: RUSTPYTHON"
- class
Frozen_ModuleLockAsRLockTests(ModuleLockAsRLockTests) (uses class RLockTests from Lib/test/lock_tests.py)
-
test_acquire_contended: sometimes hangs
-
test_thread_leak: sometimes hangs
- class
Source_DeadlockAvoidanceTests(DeadlockAvoidanceTests)
-
test_deadlock: sometimes hangs
-
test_no_deadlock: flaky test
- class
Source_ModuleLockAsRLockTests(ModuleLockAsRLockTests) (uses class RLockTests from Lib/test/lock_tests.py)
-
test_acquire_contended: sometimes hangs
-
test_thread_leak: sometimes hangs
Lib/test/test_importlib/test_threaded_import.py
- class
ThreadedImportTests
-
test_concurrent_futures_circular_import: hang; Suspected cause of crashes in Windows CI - PermissionError: [WinError 32] Permission denied: "C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\test_python_0cdrhhs_\\test_python_6340æ"
-
test_multiprocessing_pool_circular_import: hang
-
test_parallel_module_init: flaky
Lib/test/test_int.py
- class
IntStrDigitLimitsTests
-
test_denial_of_service_prevented_str_to_int: flaky test
- class
IntSubclassStrDigitLimitsTests(IntStrDigitLimitsTests)
-
test_denial_of_service_prevented_str_to_int: flaky test
Lib/test/test_io.py
- class
CSignalsTest(SignalsTest)
-
test_interrupted_read_retry_buffered: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"
-
test_interrupted_write_retry_buffered: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"
-
test_interrupted_write_retry_text: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"
- class
PySignalsTest(SignalsTest)
-
test_interrupted_write_retry_buffered: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"
-
test_interrupted_write_retry_text: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"
Lib/test/test_list.py
- class
ListTest
-
test_equal_operator_modifying_operand: hang
-
test_list_resize_overflow: crash
-
test_lt_operator_modifying_operand: hang
Lib/test/test_logging.py
- class
ConfigDictTest
-
test_multiprocessing_queues: flaky EOFError
- class
HandlerTest
-
test_post_fork_child_no_deadlock: Flaky
Lib/test/test_marshal.py
- class
BugsTestCase
-
test_unmarshallable: unexpected payload for constant python value
- class
InstancingTestCase
-
testRecursion:
N.B. no reason given other than "TODO: RUSTPYTHON"
Lib/test/test_mmap.py
- class
MmapTests
test_flush_return_value: memmap2 doesn't throw OSError when offset is not a multiple of mmap.PAGESIZE on Linux
Lib/test/test_modulefinder.py
- class
ModuleFinderTest
-
test_replace_paths: panics in co_filename replacement
Lib/test/test_multiprocessing_fork/test_manager.py (hooks into Lib/test/_test_multiprocessing.py)
- class
WithManagerTestLock
-
test_repr_rlock: flaky timeout - thread start latency
- class
WithManagerTestPool
-
test_release_task_refs:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, reference counting differences')
Lib/test/test_multiprocessing_fork/test_processes.py (hooks into Lib/test/_test_multiprocessing.py)
- class
WithProcessesTestFinalize
-
test_thread_safety:
N.B. no reason given other than "TODO: RUSTPYTHON"
- class
WithProcessesTestLock
-
test_repr_rlock: flaky timeout - thread start latency
- class
WithProcessesTestPicklingConnections
-
test_pickling:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, hangs')
- class
WithProcessesTestPool
-
test_release_task_refs:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, reference counting differences')
Lib/test/test_multiprocessing_fork/test_threads.py (hooks into Lib/test/_test_multiprocessing.py)
- class
WithThreadsTestLock
-
test_repr_rlock: flaky timeout - thread start latency
- class
WithThreadsTestPool
-
test_release_task_refs:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, reference counting differences')
Lib/test/test_multiprocessing_forkserver/test_manager.py (hooks into Lib/test/_test_multiprocessing.py)
- class
WithManagerTestLock
-
test_repr_rlock: flaky timeout - thread start latency
- class
WithManagerTestPool
-
test_release_task_refs:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, reference counting differences')
Lib/test/test_multiprocessing_forkserver/test_processes.py (hooks into Lib/test/_test_multiprocessing.py)
- class
WithProcessesTestCondition
-
test_notify: flaky timeout
-
test_notify_n: flaky timeout
- class
WithProcessesTestFinalize
-
test_thread_safety:
N.B. no reason given other than "TODO: RUSTPYTHON"
- class
WithProcessesTestLock
-
test_repr_rlock: flaky BrokenPipeError, flaky ConnectionRefusedError, flaky ConnectionResetError, flaky EOFError
- class
WithProcessesTestPicklingConnections
-
test_pickling:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, hangs')
- class
WithProcessesTestPool
-
test_release_task_refs:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, reference counting differences')
Lib/test/test_multiprocessing_forkserver/test_threads.py (hooks into Lib/test/_test_multiprocessing.py)
- class
WithThreadsTestLock
-
test_repr_rlock: flaky timeout - thread start latency
- class
WithThreadsTestPool
-
test_release_task_refs:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, reference counting differences')
Lib/test/test_multiprocessing_spawn/test_manager.py (hooks into Lib/test/_test_multiprocessing.py)
- class
WithManagerTestLock
-
test_repr_rlock: flaky timeout - thread start latency
- class
WithManagerTestPool
-
test_release_task_refs:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, reference counting differences')
Lib/test/test_multiprocessing_spawn/test_processes.py (hooks into Lib/test/_test_multiprocessing.py)
- class
WithProcessesTestFinalize
-
test_thread_safety:
N.B. no reason given other than "TODO: RUSTPYTHON"
- class
WithProcessesTestLock
-
test_repr_rlock: flaky BrokenPipeError, flaky ConnectionRefusedError, flaky ConnectionResetError, flaky EOFError
- class
WithProcessesTestPicklingConnections
-
test_pickling:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, hangs')
- class
WithProcessesTestPool
-
test_release_task_refs:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, reference counting differences')
Lib/test/test_multiprocessing_fork/test_threads.py (hooks into Lib/test/_test_multiprocessing.py)
- class
WithThreadsTestLock
-
test_repr_rlock: flaky timeout - thread start latency
- class
WithThreadsTestPool
-
test_release_task_refs:
N.B. no reason given other than "TODO: RUSTPYTHON"; should say @unittest.skip('TODO: RUSTPYHON, reference counting differences')
-
test_terminate: flaky environment pollution when running rustpython -m test --fail-env-changed due to unknown reason
N.B. this decorator should be removed altogether
Lib/test/test_os.py
- class
TestScandir
-
test_attributes: flaky test
Lib/test/test_poll.py
- class
PollTests
-
test_threaded_poll: fd reallocation
Lib/test/test_pty.py
- class
PtyTest (conditioned on the presence of tty.getwinsize) tty.tcgetwinsize is required for setUp
Lib/test/test_range.py
- class
RangeTest
-
test_iterator_setstate: crash
-
test_iterator_unpickle_compat: crash
Lib/test/test_re.py
- class
ReTests
-
test_search_anchor_at_beginning: flaky, improve perf
Lib/test/test_regrtest.py
- class
ArgsTestCase
-
test_leak_tmp_file: flaky
-
test_python_command: flaky
-
test_random_seed_workers: flaky
- class
ProgramsTestCase
-
test_module_autotest: flaky
-
test_module_from_test_autotest: flaky
-
test_module_test: flaky
-
test_script_autotest: flaky
Lib/test/test_select.py
- class
SelectTestCase
-
test_select_mutated: timed out
Lib/test/test_socket.py
- class
SendfileUsingSendfileTest
-
testCount: os.sendfile count parameter not handled correctly; flaky
-
testWithTimeout: os.sendfile count parameter not handled correctly; flaky
Lib/test/test_sqlite3/test_dbapi.py
- class
UninitialisedConnectionTests
-
test_uninit_operations:
N.B. no reason given other than "TODO: RUSTPYTHON"
Lib/test/test_sqlite3/test_regression.py
- class
RecursiveUseOfCursors: recursive cursor use causes lock contention
N.B. 3 methods are skipped here
- class
RegressionTests
-
test_recursive_cursor_use: recursive cursor use causes lock contention
-
test_table_lock_cursor_non_readonly_select: recursive cursor use causes lock contention
Lib/test/test_ssl.py
- class
ThreadedTests
-
test_ssl_in_multiple_threads: flaky
-
test_thread_recv_while_main_thread_sends: Hangs
Lib/test/test_strtod.py
- class
StrtodTests
-
test_halfway_cases: fails on debug mode, flaky in release mode
-
test_parsing: flaky test
Lib/test/test_str.py
- class
StrTest(test.string_tests.StringLikeTest(test.string_tests.BaseTest))
-
test_replace_overflow: may only apply to 32-bit platforms
N.B. this skip may seem redundant on its face, but when I put it in there in 7ab8ab1, the test was running when it wasn't supposed to. Perhaps we have better detection now?
Lib/test/test_subprocess.py
- class
POSIXProcessTestCase
-
test_close_fds: flaky test
-
test_extra_groups_invalid_gid_t_values: clarify failure condition
-
test_pass_fds: flaky test
Lib/test/test_support.py
- class
TestSupport
-
test_recursion: - causes segfault in debug builds
Lib/test/test_sys.py
- class
SysModuleTest
-
test_ioencoding: cp424 encoding not supported, causes panic
Lib/test/test_sys_settrace.py
- class
JumpTestCase
-
test_no_jump_infinite_while_loop: infinite loop never exits
- class
RaisingTraceFuncTestCase
-
test_call: spurious error?
-
test_exception: spurious error?
-
test_line: spurious error?
-
test_return: spurious error?
- class
SkipLineEventsTraceTestCase(TraceTestCase): gc
N.B. 47 methods are skipped here
- class
TraceTestCase: gc
N.B. 47 methods are skipped here
Lib/test/test_tempfile.py
- class
TestTemporaryDirectory
-
test_cleanup_with_symlink_modes: pollutes environment on Windows
Lib/test/test_thread.py
- class
LockTests
-
test_acquire_contended: sometimes hangs
-
test_thread_leak: sometimes hangs
Lib/test/test_threading.py
- class
CRLockTests(lock_tests.RLockTests)
-
test_acquire_contended: sometimes hangs
-
test_different_thread: flaky test
-
test_thread_leak: sometimes hangs
- class
ConditionAsRLockTests(lock_tests.RLockTests)
-
test_acquire_contended: sometimes hangs
-
test_different_thread: flaky test
-
test_thread_leak: sometimes hangs
- class
InterruptMainTests
-
test_interrupt_main_subthread: flaky
- class
LockTests(lock_tests.LockTests):
-
test_acquire_contended: sometimes hangs
-
test_thread_leak: sometimes hangs
- class
PyRLockTests(lock_tests.RLockTests):
-
test_acquire_contended: sometimes hangs
-
test_thread_leak: sometimes hangs
- class
ThreadJoinOnShutdown
-
test_3_join_in_forked_from_thread: flaky test
-
test_reinit_tls_after_fork: - flaky, parking_lot mutex not fork-safe
- class
ThreadTests
-
test_is_alive_after_fork: flaky
-
test_limbo_cleanup: threading._start_new_thread not exposed
-
test_main_thread_after_fork_from_nonmain_thread: flaky; process timeout after fork
-
test_tstate_lock: Thread._tstate_lock not implemented
- class
ThreadingExceptionTests
-
test_recursion_limit: flaky test
Lib/test/test_threading_local.py
- class
PyThreadingLocalTest(BaseLocalTest)
-
test_local_refs: flaky test
- class
ThreadLocalTest(BaseLocalTest)
-
test_local_refs: flaky test
Lib/test/test_types.py
- class
SubinterpreterTests: no subinterpreters yet
N.B. 1 method is skipped here
Lib/test/test_userstring.py
- class
UserStringTest(string_tests.StringLikeTest(test.string_tests.BaseTest))
-
test_replace_overflow: may only apply to 32-bit platforms
N.B. this skip may seem redundant on its face, but when I put it in there in 7ab8ab1, the test was running when it wasn't supposed to. Perhaps we have better detection now?
Lib/test/test_weakref.py
- class
MappingTestCase
-
test_threaded_weak_key_dict_copy: occasionally crash (malloc corruption)
-
test_threaded_weak_key_dict_deepcopy: occasionally crash (malloc corruption)
-
test_threaded_weak_value_dict_copy: occasionally crash (malloc corruption)
-
test_threaded_weak_value_dict_deepcopy: occasionally crash (malloc corruption)
-
test_threaded_weak_valued_consistency: race condition between GC and WeakValueDictionary callback
-
test_threaded_weak_valued_setdefault: occasionally crash (malloc corruption)
Lib/test/test_zipfile/test_core.py
- class
EncodedMetadataTests: shift_jis encoding unsupported
N.B. 8 methods are skipped here
Lib/test/test_zipimport.py
- class
CompressedZipImportTestCase(UncompressedZipImportTestCase)
-
testZip64LargeFile: (intermittent success/failures); ValueError: name="RustPython/crates/pylib/Lib/test/zipimport_data/sparse-zip64-c0-0x000000000.part" does not fit expected pattern.
N.B. does this really need to be a skip? Or is it the case that this only fails in one class but not the other?
- class
UncompressedZipImportTestCase
-
testZip64LargeFile: (intermittent success/failures); ValueError: name="RustPython/crates/pylib/Lib/test/zipimport_data/sparse-zip64-c0-0x000000000.part" does not fit expected pattern.
N.B. does this really need to be a skip? Or is it the case that this only fails in one class but not the other?
Some of these actually get misreported on the website, which may indicate a bug either in
extra_tests/custom_text_test_runner.pyor in the website code, particularly when class names are identical between files. So far, I only got the ones on Linux, as they're the most visible on the website.Lib/test/_test_eintr.pySelectEINTRTest(EINTRBaseTest)test_poll: timed out at the 10 minute markLib/test/test_asyncio/test_events.pyEPollEventLoopTests(EventLoopTestsMixin), classPollEventLoopTests(EventLoopTestsMixin), and classSelectEventLoopTests(EventLoopTestsMixin):test_create_connection_local_addr_skip_different_family: (Linux only) Flaky on CILib/test/test_asyncio/test_free_threading.pyTestEagerPyFreeThreading(TestPyFreeThreading)and classTestPyFreeThreadingtest_all_tasks_race: hangs - Python_current_tasksdict not thread-safeLib/test/test_asyncio/test_subprocess.pySubprocessWatcherMixin(SubprocessMixin)test_close_dont_kill_finished(Linux only) Flaky on CILib/test/test_pydoc/test_pydoc.pyPydocUrlHandlerTesttest_content_type_err: PanicLib/test/test_array.pyLargeArrayTesttest_setitem_use_after_clear_with_float_data: Hangstest_setitem_use_after_clear_with_int_data: Hangstest_setitem_use_after_shrink_with_int_data: HangsLib/test/test_atexit.pyFunctionalTesttest_atexit_thread_safety: Flakey on CILib/test/test_builtin.pyBuiltinTesttest_bytearray_join_with_misbehaving_iterator: Hangstest_filter_dealloc: SegfaultPtyTeststest_input_no_stdout_fileno: FAILURE, WORKER BUGTestTypetest_namespace_order: random failureLib/test/test_bytes.pyByteArrayAsStringTest(FixedStringTest(test.string_tests.BaseTest))test_replace_overflow: may only apply to 32-bit platformsN.B. this skip may seem redundant on its face, but when I put it in there in 7ab8ab1, the test was running when it wasn't supposed to. Perhaps we have better detection now?
BytesAsStringTest(FixedStringTest(test.string_tests.BaseTest))test_replace_overflow: may only apply to 32-bit platformsN.B. this skip may seem redundant on its face, but when I put it in there in 7ab8ab1, the test was running when it wasn't supposed to. Perhaps we have better detection now?
Lib/test/test_class.pyClassTeststestSFBug532646: segmentation faultTestInlineValuestest_store_attr_type_cache: unexpectedly long runtimeLib/test/test_cmd_line_script.pyCmdLineTesttest_repl_stderr_flush: test hang in middletest_repl_stderr_flush_separate_stderr: test hang in middletest_repl_stdout_flush: test hang in middletest_repl_stdout_flush_separate_stderr: test hang in middleLib/test/test_codecencodings_cn.pyTest_GB18030(test.multibytecodec_support.TestBase): unknown encoding: gb18030N.B. 19 methods are skipped here
Test_GB2312(test.multibytecodec_support.TestBase): unknown encoding: gb2312N.B. 19 methods are skipped here
Test_GBK(test.multibytecodec_support.TestBase): unknown encoding: gbkN.B. 19 methods are skipped here
Test_HZ(test.multibytecodec_support.TestBase): unknown encoding: hzN.B. 19 methods are skipped here
Lib/test/test_codecencodings_hk.pyTest_Big5HKSCS(test.multibytecodec_support.TestBase): unknown encoding: big5hkscsN.B. 19 methods are skipped here
Lib/test/test_codecencodings_iso2022.pyTest_ISO2022_JP(test.multibytecodec_support.TestBase): unknown encoding: iso2022_jpN.B. 19 methods are skipped here
Test_ISO2022_JP2(test.multibytecodec_support.TestBase): unknown encoding: iso2022_jp_2N.B. 19 methods are skipped here
Test_ISO2022_JP2004(test.multibytecodec_support.TestBase): unknown encoding: iso2022_jp_2004N.B. 19 methods are skipped here
Test_ISO2022_JP3(test.multibytecodec_support.TestBase): unknown encoding: iso2022_jp_3N.B. 19 methods are skipped here
Test_ISO2022_KR(test.multibytecodec_support.TestBase): unknown encoding: iso2022_krN.B. 19 methods are skipped here
Lib/test/test_codecencodings_jp.pyTest_CP932(test.multibytecodec_support.TestBase): unknown encoding: cp932N.B. 19 methods are skipped here
Test_EUC_JISX0213(test.multibytecodec_support.TestBase): unknown encoding: euc_jisx0213N.B. 19 methods are skipped here
Test_EUC_JIS_2004(test.multibytecodec_support.TestBase): unknown encoding: euc_jis_2004N.B. 19 methods are skipped here
Test_EUC_JP_COMPAT(test.multibytecodec_support.TestBase): unknown encoding: euc_jpN.B. 19 methods are skipped here
Test_SJISX0213(test.multibytecodec_support.TestBase): unknown encoding: shift_jisx0213N.B. 19 methods are skipped here
Test_SJIS_2004(test.multibytecodec_support.TestBase): unknown encoding: shift_jis_2004N.B. 19 methods are skipped here
Test_SJIS_COMPAT(test.multibytecodec_support.TestBase): unknown encoding: shift_jisN.B. 19 methods are skipped here
Lib/test/test_codecencodings_kr.pyTest_CP949(test.multibytecodec_support.TestBase): unknown encoding: cp949N.B. 19 methods are skipped here
Test_EUCKR(test.multibytecodec_support.TestBase): unknown encoding: euc_krN.B. 19 methods are skipped here
Test_JOHAB(test.multibytecodec_support.TestBase): unknown encoding: johabN.B. 19 methods are skipped here
Lib/test/test_codecencodings_tw.pyTest_Big5(test.multibytecodec_support.TestBase): unknown encoding: big5N.B. 19 methods are skipped here
Lib/test/test_concurrent_futures/test_shutdown.pyProcessPoolForkProcessPoolShutdownTest(ProcessPoolShutdownTest(ExecutorShutdownTest))test_cancel_futures: - flaky race condition on macOSN.B. should be
@unittest.skipIf(sys.platform == 'darwin', 'TODO: RUSTPYTHON, flaky race condition)test_interpreter_shutdown: hangstest_submit_after_interpreter_shutdown: HangsProcessPoolForkserverProcessPoolShutdownTest(ProcessPoolShutdownTest(ExecutorShutdownTest))test_cancel_futures: - flaky race condition on macOSN.B. should be
@unittest.skipIf(sys.platform == 'darwin', 'TODO: RUSTPYTHON, flaky race condition)test_interpreter_shutdown: hangstest_submit_after_interpreter_shutdown: HangsProcessPoolSpawnProcessPoolShutdownTest(ProcessPoolShutdownTest(ExecutorShutdownTest))test_cancel_futures: - flaky race condition on macOSN.B. should be
@unittest.skipIf(sys.platform == 'darwin', 'TODO: RUSTPYTHON, flaky race condition)test_interpreter_shutdown: hangstest_submit_after_interpreter_shutdown: HangsThreadPoolShutdownTest(ExecutorShutdownTest)test_interpreter_shutdown: hangstest_submit_after_interpreter_shutdown: HangsLib/test/test_csv.pyTest_Csvtest_read_skipinitialspace: slice index starts at 1 but ends at 0Lib/test/test_ctypes/test_dllist.pyListSharedLibrariestest_lists_system:N.B. no reason given other than "TODO: RUSTPYTHON"
Lib/test/datetimetester.py(invoked fromLib/test/test_datetime.py)N.B. the
*_Fastvariant is not yet supportedLib/test/test_descr.pyClassPropertiesAndMethodstest_gh55664: crash. "dict has non-string keys:[PyObject PyInt { value: 1 }]"test_recursive_call: rustpython segmentation faulttest_slots_trash: rustpython segmentation faulttest_wrapper_segfault: rustpython segmentation faultMiscTeststest_type_lookup_mro_reference: rustpython panicked at 'dict has non-string keys:[PyObject PyBaseObject]'PicklingTeststest_issue24097:N.B. no reason given other than "TODO: RUSTPYTHON"
Lib/test/test_dictviews.pyDictSetTesttest_deeply_nested_repr: segfaultLib/test/test_dict.pyDictTesttest_repr_deep: segfaultLib/test/test_exception_group.pyDeepRecursionInSplitAndSubgrouptest_deep_split: Segfaulttest_deep_subgroup: SegfaultLib/test/test_enum.pyOldTestFlagtest_unique_composite: flaky testOldTestIntFlagtest_unique_composite: flaky testLib/test/test_email/test__encoded_words.pyTestDecodetest_invalid_character_in_charset: str has surrogatesLib/test/test_email/test_email.pyTestMessageAPItest_get_param_linear_complexity: Takes a long time to the point of timeoutingN.B. "timeouting" should be "timing out" or just "timeout"
Lib/test/test_email/test_headerregistry.pyTestUnstructuredHeadertest_value_rfc2047_gb2312_base64viastring_as_value: RustPython currently does not support non-utf8 encodingN.B. The stated reason may be misleading.
Lib/test/test_fileio.pyCAutoFileTeststest_syscalls_read: extra ioctl(TCGETS) syscall for isatty checkPyAutoFileTeststest_syscalls_read: extra ioctl(TCGETS) syscall for isatty checkLib/test/test_httplib.pyHTTPSTesttest_local_bad_hostname: Flaky on CILib/test/test_httpservers.pyCGIHTTPServerTestCasetest_large_content_length_truncated: flaky testLib/test/test_import/__init__.pyImportTeststest_create_dynamic_null:_imp.create_dynamicnot implementedLib/test/test_importlib/test_locks.pyFrozen_DeadlockAvoidanceTests(DeadlockAvoidanceTests)test_deadlock: sometimes hangstest_no_deadlock: flaky testFrozen_LifetimeTests(LifetimeTests)test_all_locks:N.B. no reason given other than "TODO: RUSTPYTHON"
Frozen_ModuleLockAsRLockTests(ModuleLockAsRLockTests)(uses classRLockTestsfromLib/test/lock_tests.py)test_acquire_contended: sometimes hangstest_thread_leak: sometimes hangsSource_DeadlockAvoidanceTests(DeadlockAvoidanceTests)test_deadlock: sometimes hangstest_no_deadlock: flaky testSource_ModuleLockAsRLockTests(ModuleLockAsRLockTests)(uses classRLockTestsfromLib/test/lock_tests.py)test_acquire_contended: sometimes hangstest_thread_leak: sometimes hangsLib/test/test_importlib/test_threaded_import.pyThreadedImportTeststest_concurrent_futures_circular_import: hang; Suspected cause of crashes in Windows CI - PermissionError: [WinError 32] Permission denied:"C:\\Users\\RUNNER~1\\AppData\\Local\\Temp\\test_python_0cdrhhs_\\test_python_6340æ"test_multiprocessing_pool_circular_import: hangtest_parallel_module_init: flakyLib/test/test_int.pyIntStrDigitLimitsTeststest_denial_of_service_prevented_str_to_int: flaky testIntSubclassStrDigitLimitsTests(IntStrDigitLimitsTests)test_denial_of_service_prevented_str_to_int: flaky testLib/test/test_io.pyCSignalsTest(SignalsTest)test_interrupted_read_retry_buffered: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"test_interrupted_write_retry_buffered: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"test_interrupted_write_retry_text: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"PySignalsTest(SignalsTest)test_interrupted_write_retry_buffered: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"test_interrupted_write_retry_text: thread 'main' (103833) panicked at crates/vm/src/stdlib/signal.rs:233:43: RefCell already borrowed"Lib/test/test_list.pyListTesttest_equal_operator_modifying_operand: hangtest_list_resize_overflow: crashtest_lt_operator_modifying_operand: hangLib/test/test_logging.pyConfigDictTesttest_multiprocessing_queues: flaky EOFErrorHandlerTesttest_post_fork_child_no_deadlock: FlakyLib/test/test_marshal.pyBugsTestCasetest_unmarshallable: unexpected payload for constant python valueInstancingTestCasetestRecursion:N.B. no reason given other than "TODO: RUSTPYTHON"
Lib/test/test_mmap.pyMmapTeststest_flush_return_value: memmap2 doesn't throw OSError when offset is not a multiple of mmap.PAGESIZE on LinuxLib/test/test_modulefinder.pyModuleFinderTesttest_replace_paths: panics in co_filename replacementLib/test/test_multiprocessing_fork/test_manager.py(hooks intoLib/test/_test_multiprocessing.py)WithManagerTestLocktest_repr_rlock: flaky timeout - thread start latencyWithManagerTestPooltest_release_task_refs:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, reference counting differences')Lib/test/test_multiprocessing_fork/test_processes.py(hooks intoLib/test/_test_multiprocessing.py)WithProcessesTestFinalizetest_thread_safety:N.B. no reason given other than "TODO: RUSTPYTHON"
WithProcessesTestLocktest_repr_rlock: flaky timeout - thread start latencyWithProcessesTestPicklingConnectionstest_pickling:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, hangs')WithProcessesTestPooltest_release_task_refs:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, reference counting differences')Lib/test/test_multiprocessing_fork/test_threads.py(hooks intoLib/test/_test_multiprocessing.py)WithThreadsTestLocktest_repr_rlock: flaky timeout - thread start latencyWithThreadsTestPooltest_release_task_refs:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, reference counting differences')Lib/test/test_multiprocessing_forkserver/test_manager.py(hooks intoLib/test/_test_multiprocessing.py)WithManagerTestLocktest_repr_rlock: flaky timeout - thread start latencyWithManagerTestPooltest_release_task_refs:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, reference counting differences')Lib/test/test_multiprocessing_forkserver/test_processes.py(hooks intoLib/test/_test_multiprocessing.py)WithProcessesTestConditiontest_notify: flaky timeouttest_notify_n: flaky timeoutWithProcessesTestFinalizetest_thread_safety:N.B. no reason given other than "TODO: RUSTPYTHON"
WithProcessesTestLocktest_repr_rlock: flaky BrokenPipeError, flaky ConnectionRefusedError, flaky ConnectionResetError, flaky EOFErrorWithProcessesTestPicklingConnectionstest_pickling:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, hangs')WithProcessesTestPooltest_release_task_refs:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, reference counting differences')Lib/test/test_multiprocessing_forkserver/test_threads.py(hooks intoLib/test/_test_multiprocessing.py)WithThreadsTestLocktest_repr_rlock: flaky timeout - thread start latencyWithThreadsTestPooltest_release_task_refs:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, reference counting differences')Lib/test/test_multiprocessing_spawn/test_manager.py(hooks intoLib/test/_test_multiprocessing.py)WithManagerTestLocktest_repr_rlock: flaky timeout - thread start latencyWithManagerTestPooltest_release_task_refs:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, reference counting differences')Lib/test/test_multiprocessing_spawn/test_processes.py(hooks intoLib/test/_test_multiprocessing.py)WithProcessesTestFinalizetest_thread_safety:N.B. no reason given other than "TODO: RUSTPYTHON"
WithProcessesTestLocktest_repr_rlock: flaky BrokenPipeError, flaky ConnectionRefusedError, flaky ConnectionResetError, flaky EOFErrorWithProcessesTestPicklingConnectionstest_pickling:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, hangs')WithProcessesTestPooltest_release_task_refs:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, reference counting differences')Lib/test/test_multiprocessing_fork/test_threads.py(hooks intoLib/test/_test_multiprocessing.py)WithThreadsTestLocktest_repr_rlock: flaky timeout - thread start latencyWithThreadsTestPooltest_release_task_refs:N.B. no reason given other than "TODO: RUSTPYTHON"; should say
@unittest.skip('TODO: RUSTPYHON, reference counting differences')test_terminate: flaky environment pollution when running rustpython -m test --fail-env-changed due to unknown reasonN.B. this decorator should be removed altogether
Lib/test/test_os.pyTestScandirtest_attributes: flaky testLib/test/test_poll.pyPollTeststest_threaded_poll: fd reallocationLib/test/test_pty.pyPtyTest(conditioned on the presence oftty.getwinsize)tty.tcgetwinsizeis required for setUpLib/test/test_range.pyRangeTesttest_iterator_setstate: crashtest_iterator_unpickle_compat: crashLib/test/test_re.pyReTeststest_search_anchor_at_beginning: flaky, improve perfLib/test/test_regrtest.pyArgsTestCasetest_leak_tmp_file: flakytest_python_command: flakytest_random_seed_workers: flakyProgramsTestCasetest_module_autotest: flakytest_module_from_test_autotest: flakytest_module_test: flakytest_script_autotest: flakyLib/test/test_select.pySelectTestCasetest_select_mutated: timed outLib/test/test_socket.pySendfileUsingSendfileTesttestCount: os.sendfile count parameter not handled correctly; flakytestWithTimeout: os.sendfile count parameter not handled correctly; flakyLib/test/test_sqlite3/test_dbapi.pyUninitialisedConnectionTeststest_uninit_operations:N.B. no reason given other than "TODO: RUSTPYTHON"
Lib/test/test_sqlite3/test_regression.pyRecursiveUseOfCursors: recursive cursor use causes lock contentionN.B. 3 methods are skipped here
RegressionTeststest_recursive_cursor_use: recursive cursor use causes lock contentiontest_table_lock_cursor_non_readonly_select: recursive cursor use causes lock contentionLib/test/test_ssl.pyThreadedTeststest_ssl_in_multiple_threads: flakytest_thread_recv_while_main_thread_sends: HangsLib/test/test_strtod.pyStrtodTeststest_halfway_cases: fails on debug mode, flaky in release modetest_parsing: flaky testLib/test/test_str.pyStrTest(test.string_tests.StringLikeTest(test.string_tests.BaseTest))test_replace_overflow: may only apply to 32-bit platformsN.B. this skip may seem redundant on its face, but when I put it in there in 7ab8ab1, the test was running when it wasn't supposed to. Perhaps we have better detection now?
Lib/test/test_subprocess.pyPOSIXProcessTestCasetest_close_fds: flaky testtest_extra_groups_invalid_gid_t_values: clarify failure conditiontest_pass_fds: flaky testLib/test/test_support.pyTestSupporttest_recursion: - causes segfault in debug buildsLib/test/test_sys.pySysModuleTesttest_ioencoding: cp424 encoding not supported, causes panicLib/test/test_sys_settrace.pyJumpTestCasetest_no_jump_infinite_while_loop: infinite loop never exitsRaisingTraceFuncTestCasetest_call: spurious error?test_exception: spurious error?test_line: spurious error?test_return: spurious error?SkipLineEventsTraceTestCase(TraceTestCase): gcN.B. 47 methods are skipped here
TraceTestCase: gcN.B. 47 methods are skipped here
Lib/test/test_tempfile.pyTestTemporaryDirectorytest_cleanup_with_symlink_modes: pollutes environment on WindowsLib/test/test_thread.pyLockTeststest_acquire_contended: sometimes hangstest_thread_leak: sometimes hangsLib/test/test_threading.pyCRLockTests(lock_tests.RLockTests)test_acquire_contended: sometimes hangstest_different_thread: flaky testtest_thread_leak: sometimes hangsConditionAsRLockTests(lock_tests.RLockTests)test_acquire_contended: sometimes hangstest_different_thread: flaky testtest_thread_leak: sometimes hangsInterruptMainTeststest_interrupt_main_subthread: flakyLockTests(lock_tests.LockTests):test_acquire_contended: sometimes hangstest_thread_leak: sometimes hangsPyRLockTests(lock_tests.RLockTests):test_acquire_contended: sometimes hangstest_thread_leak: sometimes hangsThreadJoinOnShutdowntest_3_join_in_forked_from_thread: flaky testtest_reinit_tls_after_fork: - flaky, parking_lot mutex not fork-safeThreadTeststest_is_alive_after_fork: flakytest_limbo_cleanup:threading._start_new_threadnot exposedtest_main_thread_after_fork_from_nonmain_thread: flaky; process timeout after forktest_tstate_lock:Thread._tstate_locknot implementedThreadingExceptionTeststest_recursion_limit: flaky testLib/test/test_threading_local.pyPyThreadingLocalTest(BaseLocalTest)test_local_refs: flaky testThreadLocalTest(BaseLocalTest)test_local_refs: flaky testLib/test/test_types.pySubinterpreterTests: no subinterpreters yetN.B. 1 method is skipped here
Lib/test/test_userstring.pyUserStringTest(string_tests.StringLikeTest(test.string_tests.BaseTest))test_replace_overflow: may only apply to 32-bit platformsN.B. this skip may seem redundant on its face, but when I put it in there in 7ab8ab1, the test was running when it wasn't supposed to. Perhaps we have better detection now?
Lib/test/test_weakref.pyMappingTestCasetest_threaded_weak_key_dict_copy: occasionally crash (malloc corruption)test_threaded_weak_key_dict_deepcopy: occasionally crash (malloc corruption)test_threaded_weak_value_dict_copy: occasionally crash (malloc corruption)test_threaded_weak_value_dict_deepcopy: occasionally crash (malloc corruption)test_threaded_weak_valued_consistency: race condition between GC and WeakValueDictionary callbacktest_threaded_weak_valued_setdefault: occasionally crash (malloc corruption)Lib/test/test_zipfile/test_core.pyEncodedMetadataTests: shift_jis encoding unsupportedN.B. 8 methods are skipped here
Lib/test/test_zipimport.pyCompressedZipImportTestCase(UncompressedZipImportTestCase)testZip64LargeFile: (intermittent success/failures); ValueError: name="RustPython/crates/pylib/Lib/test/zipimport_data/sparse-zip64-c0-0x000000000.part"does not fit expected pattern.N.B. does this really need to be a skip? Or is it the case that this only fails in one class but not the other?
UncompressedZipImportTestCasetestZip64LargeFile: (intermittent success/failures); ValueError: name="RustPython/crates/pylib/Lib/test/zipimport_data/sparse-zip64-c0-0x000000000.part"does not fit expected pattern.N.B. does this really need to be a skip? Or is it the case that this only fails in one class but not the other?