gh-113628: Fix test_site test with long stdlib paths by itamaro · Pull Request #113640 · python/cpython

Summary:
upstream issue: python/cpython#113628
upstream PR: python/cpython#113640
upstream commit: python/cpython@5dc79e3

 ---

The `getpath` module [tries to read certain `_pth` files during initialization](https://github.com/python/cpython/blob/b4b2cc101216ae1017898dfbe43c90da2fd0a308/Modules/getpath.py#L462)).

This is tested in `test_site` with generated `_pth` files that [include the stdlib path 200 times](https://github.com/python/cpython/blob/b4b2cc101216ae1017898dfbe43c90da2fd0a308/Lib/test/test_site.py#L669).

The `getpath` module [disallows reading files over 32KB during initialization](https://github.com/python/cpython/blob/b4b2cc101216ae1017898dfbe43c90da2fd0a308/Modules/getpath.c#L375).

If the test suite runs from a very long base path, 200 repetitions of the stdlib path in the `_pth` file would be enough to exceed the 32KB limit.

To demonstrate, artificially increase the number of repetitions to some high number that would exceed 32KB, e.g. this patch:

```
 diff --git a/Lib/test/test_site.py b/Lib/test/test_site.py
 --- a/Lib/test/test_site.py
+++ b/Lib/test/test_site.py
@@ -631,5 +631,5 @@
         pth_lines = [
             'fake-path-name',
-            *[libpath for _ in range(200)],
+            *[libpath for _ in range(5000)],
             '',
             '# comment',
```

and observe the test failing with the following error:

```
python3.12 -m test test_site -v -m '*_pthFileTests.test_underpth_nosite_file'
== CPython 3.12.1+meta (3.12:2305ca5, Dec 07 2023, 21:46:47) [Clang 15.0.7 (mononoke://mononoke.internal.tfbnw.net/fbsource 3e29b2044f484840f
== Linux-5.12.0-0_fbk16_hardened_7661_geb00762ce6d2-x86_64-with-glibc2.34 little-endian
== Python build: release ThinLTO dtrace
== cwd: /tmp/test_python_worker_2504793æ
== CPU count: 72
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 3001644498
0:00:00 load avg: 6.52 Run 1 test sequentially
0:00:00 load avg: 6.52 [1/1] test_site
test_underpth_nosite_file (test.test_site._pthFileTests.test_underpth_nosite_file) ... Exception ignored error evaluating path:
Traceback (most recent call last):
  File "<frozen getpath>", line 463, in <module>
MemoryError: cannot read file larger than 32KB during initialization
Fatal Python error: error evaluating path
Python runtime state: core initialized

Current thread 0x00007fb01f2d0740 (most recent call first):
  <no Python frame>
ERROR

======================================================================
ERROR: test_underpth_nosite_file (test.test_site._pthFileTests.test_underpth_nosite_file)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/data/users/itamaro/fbsource/buck-out/v2/gen/fbsource/da203790281a65b9/third-party/python/3.12/__install-base__/out/install/lib/python3.12/test/test_site.py", line 647, in test_underpth_nosite_file
    output = subprocess.check_output([exe_file, '-c',
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/users/itamaro/fbsource/buck-out/v2/gen/fbsource/da203790281a65b9/third-party/python/3.12/__install-base__/out/install/lib/python3.12/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/users/itamaro/fbsource/buck-out/v2/gen/fbsource/da203790281a65b9/third-party/python/3.12/__install-base__/out/install/lib/python3.12/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/tmp/tmpsq9c4frs/python3.12', '-c', 'import sys; print("\\n".join(sys.path) if sys.flags.no_site else "")']' returned non-zero exit status 1.

----------------------------------------------------------------------
Ran 1 test in 0.019s

FAILED (errors=1)
test test_site failed
test_site failed (1 error)

== Tests result: FAILURE ==

1 test failed:
    test_site

Total duration: 118 ms
Total tests: run=1 (filtered)
Total test files: run=1/1 (filtered) failed=1
Result: FAILURE
```

See python/cpython#113628 for an upstream issue report.

Reviewed By: carljm

Differential Revision: D52475050

fbshipit-source-id: 3d8d19bc9058d8c78eb65fd4477e0c7e848c7e81