[3.9] gh-95778: CVE-2020-10735: Prevent DoS by very large int()#96502

gpshead

Integer to and from text conversions via CPython's bignum int type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.

This PR comes fresh from a pile of work done in our private PSRT security response team repo.

This backports #96499 aka 511ca94

Signed-off-by: Christian Heimes [Red Hat] christian@python.org
Tons-of-polishing-up-by: Gregory P. Smith [Google] greg@krypto.org
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).

Issue: CVE-2020-10735: Prevent DoS by large int<->str conversions #95778

I wrote up a one pager for the release managers.

Co-authored-by: Christian Heimes <christian@python.org>

Ned pointed this out on the 3.7 review, it matches other patch changes and stands out better.

…nto CVE-2020-10735-3.9backport

bedevere-bot

🤖 New build scheduled with the buildbot fleet by @tiran for commit 3e7eb9c 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

gpshead

I've been using the opening message text for the PR as the commit message when merging. The automatic one is gross.

gpshead

wait for #96537 to be integrated into this PR before merging, i'll remove the do-not-merge label then.

python#96542

Per mdickinson@'s comment on the main branch PR.

…#96537) Converting a large enough `int` to a decimal string raises `ValueError` as expected. However, the raise comes _after_ the quadratic-time base-conversion algorithm has run to completion. For effective DOS prevention, we need some kind of check before entering the quadratic-time loop. Oops! =) The quick fix: essentially we catch _most_ values that exceed the threshold up front. Those that slip through will still be on the small side (read: sufficiently fast), and will get caught by the existing check so that the limit remains exact. The justification for the current check. The C code check is: ```c max_str_digits / (3 * PyLong_SHIFT) <= (size_a - 11) / 10 ``` In GitHub markdown math-speak, writing $M$ for `max_str_digits`, $L$ for `PyLong_SHIFT` and $s$ for `size_a`, that check is: $$\left\lfloor\frac{M}{3L}\right\rfloor \le \left\lfloor\frac{s - 11}{10}\right\rfloor$$ From this it follows that $$\frac{M}{3L} < \frac{s-1}{10}$$ hence that $$\frac{L(s-1)}{M} > \frac{10}{3} > \log_2(10).$$ So $$2^{L(s-1)} > 10^M.$$ But our input integer $a$ satisfies $|a| \ge 2^{L(s-1)}$, so $|a|$ is larger than $10^M$. This shows that we don't accidentally capture anything _below_ the intended limit in the check.  * Issue: pythongh-95778  Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>

bedevere-bot

🤖 New build scheduled with the buildbot fleet by @gpshead for commit a1247fd 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

ambv

@tiran can wasm32 failures be ignored here? This is a 3.9 backport and that version is not supported on WASM?

Those fail to build with errors like FileNotFoundError: [Errno 2] No such file or directory: b'../../Tools/wasm/wasi-env', configure: error: cross build not supported for wasm32-unknown-emscripten, and emconfigure: error: (...) failed (returned 1).

ambv

The FeeBSD Shared PR failure is unrelated. It's an openssl test failure due to trying to use a protocol that is not supported on the machine:

ValueError: invalid or unsupported protocol version

In GH-95312 we did backport test changes that were supposed to fix this but apparently not. This is to be followed up separately.

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD Shared 3.9 has failed when building commit cec1e9d.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/151/builds/591) and take a look at the build logs.
Check if the failure is related to this commit (cec1e9d) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/151/builds/591

Failed tests:

test_ssl

Failed subtests:

test__create_stdlib_context - test.test_ssl.ContextTests
test_session_stats - test.test_ssl.ContextTests
test_protocol - test.test_ssl.ContextTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

406 tests OK.

10 slowest tests:

test_tokenize: 8 min 37 sec
test_multiprocessing_spawn: 7 min 23 sec
test_lib2to3: 6 min 28 sec
test_gdb: 5 min 21 sec
test_multiprocessing_forkserver: 4 min 50 sec
test_concurrent_futures: 4 min 30 sec
test_unparse: 4 min 10 sec
test_asyncio: 3 min 49 sec
test_capi: 3 min 12 sec
test_pickle: 3 min 7 sec

1 test failed:
test_ssl

18 tests skipped:
test_dbm_gnu test_devpoll test_epoll test_idle test_ioctl
test_msilib test_spwd test_startfile test_tcl test_tix test_tk
test_ttk_guionly test_ttk_textonly test_turtle test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_ssl

Total duration: 33 min 15 sec

Click to see traceback logs

Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1718, in test__create_stdlib_context
    ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1,
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/ssl.py", line 776, in _create_unverified_context
    context = SSLContext(protocol)
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/ssl.py", line 484, in __new__
    self = _SSLContext.__new__(cls, protocol)
ValueError: invalid or unsupported protocol version


Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1144, in test_protocol
    ctx = ssl.SSLContext(proto)
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/ssl.py", line 484, in __new__
    self = _SSLContext.__new__(cls, protocol)
ValueError: invalid or unsupported protocol version


Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/test/test_ssl.py", line 1527, in test_session_stats
    ctx = ssl.SSLContext(proto)
  File "/usr/home/buildbot/python/3.9.koobs-freebsd-564d/build/Lib/ssl.py", line 484, in __new__
    self = _SSLContext.__new__(cls, protocol)
ValueError: invalid or unsupported protocol version

gpshead and others added 9 commits August 19, 2022 01:27

Backport to 3.9 of psrt/CVE-2020-10735-3.10backport.

576d772

Fix versionadded/versionchanged.

e5cd3fc

headers += Include/internal/pycore_long.h

9a3cd05

Update attribution in Misc/NEWS.d …

c190dc9

Co-authored-by: Christian Heimes <christian@python.org>

Manually add new field to abi file

a7213dd

Move the whatsnew text per review. …

2384ef4

Ned pointed this out on the 3.7 review, it matches other patch changes and stands out better.

Merge branch 'CVE-2020-10735-3.9backport' of github.com:python/psrt i… …

fcb123b

…nto CVE-2020-10735-3.9backport

Make the doctest actually run & fix it.

1378bde

Fix the docs build.

e51a8e2

gpshead assigned ambv and gpshead Sep 2, 2022

bedevere-bot added the awaiting core review label Sep 2, 2022

Rename the news file to appease the Bedevere bot.

3e7eb9c

gpshead added the release-blocker label Sep 2, 2022

tiran added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 2, 2022

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 2, 2022

hexadecimal spelling =)

5c64ec6

gpshead marked this pull request as ready for review September 2, 2022 16:52

gpshead requested review from lysnikolaou, pablogsal and rhettinger as code owners September 2, 2022 16:52

gpshead removed their assignment Sep 2, 2022

gpshead added the DO-NOT-MERGE label Sep 4, 2022

gpshead and others added 4 commits September 3, 2022 23:17

doc typo: limitation …

a37041f

python#96542

Misc: Fix a typo in the header comment.

b4957d8

remove unneeded doc note on float.as_integer_ratio …

0515141

Per mdickinson@'s comment on the main branch PR.

gpshead added 🔨 test-with-buildbots Test PR w/ buildbots; report in status section and removed DO-NOT-MERGE labels Sep 4, 2022

bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Sep 4, 2022

ambv merged commit cec1e9d into python:3.9 Sep 5, 2022

bedevere-bot removed the awaiting core review label Sep 5, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[3.9] gh-95778: CVE-2020-10735: Prevent DoS by very large int()#96502

[3.9] gh-95778: CVE-2020-10735: Prevent DoS by very large int()#96502
ambv merged 15 commits into
python:3.9from
gpshead:CVE-2020-10735-3.9backport

gpshead commented Sep 2, 2022 •

edited

Loading

Uh oh!

bedevere-bot commented Sep 2, 2022

Uh oh!

gpshead commented Sep 2, 2022

Uh oh!

gpshead commented Sep 4, 2022

Uh oh!

bedevere-bot commented Sep 4, 2022

Uh oh!

ambv commented Sep 5, 2022

Uh oh!

ambv commented Sep 5, 2022

Uh oh!

bedevere-bot commented Sep 5, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

gpshead commented Sep 2, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Sep 2, 2022

Uh oh!

gpshead commented Sep 2, 2022

Uh oh!

gpshead commented Sep 4, 2022

Uh oh!

bedevere-bot commented Sep 4, 2022

Uh oh!

ambv commented Sep 5, 2022

Uh oh!

ambv commented Sep 5, 2022

Uh oh!

bedevere-bot commented Sep 5, 2022

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

gpshead commented Sep 2, 2022 •

edited

Loading