Issue 41944: [security][CVE-2020-27619] Python testsuite calls eval() on content received via HTTP

As was reported by Florian Bruhin, Python testsuite calls eval() on content received via HTTP (in Lib/test/multibytecodec_support.py).

I wonder if I should request a CVE for this as well? Just to make sure the word gets out to distributions/organizations/etc. running the Python testsuite, given that we can't be sure it which contexts this happens (and as it could be exploited by e.g. spoofing a WiFi network or so).

I don't think that a CVE is justified.

I don't know anyone running the Python test suite on production. Only developers of Python itself run Python.

Oops: Only developers of Python itself run the Python test suite.

That assumption is false. For starters, distribution packagers do:

https://github.com/archlinux/svntogit-packages/blob/3fc85177e35d1ff9ab000950c5d1af9567730434/trunk/PKGBUILD#L72-L84

https://src.fedoraproject.org/rpms/python3.9/blob/master/f/python3.9.spec#_1168

When I build a Python from source (via an Arch User Repository package), I do as well, and so does anyone installing those packages by default.

Anyone building with --enable-optimizations (PGO) will likely do so as well, though I'm not sure if that runs this part of the testsuite.

I'm not saying that this issue is not a vulnerability, just that the scope is limited.

By default, downloaded from the Internet are disabled. You have to opt-in for that using -u network (or -u all which enables the network resource) command line option of "./python -m test".

Impacted:

* "make testall", "make testuniversal" and "make buildbottest" commands are impacted (pass -u all to the test suite).

* Python buildbot workers are impacted: they run the "make buildbottest" command.

* Travis CI is impacted: it runs "./python -m test -uall,-cpu (...)".

* Multiple GitHub Action jobs are impacted (coverage, Windows, macOS, Ubuntu): run "-uall,-cpu".

* Azure Pipelines jobs are impacted: use -uall,-cpu.


> https://src.fedoraproject.org/rpms/python3.9/blob/master/f/python3.9.spec#_1168

Fedora packages are not impacted: no -u option is passed to the test suite.


> Anyone building with --enable-optimizations (PGO) will likely do so as well, though I'm not sure if that runs this part of the testsuite.

PGO build is not impacted, it uses "./python -m test --pgo" (download is disabled). Moreover, multibyte codec checks are not run by this command (see Lib/test/libregrtest/pgo.py, only test_codecs of codec tests is run).

Thanks for the clarification - I wasn't aware those tests aren't run by default.

FWIW I found another place where a similar thing is done, though by chance it's probably not exploitable - see GH-22575.

> FWIW I found another place where a similar thing is done, though by chance it's probably not exploitable - see GH-22575.

I agree that test_ucn is not exploitable, but it would be nice to harden it anyway.

Extract of the code:

            self.assertEqual(unicodedata.lookup(seqname), codepoints)
            with self.assertRaises(SyntaxError):
                self.checkletter(seqname, None)

test_ucn downloads http://www.pythontest.net/unicode/13.0.0/NamedSequences.txt and calls checkletter() on each line, but first it ensures that unicodedata.lookup(seqname) works as expected.

I don't see how it would be possible to inject arbitrary Python code in the 'seqname' variable without making unicodedata.lookup() to fail.

I'm now tracking this vulnerability at:
https://python-security.readthedocs.io/vuln/cjk-codec-download-eval.html

New changeset 2ef5caa58febc8968e670e39e3d37cf8eef3cab8 by Serhiy Storchaka in branch 'master':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566)
https://github.com/python/cpython/commit/2ef5caa58febc8968e670e39e3d37cf8eef3cab8

New changeset b664a1df4ee71d3760ab937653b10997081b1794 by Miss Skeleton (bot) in branch '3.9':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566)
https://github.com/python/cpython/commit/b664a1df4ee71d3760ab937653b10997081b1794

New changeset 6c6c256df3636ff6f6136820afaefa5a10a3ac33 by Miss Skeleton (bot) in branch '3.8':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) (GH-22577)
https://github.com/python/cpython/commit/6c6c256df3636ff6f6136820afaefa5a10a3ac33

Since it's a security vulnerability, I created backports to 3.6 and 3.7 as well.

New changeset a8bf44d04915f7366d9f8dfbf84822ac37a4bab3 by Florian Bruhin in branch 'master':
bpo-41944: No longer call eval() on content received via HTTP in the UnicodeNames tests (GH-22575)
https://github.com/python/cpython/commit/a8bf44d04915f7366d9f8dfbf84822ac37a4bab3

New changeset 43e523103886af66d6c27cd72431b5d9d14cd2a9 by Miss Skeleton (bot) in branch '3.7':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) (GH-22578)
https://github.com/python/cpython/commit/43e523103886af66d6c27cd72431b5d9d14cd2a9

New changeset e912e945f2960029d039d3390ea08835ad39374b by Miss Skeleton (bot) in branch '3.6':
bpo-41944: No longer call eval() on content received via HTTP in the CJK codec tests (GH-22566) (GH-22579)
https://github.com/python/cpython/commit/e912e945f2960029d039d3390ea08835ad39374b

Thanks for the fix Serhiy and thanks Florian Bruhin for the bug report!

The CVE-2020-27619 has been assigned to this issue.

Red Hat advisory: https://access.redhat.com/security/cve/CVE-2020-27619