gh-128110: Fix rfc2047 handling in email parser address headers#130749

medmunds

RFC 2047 Section 6.2 requires that "any 'linear-white-space' that separates a pair of adjacent 'encoded-word's is ignored." The modern header value parser correctly implements that for unstructured headers, but had missed a case in structured headers. This could cause a parsed address header to include extraneous spaces in a display-name.

Fixed in get_atom() by converting a trailing CFWSList token after an encoded-word to an EWWhiteSpaceTerminal if another encoded-word follows.

Deliberately left similar code in get_dotatom() unmodified. A dotatom can only appear within an addr-spec. RFC 2047 Section 5 prohibits use of an encoded-word in any portion of an addr-spec, so its appearance in a dotatom is invalid. Adding (and testing) special white-space handling in an invalid dotatom seems an unnecessary complication.

Fixes gh-128110

Suggest label: topic-email

RFC 2047 Section 6.2 requires that "any 'linear-white-space' that separates a pair of adjacent 'encoded-word's is ignored." The modern header value parser correctly implements that for unstructured headers, but had missed a case in structured headers. This could cause a parsed address header to include extraneous spaces in a display-name. Fixed in get_atom() by converting a trailing CFWSList token after an encoded-word to an EWWhiteSpaceTerminal if another encoded-word follows. Deliberately left similar code in get_dotatom() unmodified. A dotatom can only appear within an addr-spec. RFC 2047 Section 5 prohibits use of an encoded-word in any portion of an addr-spec, so its appearance in a dotatom is invalid. Adding (and testing) special white-space handling in an invalid dotatom seems an unnecessary complication.

github-actions

This PR is stale because it has been open for 30 days with no activity.

bitdancer

The fact that you put the tests in get_phrase points to the fact that it's really get_phrase that is the locus of the bug. That's where the ews can end up next to each other. Here is a fix to get_phrase that passes all your tests. The if is complex, but that's because the circumstances where this situation comes up is very specific.

@@ -1473,6 +1473,16 @@ def get_phrase(value):
         else:
             try:
                 token, value = get_word(value)
+                if (token[0].token_type == 'encoded-word'
+                        and phrase
+                        and phrase[-1].token_type == 'atom'
+                        and len(phrase[-1]) > 1
+                        and phrase[-1][-2].token_type == 'encoded-word'
+                        and phrase[-1][-1].token_type == 'cfws'
+                        and not phrase[-1][-1].comments
+                    ):
+                    # linear ws between ews needs special handing...
+                    phrase[-1][-1] = EWWhiteSpaceTerminal(phrase[-1], 'fws')
             except errors.HeaderParseError:
                 if value[0] in CFWS_LEADER:
                     token, value = get_cfws(value)

This is dependent on the fact that "subsequent" atoms will never have leading whitespace because that's been consumed already. I don't think it's worth adding extra code for the possibility of leading whitespace because the parser won't produce it. It's a bit of parser fragility in the face of code changes, but I think that's a minor concern given the parser design (which is that it consumes whitespace greedily)

bedevere-app

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@bitdancer

Switch to @bitdancer's fix from review feedback. Recharacterize space between ews as fws after parsing in get_phrase (rather than peeking ahead after first ew in get_word). Co-authored-by: R David Murray <rdmurray@bitdance.com>

medmunds

I have made the requested changes; please review again

bedevere-app

Thanks for making the requested changes!

@bitdancer: please review the changes made to this pull request.

miss-islington-app

Thanks @medmunds for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

miss-islington-app

Thanks @medmunds for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

miss-islington-app

Thanks @medmunds for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

bedevere-bot

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot iOS ARM64 Simulator 3.x (tier-3) has failed when building commit 7a4c6df.

What do you need to do:

Don't panic.
Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1380/builds/6615) and take a look at the build logs.
Check if the failure is related to this commit (7a4c6df) or if it is a false positive.
If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1380/builds/6615

Summary of the results of the build (if available):

Click to see traceback logs

remote: Enumerating objects: 13, done.        
remote: Counting objects:   7% (1/13)        
remote: Counting objects:  15% (2/13)        
remote: Counting objects:  23% (3/13)        
remote: Counting objects:  30% (4/13)        
remote: Counting objects:  38% (5/13)        
remote: Counting objects:  46% (6/13)        
remote: Counting objects:  53% (7/13)        
remote: Counting objects:  61% (8/13)        
remote: Counting objects:  69% (9/13)        
remote: Counting objects:  76% (10/13)        
remote: Counting objects:  84% (11/13)        
remote: Counting objects:  92% (12/13)        
remote: Counting objects: 100% (13/13)        
remote: Counting objects: 100% (13/13), done.        
remote: Compressing objects:   7% (1/13)        
remote: Compressing objects:  15% (2/13)        
remote: Compressing objects:  23% (3/13)        
remote: Compressing objects:  30% (4/13)        
remote: Compressing objects:  38% (5/13)        
remote: Compressing objects:  46% (6/13)        
remote: Compressing objects:  53% (7/13)        
remote: Compressing objects:  61% (8/13)        
remote: Compressing objects:  69% (9/13)        
remote: Compressing objects:  76% (10/13)        
remote: Compressing objects:  84% (11/13)        
remote: Compressing objects:  92% (12/13)        
remote: Compressing objects: 100% (13/13)        
remote: Compressing objects: 100% (13/13), done.        
remote: Total 13 (delta 0), reused 5 (delta 0), pack-reused 0 (from 0)        
From https://github.com/python/cpython
 * branch                    main       -> FETCH_HEAD
Note: switching to '7a4c6dfb8839eb05fb87baf70364680e45001dd4'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

HEAD is now at 7a4c6dfb883 gh-128110: Fix rfc2047 whitespace handling in email parser address headers (#130749)
Switched to and reset branch 'main'

configure: WARNING: no system libmpdec found; falling back to pure-Python version for the decimal module
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.

configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.
ld: warning: ignoring duplicate libraries: '-lm'
In file included from ../../Modules/_remote_debugging/module.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/gc_stats.c:7:
In file included from ../../Modules/_remote_debugging/gc_stats.h:14:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/code_objects.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/object_reading.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/frames.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
1 warning generated.
In file included from ../../Modules/_remote_debugging/frame_cache.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
1 warning generated.
In file included from ../../Modules/_remote_debugging/threads.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/asyncio.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/binary_io_reader.c:13:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/binary_io_writer.c:13:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/interpreters.c:7:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/subprocess.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/_remote_debugging/subprocess.c:100:1: warning: unused function 'find_children_bfs' [-Wunused-function]
  100 | find_children_bfs(pid_t target_pid, int recursive,
      | ^~~~~~~~~~~~~~~~~
2 warnings generated.
1 warning generated.
1 warning generated.
1 warning generated.
1 warning generated.
ld: warning: ignoring duplicate libraries: '-lm'
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.
ld: warning: ignoring duplicate libraries: '-lm'
In file included from ../../Modules/_remote_debugging/gc_stats.c:7:
In file included from ../../Modules/_remote_debugging/gc_stats.h:14:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/module.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/object_reading.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/code_objects.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/frames.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/frame_cache.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
1 warning generated.
In file included from ../../Modules/_remote_debugging/threads.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/binary_io_writer.c:13:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/asyncio.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
1 warning generated.
In file included from ../../Modules/_remote_debugging/binary_io_reader.c:13:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/subprocess.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/_remote_debugging/subprocess.c:100:1: warning: unused function 'find_children_bfs' [-Wunused-function]
  100 | find_children_bfs(pid_t target_pid, int recursive,
      | ^~~~~~~~~~~~~~~~~
2 warnings generated.
In file included from ../../Modules/_remote_debugging/interpreters.c:7:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
1 warning generated.
1 warning generated.
1 warning generated.
ld: warning: ignoring duplicate libraries: '-lm'
configure: WARNING: x86_64-apple-ios-simulator/clang is not supported
configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly.
configure: WARNING:

Platform "x86_64-apple-ios-simulator" with compiler "clang" is not supported by the
CPython core team, see https://peps.python.org/pep-0011/ for more information.

/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(brc.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(dynamic_annotations.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(gc_free_threading.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(index_pool.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(jit.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(jit_publish.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(optimizer_analysis.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(optimizer_symbols.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(pystats.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(stackrefs.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(uniqueid.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(perf_jit_trampoline.o) has no symbols
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ranlib: file: libpython3.16.a(jit_unwind.o) has no symbols
ld: warning: ignoring duplicate libraries: '-lm'
In file included from ../../Modules/_remote_debugging/module.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/gc_stats.c:7:
In file included from ../../Modules/_remote_debugging/gc_stats.h:14:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/object_reading.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/frames.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/code_objects.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/frame_cache.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
1 warning generated.
1 warning generated.
In file included from ../../Modules/_remote_debugging/threads.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/binary_io_writer.c:13:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/asyncio.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
In file included from ../../Modules/_remote_debugging/binary_io_reader.c:13:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/subprocess.c:8:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../Modules/_remote_debugging/subprocess.c:100:1: warning: unused function 'find_children_bfs' [-Wunused-function]
  100 | find_children_bfs(pid_t target_pid, int recursive,
      | ^~~~~~~~~~~~~~~~~
In file included from ../../Modules/_remote_debugging/interpreters.c:7:
In file included from ../../Modules/_remote_debugging/_remote_debugging.h:34:
../../Modules/_remote_debugging/../../Python/remote_debug.h:165:1: warning: unused function '_Py_RemoteDebug_ValidatePyRuntimeCookie' [-Wunused-function]
  165 | _Py_RemoteDebug_ValidatePyRuntimeCookie(proc_handle_t *handle, uintptr_t address)
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2 warnings generated.
1 warning generated.
1 warning generated.
1 warning generated.
1 warning generated.
ld: warning: ignoring duplicate libraries: '-lm'

--- xcodebuild: WARNING: Using the first of multiple matching destinations:
{ platform:iOS Simulator, id:4F9AFDE7-A49A-46AD-B161-FCA22E48EA55, OS:18.3.1, name:iPhone SE (3rd generation) }
{ platform:iOS Simulator, id:4F9AFDE7-A49A-46AD-B161-FCA22E48EA55, OS:18.3.1, name:iPhone SE (3rd generation) }

medmunds

iOS ARM64 Simulator 3.x/6615
… Run test suite …
FAIL: test_preauth_data_to_tls_server (test.test_ssl.TestPreHandshakeClose.test_preauth_data_to_tls_server)
FAIL: test_preauth_data_to_tls_server (test.test_ssl.TestPreHandshakeClose.test_preauth_data_to_tls_server)

Seems extremely unlikely to be related to these changes for structured address headers in email._header_value_parser, so I'm guessing "false positive."

bitdancer

Yeah, looks like a either a network problem on the buildbot or an error from some other commit. Not our problem here either way.

miss-islington-app

Thanks @medmunds for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

bedevere-app

GH-149787 is a backport of this pull request to the 3.15 branch.

miss-islington-app

Thanks @medmunds for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

miss-islington-app

Thanks @medmunds for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

bedevere-app

GH-149788 is a backport of this pull request to the 3.14 branch.

bedevere-app

GH-149789 is a backport of this pull request to the 3.13 branch.

@bitdancer

…ress headers (GH-130749) (#149787) RFC 2047 Section 6.2 requires that "any 'linear-white-space' that separates a pair of adjacent 'encoded-word's is ignored." The modern header value parser correctly implements that for unstructured headers, but had missed a case in structured headers. This could cause a parsed address header to include extraneous spaces in a display-name. Switch to @bitdancer's fix from review feedback. Recharacterize space between ews as fws after parsing in get_phrase. RDM: This fix is dependent on the fact that "subsequent" atoms will never have leading whitespace because that's been consumed already. I don't think it's worth adding extra code for the possibility of leading whitespace because the parser won't produce it. It's a bit of parser fragility in the face of code changes, but I think that's a minor concern given the parser design (which is that it consumes whitespace greedily) (cherry picked from commit 7a4c6df) Co-authored-by: Mike Edmunds <medmunds@gmail.com> Co-authored-by: R David Murray <rdmurray@bitdance.com>

@bitdancer

…ress headers (GH-130749) (#149788) RFC 2047 Section 6.2 requires that "any 'linear-white-space' that separates a pair of adjacent 'encoded-word's is ignored." The modern header value parser correctly implements that for unstructured headers, but had missed a case in structured headers. This could cause a parsed address header to include extraneous spaces in a display-name. Switch to @bitdancer's fix from review feedback. Recharacterize space between ews as fws after parsing in get_phrase. RDM: This fix is dependent on the fact that "subsequent" atoms will never have leading whitespace because that's been consumed already. I don't think it's worth adding extra code for the possibility of leading whitespace because the parser won't produce it. It's a bit of parser fragility in the face of code changes, but I think that's a minor concern given the parser design (which is that it consumes whitespace greedily) (cherry picked from commit 7a4c6df) Co-authored-by: Mike Edmunds <medmunds@gmail.com> Co-authored-by: R David Murray <rdmurray@bitdance.com>

@bitdancer

…ress headers (GH-130749) (#149789) RFC 2047 Section 6.2 requires that "any 'linear-white-space' that separates a pair of adjacent 'encoded-word's is ignored." The modern header value parser correctly implements that for unstructured headers, but had missed a case in structured headers. This could cause a parsed address header to include extraneous spaces in a display-name. Switch to @bitdancer's fix from review feedback. Recharacterize space between ews as fws after parsing in get_phrase. RDM: This fix is dependent on the fact that "subsequent" atoms will never have leading whitespace because that's been consumed already. I don't think it's worth adding extra code for the possibility of leading whitespace because the parser won't produce it. It's a bit of parser fragility in the face of code changes, but I think that's a minor concern given the parser design (which is that it consumes whitespace greedily) (cherry picked from commit 7a4c6df) Co-authored-by: Mike Edmunds <medmunds@gmail.com> Co-authored-by: R David Murray <rdmurray@bitdance.com>

medmunds requested a review from a team as a code owner March 1, 2025 22:37

bedevere-app Bot added the awaiting review label Mar 1, 2025

bedevere-app Bot mentioned this pull request Mar 1, 2025

email.parser can insert extraneous spaces when parsing rfc2047 headers with policy.default #128110

Closed

ZeroIntensity added the topic-email label Mar 2, 2025

caje731 approved these changes Mar 12, 2025

View reviewed changes

bedevere-app Bot added awaiting core review and removed awaiting review labels Mar 12, 2025

github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 22, 2026

medmunds mentioned this pull request Apr 30, 2026

gh-81074: Allow non-ASCII addr_spec in email.headerregistry.Address #122477

Merged

bitdancer requested changes May 5, 2026

View reviewed changes

bedevere-app Bot added awaiting changes and removed awaiting core review labels May 5, 2026

medmunds and others added 2 commits May 11, 2026 12:51

Merge branch 'main' into fix-issue-128110

704c03f

Move adjacent ew detection to get_phrase …

5a92b2a

Switch to @bitdancer's fix from review feedback. Recharacterize space between ews as fws after parsing in get_phrase (rather than peeking ahead after first ew in get_word). Co-authored-by: R David Murray <rdmurray@bitdance.com>

bedevere-app Bot added awaiting change review and removed awaiting changes labels May 11, 2026

bedevere-app Bot requested a review from bitdancer May 11, 2026 20:52

Update 2025-03-01-13-36-02.gh-issue-128110.9wx_G0.rst

7c062e4

bitdancer approved these changes May 11, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting change review labels May 11, 2026

bedevere-app Bot removed the awaiting merge label May 11, 2026

bitdancer added needs backport to 3.13 needs backport to 3.14 labels May 11, 2026

bitdancer added the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 11, 2026

bitdancer removed the stale Stale PR or inactive for long period of time. label May 11, 2026

bitdancer added needs backport to 3.15 pre-release feature fixes, bugs and security fixes and removed needs backport to 3.15 pre-release feature fixes, bugs and security fixes labels May 13, 2026

bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label May 13, 2026

bitdancer added needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes and removed needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes labels May 13, 2026

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label May 13, 2026

bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label May 13, 2026

Conversation

medmunds commented Mar 1, 2025

Uh oh!

github-actions Bot commented Apr 22, 2026

Uh oh!

bitdancer left a comment

Choose a reason for hiding this comment

Uh oh!

bedevere-app Bot commented May 5, 2026

Uh oh!

medmunds commented May 11, 2026

Uh oh!

bedevere-app Bot commented May 11, 2026

Uh oh!

Uh oh!

miss-islington-app Bot commented May 11, 2026

Uh oh!

miss-islington-app Bot commented May 11, 2026

Uh oh!

miss-islington-app Bot commented May 11, 2026

Uh oh!

bedevere-bot commented May 11, 2026

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Uh oh!

medmunds commented May 12, 2026

Uh oh!

bitdancer commented May 12, 2026

Uh oh!

miss-islington-app Bot commented May 13, 2026

Uh oh!

bedevere-app Bot commented May 13, 2026

Uh oh!

miss-islington-app Bot commented May 13, 2026

Uh oh!

miss-islington-app Bot commented May 13, 2026

Uh oh!

bedevere-app Bot commented May 13, 2026

Uh oh!

bedevere-app Bot commented May 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants