gh-142352: Fix start_tls() to transfer buffered data from StreamReader#142354

kasimov-maxim

Summary

When using StreamWriter.start_tls() to upgrade a connection to TLS mid-stream, any data already buffered in the StreamReader was lost. This commonly occurs when implementing servers that support the PROXY protocol.

Changes

Added _transfer_buffered_data_to_ssl() method to BaseEventLoop that transfers buffered data from StreamReader._buffer to SSLProtocol._incoming before the TLS handshake begins
Added tests for the PROXY protocol + TLS scenario

Generative AI usage

Some parts of this PR were assisted by a generative AI tool. Specifically:

help with drafting English-language comments and documentation
suggestions for test refactoring
assistance with wording and structuring the PR description

All functional logic and final edits were reviewed, verified, and authored by
me, in accordance with the Python devguide guidelines.

Test plan

Added new tests in test_streams.py
All asyncio tests pass
make patchcheck passes

Issue: asyncio: start_tls() loses buffered data from StreamReader #142352

python-cla-bot

All commit authors signed the Contributor License Agreement.

kumaraditya303

@ambv Can you review this? You reviewed the original PR which added this API #91453

picnixz

Has this PR been generated using an LLM? if so, indicate which places were so and read https://devguide.python.org/getting-started/generative-ai/.

bedevere-app

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

picnixz

Also, I think reading and checking if we're compliant with https://datatracker.ietf.org/doc/html/rfc2817 should be done (even if the RFC is only proposed and not in the standard tracks) or whatever RFC should be considered (I do think it's correct to fix this but I don't know if we should only send the possible hello for tls or generally write whatever was buffered)

kasimov-maxim

Has this PR been generated using an LLM? if so, indicate which places were so

Noted in PR.

kasimov-maxim

@picnixz

I think reading and checking if we're compliant with https://datatracker.ietf.org/doc/html/rfc2817 should be done

RFC 2817 isn’t relevant here; it covers HTTP/1.1 Upgrade rather than asyncio.start_tls(). At this point we’ve already decided to switch to TLS, so any buffered bytes belong to the TLS layer and should be passed as‑is to the SSL BIO. If non‑TLS bytes are still buffered, that means start_tls() was called too early in the protocol, which is outside asyncio’s responsibility.

kasimov-maxim

I have made the requested changes; please review again

bedevere-app

Thanks for making the requested changes!

@picnixz: please review the changes made to this pull request.

kasimov-maxim

@fantix

The root trigger of this bug is, as you stated in the bug report, "the TLS ClientHello (arrived) in the same TCP segment (as some application flag did)".

The new regression test (test_start_tls_buffered_data) shows the trigger is broader than “ClientHello in the same TCP segment as some application flag.” The issue is any buffered TLS data that reaches StreamReader before start_tls() is called. That can happen even without proxy‑protocol or segment coalescing.

kumaraditya303

Please resolve the Github comments once they are addressed, otherwise it makes reviewing difficult.

fantix

LGTM!

miss-islington-app

Thanks @kasimov-maxim for the PR, and @kumaraditya303 for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

…a from StreamReader (pythonGH-142354) (cherry picked from commit 0598f4a) Co-authored-by: Maksym Kasimov <39828623+kasimov-maxim@users.noreply.github.com> Co-authored-by: Kumar Aditya <kumaraditya@python.org>

miss-islington-app

Sorry, @kasimov-maxim and @kumaraditya303, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 0598f4a8999b96409e0a2bf9c480afc76a876860 3.13

bedevere-app

GH-145363 is a backport of this pull request to the 3.14 branch.

bedevere-app

GH-145364 is a backport of this pull request to the 3.13 branch.

…red data from StreamReader (pythonGH-142354) (cherry picked from commit 0598f4a) Co-authored-by: Maksym Kasimov <39828623+kasimov-maxim@users.noreply.github.com> Co-authored-by: Kumar Aditya <kumaraditya@python.org>

…a from StreamReader (pythonGH-142354) (cherry picked from commit 0598f4a) Co-authored-by: Maksym Kasimov <39828623+kasimov-maxim@users.noreply.github.com> Co-authored-by: Kumar Aditya <kumaraditya@python.org>

…a from StreamReader (python#142354) Co-authored-by: Kumar Aditya <kumaraditya@python.org>

…ta from StreamReader (GH-142354) (#145363) gh-142352: Fix `asyncio` `start_tls()` to transfer buffered data from StreamReader (GH-142354) (cherry picked from commit 0598f4a) Co-authored-by: Kumar Aditya <kumaraditya@python.org> Co-authored-by: Maksym Kasimov <39828623+kasimov-maxim@users.noreply.github.com>

…rom StreamReader (GH-142354) (#145364) [3.13] gh-142352: Fix `asyncio` `start_tls()` to transfer buffered data from StreamReader (GH-142354) (cherry picked from commit 0598f4a) Co-authored-by: Maksym Kasimov <39828623+kasimov-maxim@users.noreply.github.com>

…a from StreamReader (python#142354) Co-authored-by: Kumar Aditya <kumaraditya@python.org>

kasimov-maxim requested review from 1st1, asvetlov, kumaraditya303 and willingc as code owners December 6, 2025 16:31

bedevere-app Bot mentioned this pull request Dec 6, 2025

asyncio: start_tls() loses buffered data from StreamReader #142352

Closed

bedevere-app Bot added the awaiting review label Dec 6, 2025

kumaraditya303 reviewed Dec 17, 2025

View reviewed changes

kumaraditya303 requested a review from ambv December 17, 2025 16:19

picnixz previously requested changes Jan 6, 2026

View reviewed changes

bedevere-app Bot added awaiting changes and removed awaiting review labels Jan 6, 2026

webknjaz reviewed Jan 16, 2026

View reviewed changes

kasimov-maxim force-pushed the gh-142352-start-tls-buffered-data branch from b2ed3c3 to 86d80bf Compare January 21, 2026 16:08

bedevere-app Bot added awaiting change review and removed awaiting changes labels Jan 21, 2026

bedevere-app Bot requested a review from picnixz January 21, 2026 16:15

kasimov-maxim force-pushed the gh-142352-start-tls-buffered-data branch from 86d80bf to c2cd20b Compare January 21, 2026 16:49

kumaraditya303 added topic-asyncio needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes type-bug An unexpected behavior, bug, or error labels Feb 8, 2026

kumaraditya303 reviewed Feb 8, 2026

View reviewed changes

kumaraditya303 added 2 commits February 28, 2026 22:24

address review

d65b052

fix import

dcc279c

kumaraditya303 requested a review from fantix February 28, 2026 17:05

fantix reviewed Feb 28, 2026

View reviewed changes

kumaraditya303 approved these changes Feb 28, 2026

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting review labels Feb 28, 2026

bedevere-app Bot removed the awaiting merge label Feb 28, 2026

miss-islington-app Bot assigned kumaraditya303 Feb 28, 2026

bedevere-app Bot removed the needs backport to 3.14 label Feb 28, 2026

bedevere-app Bot removed the needs backport to 3.13 bugs and security fixes label Feb 28, 2026

brijkapadia pushed a commit to brijkapadia/cpython that referenced this pull request Feb 28, 2026

pythongh-142352: Fix asyncio start_tls() to transfer buffered dat… …

78a274c

…a from StreamReader (python#142354) Co-authored-by: Kumar Aditya <kumaraditya@python.org>

ljfp pushed a commit to ljfp/cpython that referenced this pull request Apr 25, 2026

pythongh-142352: Fix asyncio start_tls() to transfer buffered dat… …

4332dd0

…a from StreamReader (python#142354) Co-authored-by: Kumar Aditya <kumaraditya@python.org>

Conversation

kasimov-maxim commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Generative AI usage

Test plan

Uh oh!

python-cla-bot Bot commented Dec 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kumaraditya303 commented Dec 17, 2025

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

bedevere-app Bot commented Jan 6, 2026

Uh oh!

picnixz commented Jan 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kasimov-maxim commented Jan 6, 2026

Uh oh!

kasimov-maxim commented Jan 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kasimov-maxim commented Jan 21, 2026

Uh oh!

bedevere-app Bot commented Jan 21, 2026

Uh oh!

kasimov-maxim commented Feb 5, 2026

Uh oh!

kumaraditya303 commented Feb 8, 2026

Uh oh!

fantix left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-app Bot commented Feb 28, 2026

Uh oh!

miss-islington-app Bot commented Feb 28, 2026

Uh oh!

bedevere-app Bot commented Feb 28, 2026

Uh oh!

bedevere-app Bot commented Feb 28, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kasimov-maxim commented Dec 6, 2025 •

edited

Loading

python-cla-bot Bot commented Dec 6, 2025 •

edited

Loading

picnixz commented Jan 6, 2026 •

edited

Loading

kasimov-maxim commented Jan 21, 2026 •

edited

Loading