asyncio: start_tls() loses buffered data from StreamReader
Bug report
Bug description:
Bug description
When using StreamWriter.start_tls() (or loop.start_tls()) to upgrade a connection to TLS mid-stream, any data already buffered in the StreamReader is lost.
This commonly occurs when implementing servers that support the PROXY protocol (e.g., behind HAProxy with send-proxy). In this scenario:
- The proxy sends the PROXY protocol header followed immediately by the TLS ClientHello in the same TCP segment
- The server reads the PROXY header using
reader.readline() - The TLS ClientHello data is now buffered in
StreamReader._buffer - When
writer.start_tls()is called, a newSSLProtocolis created, but the buffered data is not transferred to it - The TLS handshake hangs waiting for ClientHello data that was already received but lost
Reproducible example
Server (server.py)
import asyncio import ssl async def handle_client(reader, writer): # Read PROXY protocol header proxy_header = await reader.readline() print(f"PROXY header: {proxy_header}") # Upgrade to TLS - this is where data loss occurs context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH) context.load_cert_chain('server.crt', 'server.key') await writer.start_tls(context) print("TLS handshake completed") # Never reached - hangs here data = await reader.read(1024) print(f"Received: {data}") writer.close() await writer.wait_closed() async def main(): server = await asyncio.start_server(handle_client, '127.0.0.1', 8000) async with server: await server.serve_forever() asyncio.run(main())
HAProxy config (haproxy.cfg)
frontend proxy_frontend
mode tcp
bind *:5002
default_backend proxy_backend
backend proxy_backend
mode tcp
server app 127.0.0.1:8000 send-proxy
Client command
echo "Test message" | openssl s_client -connect 127.0.0.1:5002
Expected behavior
TLS handshake completes and server receives "Test message".
Actual behavior
TLS handshake hangs indefinitely because the TLS ClientHello data buffered in StreamReader is lost when start_tls() is called.
Additional context
I have a fix ready with tests and will submit a PR shortly.
The fix transfers buffered data from StreamReader._buffer to SSLProtocol._incoming before the TLS handshake begins.
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux