Issue 34975: start_tls() difficult when using asyncio.start_server()

There does not seem to be a public API for replacing the transport of the StreamReader / StreamWriter provided to the callback of a call to asyncio.start_server().

The only way I have found to use the new SSL transport is to update protected members of the StreamReaderProtocol object, e.g.:

    async def callback(reader, writer):
        loop = asyncio.get_event_loop()
        transport = writer.transport
        protocol = transport.get_protocol()
        new_transport = await loop.start_tls(
            transport, protocol, ssl_context, server_side=True)
        protocol._stream_reader = StreamReader(loop=loop)
        protocol._client_connected_cb = do_stuff_after_start_tls
        protocol.connection_made(new_transport)

    async def do_stuff_after_start_tls(ssl_reader, ssl_writer): ...

Thanks for raising the problem.
I'm in the middle of streams API refactoring.

https://github.com/asvetlov/cpython/blob/async-streams/Lib/asyncio/streams.py#L801-L812 is a draft.

A help is very appreciated.
Would you pick up this snippet and make a pull request with tests and documentation update?

One thing: I'm -1 on adding starttls to current stream api; let's add it only to the new one (same for sendfile)

I added start_tls() to StreamWriter. My implementation returns a new StreamWriter that should be used from then on, but it could be adapted to modify the current writer in-place (let me know).

I've added docs, an integration test, and done some additional "real-world" testing with an IMAP server I work on. Specifically, "openssl s_client -connect xxx -starttls imap" works like a charm, and no errors/warnings are logged on disconnect.

I believe stream transport should be replaced.
If you have time please do this change.

Also please take a look at #36889 for merging StreamWriter and StreamReader.
It can simplify the replacement strategy.

Anyway, please keep your PR open at least.
The plan is: either merge your PR before writer/reader merging or land unified streams first and adopt the PR to new API.

Another thing that should be done before 3.8 feature freeze is support for sendfile in streams API. Hopefully, it is much simpler: no need for transport changing etc.

Fixed by #36889

#36889 was reverted, so this is not resolved.

I'm guessing this needs to be moved to 3.9 now too. Is my original PR worth revisiting? https://github.com/python/cpython/pull/13143/files

I think it should be closed; Yuri thinks that current streams API is frozen for the sake of shiny brand new streams (don't exist yet).

https://discuss.python.org/t/need-reconsideration-of-bpo-34975-add-start-tls-method-to-streams-api/14720 would like to see this reconsidered.  reopening.

versions: + Python 3.11, - Python 3.8
nosy: + gregory.p.smith

messages: + msg416953
resolution: wont fix ->
stage: resolved ->

resolution: fixed
stage: patch review -> resolved