◐ Shell
clean mode source ↗

urlopen() hangs and eats 100% CPU on some https urls (git bisect found problematic commit)

Summary

RustPython 0.5.0 (and also current main) hangs and starts eating 100% CPU (one core) on first or second iteration of following script:

import urllib.request
for i in range(3):
    print(i)
    req=urllib.request.Request('https://crates.io/api/v1/crates/tokio')
    with urllib.request.urlopen(req) as r:
        r.read()

Interesting thing is, reproducibility depends on what exactly HTTPs server sends to the client. Just HTTPs plus relatively large file is not enough. For example, bug completely disappears when I replace https://crates.io/api/v1/crates/tokio with https://gist.githubusercontent.com/khaykov/a6105154becce4c0530da38e723c2330/raw/41ab415ac41c93a198f7da5b47d604956157c5c3/gistfile1.txt.

I was able to git bisect this to a single problematic commit - 073adbd (#7418). Reverting it on top of 0.5.0 fixes this issue for me. And I have no idea about how to fix this properly.

Expected

RustPython should terminate after three iterations.

Actual

RustPython hangs and starts eating 100% CPU

Python Documentation

N/A