Issue 8238: Proxy handling very slow

After I installed Python 2.6.5, I noticed a drop in performance of web connections via proxy.

This script:
---------------------------------------------------------
import time
import urllib2

timeMark = time.time()
opener = urllib2.build_opener()
textWeb = opener.open("http://www.google.com/").read()
print time.time() - timeMark
---------------------------------------------------------

takes about 60 seconds to complete (consistently)

The same script, run using Python 2.6.2 is completed in less than 1 second  (0.2seconds)

The test system:
- Windows XP SP3
- internet connection via corporate connection, using a proxy (the proxy is set in Control Panel - Internet Options etc)

How are setting up and using your proxy? It would also help you discuss your approach at python-help@python.org and if you see any specific issues, raise it in the bug report.

I attached a screenshot.

The proxy is on the corporate network. I'm not setting it up, I'm only using it. Internet Explorer needs that setting in order to connect. Firefox needs this proxy set as "manual proxy configuration".

I looked deeper into the situatioan, usign the following (modified) script:

---------------------------------------------------
import time
import urllib2

timeMark = time.time()
opener = urllib2.build_opener()

proxy = urllib2.ProxyHandler({"http" : "http://10.249.1.63:80"})
opener.add_handler(proxy)

textWeb = opener.open("http://www.google.com/").read()

print time.time() - timeMark

---------------------------------------------------

The only difference from the script above: I set up the proxy manually.
I tested the new script in 2.6.5

With Windows proxy active: same problem, long waiting time.
With Windows proxy disabled: fast execution.

I can't even try to reproduce this as I've no corporate network as a test bed.  Is this still an issue with Python 2.7 or the reworked urllib in Python 3.x?

If this is still a problem, you should narrow down what is causing the slowdown. If you interrupt it, what is the stack trace?

My best guess is perhaps there is a bypass hostname setting and a slow or failing DNS lookup. A call to urllib.proxy_bypass() was added in 2.6.4 for Issue 6894, and in 2.6.5 the Windows registry implementation called gethostname() and gethostbyname(). Perhaps this is fixed by Issue 1648102.

superseder: proxy_bypass in urllib handling of <local> macro
components: + Windows, - None
title: Proxy handling -> Proxy handling very slow
nosy: + martin.panter, paul.moore, tim.golden, steve.dower, zach.ware

messages: + msg270673

messages: + msg101745