3

The function urllib2.urlopen freezes. So my question is simple:

  • Why does urlopen freeze my script for ever even though timeout is set?
  • How can I access data at an URL (in this case: http://api.own3d.tv/live?channel=FnaticTV) without the possibility of my Python process freezing up for all eternity?

This is the part where is freezes (In own3d.py):

# Try three times to make contact
while True:
    try:
        # Connect to API 

        # Right here! It freezes here
        connection = urllib2.urlopen(request, timeout=10)

        xmlstring = connection.read() 
    except URLError as e:
        tries += 1
        if tries >= 3:
            sys.stderr.write(
                      'own3dStreamsUpdater: Fatal error: Repeated timeouts')
            exit()

This is the stack trace after my KeyboardInterrupt

Traceback (most recent call last):
  File "", line 1, in 
  File "honsapp/own3dStreamsUpdater.py", line 53, in updateStreamInfo
    streamInfo = getStreamInfo(stream)
  File "honsapp/own3d.py", line 98, in getStreamInfo
    connection = urllib2.urlopen(request, timeout=10)
  File "/usr/local/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/local/lib/python2.7/urllib2.py", line 394, in open
    response = self._open(req, data)
  File "/usr/local/lib/python2.7/urllib2.py", line 412, in _open
    '_open', req)
  File "/usr/local/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python2.7/urllib2.py", line 1199, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/local/lib/python2.7/urllib2.py", line 1170, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/local/lib/python2.7/httplib.py", line 1027, in getresponse
    response.begin()
  File "/usr/local/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python2.7/httplib.py", line 365, in _read_status
    line = self.fp.readline()
  File "/usr/local/lib/python2.7/socket.py", line 447, in readline
    data = self._sock.recv(self._rbufsize)
KeyboardInterrupt

Edit

I let my script run over night. I don't know exactly how long it took (Though more than five minutes) but the script finally gave up and gave me a stacktrace:

Traceback (most recent call last):
  File "honsapp/own3dStreamsUpdater.py", line 260, in 
    newInfo()
  File "honsapp/own3dStreamsUpdater.py", line 172, in newInfo
    result = updateStreamInfo(stream)
  File "honsapp/own3dStreamsUpdater.py", line 53, in updateStreamInfo
    streamInfo = getStreamInfo(stream)
  File "/root/Dropbox/Projects/honstreams/honsapp/own3d.py", line 98, in getStreamInfo
    connection = urllib2.urlopen(request, timeout=10)
  File "/usr/local/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/local/lib/python2.7/urllib2.py", line 394, in open
    response = self._open(req, data)
  File "/usr/local/lib/python2.7/urllib2.py", line 412, in _open
    '_open', req)
  File "/usr/local/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/usr/local/lib/python2.7/urllib2.py", line 1199, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/local/lib/python2.7/urllib2.py", line 1170, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/local/lib/python2.7/httplib.py", line 1027, in getresponse
    response.begin()
  File "/usr/local/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/local/lib/python2.7/httplib.py", line 371, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''
Community
  • 1
  • 1
Hubro
  • 56,214
  • 69
  • 228
  • 381

2 Answers2

5

This script doesn't freeze at all, but it's a brilliant example of why you shouldn't code while tired. The loop that's supposed to try to connect to the API three times will continue for ever, because I forgot to put a break in there.

This question was so stupid I wouldn't blame you for removing it.

Fixed code:

# Try three times to make contact
while True:
    try:
        # Connect to API 
        connection = urllib2.urlopen(request, timeout=10)
        xmlstring = connection.read()
        break
    except URLError as e:
        tries += 1
        if tries >= 3:
            sys.stderr.write(
                      'own3dStreamsUpdater: Fatal error: Repeated timeouts')
            exit()
Hubro
  • 56,214
  • 69
  • 228
  • 381
4

Are you sure that the urlopen() call hangs? Because your while loop does not terminate if the call is successful.

Cito
  • 5,365
  • 28
  • 30
  • Indeed. Maybe if you print xmlstring you may see worthwhile output. – Andrew Sledge Nov 09 '11 at 20:46
  • I don't understand what you guys mean. The stacktrace clearly states that the point where it freezes, waiting for my KeyboardInterrupt, is in the line `connection = urllib2.urlopen(request, timeout=10)`. How can changing any code below that line possibly make any difference? Also, I updated my question with additional information. – Hubro Nov 10 '11 at 07:01
  • 1
    The stacktrace only states that the point where you interrupted it was in the urlopen() call - which is not surprising given that there are only two statements in the loop and urlopen() is likely the slower one. Did you put a print statement inside the loop as Andrew suggested? – Cito Nov 10 '11 at 09:45