The following code sends a request every 200ms and should handle the responses asynchronously whenever they come.
Over HTTP it works as expected - a request is sent every 200ms and the response callback is called independently whenever a response arrives. Over HTTPS however, the requests get significantly delayed whenever a response arrives (even though my response handler does no work). The response callback seems to be called twice for each request, once with a zero-length response (edit: this is because of a redirect and seems unrelated to the blocking issue, thanks Padraic).
What could be causing this blocking behaviour over HTTPS? (www.bbc.co.uk is just an example that's geographically far from me, but it happens with all servers I've tested).
grequests_test.py
import time import sys import grequests import gevent def cb(res, **kwargs): print("**** Response", time.time(), len(res.text)) for i in range(10): unsent = grequests.get(sys.argv[1], hooks={'response': cb}) print("Request", time.time()) grequests.send(unsent, grequests.Pool(1)) gevent.sleep(0.2) gevent.sleep(5) $ ipython2 grequests_test.py 'http://www.bbc.co.uk' (expected result)
('Request', 1459050191.499266) ('Request', 1459050191.701466) ('Request', 1459050191.903223) ('Request', 1459050192.10403) ('Request', 1459050192.305626) ('**** Response', 1459050192.099185, 179643) ('Request', 1459050192.506476) ('**** Response', 1459050192.307869, 179643) ('Request', 1459050192.707745) ('**** Response', 1459050192.484711, 179643) ('Request', 1459050192.909376) ('**** Response', 1459050192.696583, 179643) ('Request', 1459050193.110528) ('**** Response', 1459050192.870476, 179643) ('Request', 1459050193.311601) ('**** Response', 1459050193.071679, 179639) ('**** Response', 1459050193.313615, 179680) ('**** Response', 1459050193.4959, 179643) ('**** Response', 1459050193.687054, 179680) ('**** Response', 1459050193.902827, 179639) ipython2 grequests_test.py 'https://www.bbc.co.uk' (requests are sent late)
('Request', 1459050203.24336) ('Request', 1459050203.44473) ('**** Response', 1459050204.423302, 0) ('Request', 1459050204.424748) <------------- THIS REQUEST TIME IS LATE ('**** Response', 1459050205.294426, 0) ('Request', 1459050205.296722) ('Request', 1459050205.497924) ('**** Response', 1459050206.456572, 0) ('Request', 1459050206.457875) ('**** Response', 1459050207.363188, 0) ('**** Response', 1459050208.247189, 0) ('Request', 1459050208.249579) ('**** Response', 1459050208.250645, 179643) ('**** Response', 1459050208.253638, 179643) ('Request', 1459050208.451083) ('**** Response', 1459050209.426556, 0) ('Request', 1459050209.428032) ('**** Response', 1459050209.428929, 179643) ('**** Response', 1459050210.331425, 0) ('**** Response', 1459050211.247793, 0) ('Request', 1459050211.251574) ('**** Response', 1459050211.252321, 179643) ('**** Response', 1459050211.25519, 179680) ('**** Response', 1459050212.397186, 0) ('**** Response', 1459050213.299109, 0) ('**** Response', 1459050213.588854, 179588) ('**** Response', 1459050213.590434, 179643) ('**** Response', 1459050213.593731, 179643) ('**** Response', 1459050213.90507, 179643) ('**** Response', 1459050213.909386, 179643) Notice that the first response appears to arrive long after the next request should have been sent but wasn't. Why didn't the sleep return, and the next request get sent, before that first response arrived?
