0

I have a server serving HTML with nginx, uwsgi and flask. It's a simple webapp. When you refresh a page the webapp makes some API calls elsewhere and then treats the results returned from the APIs and generate the HTML which is then returned.

Now, most of the time is spend waiting for the APIs to respond. Because of this, I'm using gevent to make the API calls and monkey patching etc etc to make it all concurrent. If for every page refresh you need to make 3 API calls, you don't need to wait for the first call to return to initiate the second call. You initiate all three of them, wait for their responses, and then generate the HTML to be returned.

However, a separate question is whether or not the page refreshes from beginning to end are themselves concurrent. To test this I wrote a small concurrent script to make refreshes to my server:

from gevent import monkey monkey.patch_all() import requests, datetime import gevent def call_func(n): before = datetime.datetime.now() print 'initiating call '+str(n),before requests.get('http://examplemyserver.com') after = datetime.datetime.now() print 'finishing call '+str(n),after, after-before gevent_list = [ gevent.spawn(call_func, n=n) for n in range(10) ] gevent.joinall(gevent_list)

Here are the results:

initiating call 0 2016-05-04 23:03:49.437540 initiating call 1 2016-05-04 23:03:49.446304 initiating call 2 2016-05-04 23:03:49.447911 initiating call 3 2016-05-04 23:03:49.450232 initiating call 4 2016-05-04 23:03:49.451774 initiating call 5 2016-05-04 23:03:49.453331 initiating call 6 2016-05-04 23:03:49.454759 initiating call 7 2016-05-04 23:03:49.456301 initiating call 8 2016-05-04 23:03:49.457663 initiating call 9 2016-05-04 23:03:49.458981 finishing call 0 2016-05-04 23:03:51.270078 0:00:01.832538 finishing call 6 2016-05-04 23:03:52.169842 0:00:02.715083 finishing call 3 2016-05-04 23:03:52.907892 0:00:03.457660 finishing call 1 2016-05-04 23:03:53.498008 0:00:04.051704 finishing call 8 2016-05-04 23:03:54.150188 0:00:04.692525 finishing call 4 2016-05-04 23:03:55.032089 0:00:05.580315 finishing call 7 2016-05-04 23:03:55.994570 0:00:06.538269 finishing call 2 2016-05-04 23:03:56.659146 0:00:07.211235 finishing call 9 2016-05-04 23:03:57.149156 0:00:07.690175 finishing call 5 2016-05-04 23:03:58.002210 0:00:08.548879

So you see, ten page refreshes are initiated virtually at the same time, however they are returned sequentially with a difference of about 1 second. This suggests that while internally the external API calls are done concurrently, until an HTTP response is given, no new HTTP responses are being accepted.

So, given that most of the webapp time is spent waiting for API responses, how can I make the whole process concurrent? Is it something that I need to change with nginx? With uwsgi? With flask? I have no idea about the architecture of these things.

Thanks.

EDIT 1: Following Rob's idea, I replaced the server to be called with 'http://httpbin.org/delay/1'. This is the result I obtain:

initiating call 0 2016-05-04 23:36:18.697813 initiating call 1 2016-05-04 23:36:18.706510 initiating call 2 2016-05-04 23:36:18.708645 initiating call 3 2016-05-04 23:36:18.711055 initiating call 4 2016-05-04 23:36:18.712679 initiating call 5 2016-05-04 23:36:18.714051 initiating call 6 2016-05-04 23:36:18.715471 initiating call 7 2016-05-04 23:36:18.717015 initiating call 8 2016-05-04 23:36:18.718412 initiating call 9 2016-05-04 23:36:18.720193 finishing call 0 2016-05-04 23:36:20.599483 0:00:01.901670 finishing call 2 2016-05-04 23:36:20.600829 0:00:01.892184 finishing call 8 2016-05-04 23:36:20.611292 0:00:01.892880 finishing call 5 2016-05-04 23:36:20.618842 0:00:01.904791 finishing call 7 2016-05-04 23:36:20.620065 0:00:01.903050 finishing call 6 2016-05-04 23:36:20.621344 0:00:01.905873 finishing call 1 2016-05-04 23:36:20.622407 0:00:01.915897 finishing call 4 2016-05-04 23:36:20.623392 0:00:01.910713 finishing call 9 2016-05-04 23:36:20.624236 0:00:01.904043 finishing call 3 2016-05-04 23:36:20.625075 0:00:01.914020

This is what you'd expect from a server that can take HTTP calls concurrently. So this shows that the above code that I used for the test is correct, and that the problem is indeed on the server. It's not taking HTTP calls concurrently.

EDIT2: I was playing around and checked that the simplest possible flask app has the same behavior as my server, so I'm giving that as my "minimal example"

Consider the following server:

import time from flask import Flask, jsonify app = Flask(__name__) @app.route("/") def hello(): time.sleep(2) return '' if __name__ == "__main__": app.run(host='0.0.0.0', port=80, debug=True)

Now connect to it by taking the previous code and pointing it to http://localhost. The result is the following:

initiating call 0 2016-05-04 23:49:12.629250 initiating call 1 2016-05-04 23:49:12.638554 initiating call 2 2016-05-04 23:49:12.643844 initiating call 3 2016-05-04 23:49:12.645630 initiating call 4 2016-05-04 23:49:12.647866 initiating call 5 2016-05-04 23:49:12.649332 initiating call 6 2016-05-04 23:49:12.650853 initiating call 7 2016-05-04 23:49:12.652448 initiating call 8 2016-05-04 23:49:12.653865 initiating call 9 2016-05-04 23:49:12.655348 finishing call 0 2016-05-04 23:49:14.671281 0:00:02.042031 finishing call 1 2016-05-04 23:49:16.673649 0:00:04.035095 finishing call 2 2016-05-04 23:49:18.676576 0:00:06.032732 finishing call 3 2016-05-04 23:49:20.679746 0:00:08.034116 finishing call 4 2016-05-04 23:49:22.681615 0:00:10.033749 finishing call 5 2016-05-04 23:49:24.683817 0:00:12.034485 finishing call 6 2016-05-04 23:49:26.686309 0:00:14.035456 finishing call 7 2016-05-04 23:49:28.688079 0:00:16.035631 finishing call 8 2016-05-04 23:49:30.691382 0:00:18.037517 finishing call 9 2016-05-04 23:49:32.696471 0:00:20.041123

So basically a simple flask webapp does not take HTTP calls concurrently. How can I change this?

Improve this question
13
  • It is also possible that examplemyserver.com is single-threaded. It "examplemyserver.com" a production server or something you whipped up for this experiment? Commented May 4, 2016 at 22:10
  • Possible dup, certainly related: stackoverflow.com/questions/9501663/… Commented May 4, 2016 at 22:15
  • With Python2.7.6, I cannot reproduce your results. I replaced the URL with http://httpbin.org/delay/1, but otherwise my code is identical to yours. For me, all of the requests lines occur more-or-less simultaneously, there is a 1-sec delay, and all of the response lines occur more-or-less simultaneously. Commented May 4, 2016 at 22:23
  • @Robᵩ I replaced the url of my server for privacy. That's not the real url. Commented May 4, 2016 at 22:32
  • @Robᵩ Also, it's not related to the question you linked. In that question, the OP is doing something wrong in the calls. In my case each of the calls has the behavior expected, but I think there's something wrong in the server. Commented May 4, 2016 at 22:35

1 Answer 1

Reset to default
0

Short answer: use --processes=N and/or --threads=M when you start uwsgi.

Long answer: The stack you've described involved three components: nginx in the role of a reverse-proxy web server; uwsgi in the role of an application server; and flask as an application framework. The data flow occurs in that order, with the response flowing in the oppose order.

nginx is inherently multi-tasking and flask is inherently single-tasking. uwsgi can be configured to be either. By default, uwsgi is single-tasking.

Here is the setup I used to investigate this. My OS is Ubuntu 14.04.

/etc/nginx/sites-available/myapp:

server { listen 8100; location / { include uwsgi_params; uwsgi_pass unix:/tmp/uwsgi.sock; } }

uwsgi command line:

One of: uwsgi -s /tmp/uwsgi.sock -w app:app uwsgi -s /tmp/uwsgi.sock -w app:app --processes=4 uwsgi -s /tmp/uwsgi.sock -w app:app --threads=4 Followed by: 
 chmod 777 /tmp/uwsgi.sock

app.py:

import time from flask import Flask, render_template_string app = Flask(__name__) @app.route('/') @app.route('/<name>') def hello(name='World'): time.sleep(1) return render_template_string(''' <html><body><p> Hello, <b>{{ name }}</b>! </p></body></html>''', name=name) if __name__=="__main__": app.run(debug=True)

Client: I used your gevent/requests client from above, substituting the following url: http://localhost:8100/World.

Typical result:

initiating call 0 2016-05-04 21:08:16.473978 initiating call 1 2016-05-04 21:08:16.485589 initiating call 2 2016-05-04 21:08:16.487258 initiating call 3 2016-05-04 21:08:16.490140 initiating call 4 2016-05-04 21:08:16.493158 initiating call 5 2016-05-04 21:08:16.494417 initiating call 6 2016-05-04 21:08:16.495620 initiating call 7 2016-05-04 21:08:16.497180 initiating call 8 2016-05-04 21:08:16.498413 initiating call 9 2016-05-04 21:08:16.499528 finishing call 3 2016-05-04 21:08:17.512949 0:00:01.022809 finishing call 6 2016-05-04 21:08:17.515944 0:00:01.020324 finishing call 7 2016-05-04 21:08:17.517655 0:00:01.020475 finishing call 4 2016-05-04 21:08:17.519555 0:00:01.026397 finishing call 2 2016-05-04 21:08:18.520249 0:00:02.032991 finishing call 0 2016-05-04 21:08:18.522902 0:00:02.048924 finishing call 5 2016-05-04 21:08:18.524902 0:00:02.030485 finishing call 1 2016-05-04 21:08:18.526682 0:00:02.041093 finishing call 9 2016-05-04 21:08:19.526909 0:00:03.027381 finishing call 8 2016-05-04 21:08:19.528926 0:00:03.030513

References:

Improve this answer
Sign up to request clarification or add additional context in comments.

2 Comments

I think you nailed it with your distinction between the several components. Thank you very much.
Hi, I just tested this and it totally works. I didn't use your config though. Since I already had my own config in place, with Emperor. The only thing I needed to do was in the uwsgi .ini file add the line processes = 4, it's that simple. Like you said the optimal number depends on many things. In my case since 99% of the time is spent waiting for external APIs, I suspect that my optimal number of processes will be quite a big number. Thanks again.