- Notifications
You must be signed in to change notification settings - Fork 179
Description
I've a python application that can also serve files.
Currently I'm returning an open file stream to waitress when serving files.
I've noticed large performance degradation since version 1.3 in this case.
This is especially true for windows, but also on linux, to a lesser extent, they are present.
I think that the problem was introduced by this change #246, since setting --send-bytes=18000 reduces the problem (at least on windows).
Below is the example file, let's call it example.py:
def app(env, start_response): body = open("demo-file.txt", 'rb') start_response("200", {}) return body if __name__ == "__main__": data = "The quick brown fox jumps over the lazy dog\n" with open("demo-file.txt", "w") as f: data *= 25000 f.write(data) # ~ 1mbRunning it with python example.py creates a simple example file. It creates a file of about 1MB
I serve it with waitress using waitress-serve example:app
In the example above if I change return body to return [body.read()] there is no change in performance using send-bytes has no effect. On windows >= 1.3 and <1.3 have the same performance, while on linux the older version seems to perform better.
I'm testing with the curl command (replace -o nul with -o /dev/null on linux):
curl -o nul -w "time_connect: %{time_connect}s\ntime_pretransfer: %{time_pretransfer}s\ntime_starttransfer: %{time_starttransfer}s\ntime_total: %{time_total}s\n" http://localhost:8080
I get the following results:
windows
waitress >= 1.3
time_connect: 0,234000s time_pretransfer: 0,234000s time_starttransfer: 0,266000s time_total: 4,656000s waitress < 1.3
time_connect: 0,250000s time_pretransfer: 0,266000s time_starttransfer: 0,406000s time_total: 0,406000s waitress >= 1.3 setting --send-bytes=18000
time_connect: 0,250000s time_pretransfer: 0,265000s time_starttransfer: 0,265000s time_total: 0,656000s waitress using return [body.read()]
time_connect: 0,234000s time_pretransfer: 0,234000s time_starttransfer: 0,265000s time_total: 0,265000s linux
waitress >= 1.3
time_connect: 0.004488s time_pretransfer: 0.004507s time_starttransfer: 0.014586s time_total: 0.519794s waitress < 1.3
time_connect: 0.005525s time_pretransfer: 0.005788s time_starttransfer: 0.126952s time_total: 0.132460s waitress >= 1.3 setting --send-bytes=18000
time_connect: 0.005574s time_pretransfer: 0.006167s time_starttransfer: 0.012211s time_total: 0.508851s waitress >= 1.3 using return [body.read()]
time_connect: 0.005984s time_pretransfer: 0.006004s time_starttransfer: 0.019397s time_total: 0.019927s waitress < 1.3 using return [body.read()]
time_connect: 0.004953s time_pretransfer: 0.005193s time_starttransfer: 0.010879s time_total: 0.011450s Just a footnote, I always assumed that passing a stream would be faster that reading in python, but I get the same behavior when trying gunicorn between return body and return [body.read()]
Thanks for the package