24

I'm using ApacheBench (ab) to measure the performance of two nginx on Linux. They have same config file. The Only difference is one of nginx is running in a docker container.

Nginx on Host System:

Running: ab -n 50000 -c 1000 http://172.17.0.2:7082/ Concurrency Level: 1000 Time taken for tests: 9.376 seconds Complete requests: 50000 Failed requests: 0 Total transferred: 8050000 bytes HTML transferred: 250000 bytes Requests per second: 5332.94 [#/sec] (mean) Time per request: 187.514 [ms] (mean) Time per request: 0.188 [ms] (mean, across all concurrent requests) Transfer rate: 838.48 [Kbytes/sec] received

Nginx in docker container:

Running: ab -n 50000 -c 1000 http://172.17.0.2:6066/ Concurrency Level: 1000 Time taken for tests: 31.274 seconds Complete requests: 50000 Failed requests: 0 Total transferred: 8050000 bytes HTML transferred: 250000 bytes Requests per second: 1598.76 [#/sec] (mean) Time per request: 625.484 [ms] (mean) Time per request: 0.625 [ms] (mean, across all concurrent requests) Transfer rate: 251.37 [Kbytes/sec] received

Just wondering why the container one has such a poor performance

nginx.conf:

worker_processes auto; worker_rlimit_nofile 10240; events { use epoll; multi_accept on; worker_connections 4096; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 10; client_header_timeout 10; client_body_timeout 10; send_timeout 10; tcp_nopush on; tcp_nodelay on; server { listen 80; server_name localhost; location / { return 200 'hello'; } error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } }
Improve this question

2 Answers 2

Reset to default
17

I'd like to add to @Andrian Mouat's answer, something I've just found in the docs.

It is written in the Docker run reference:

NETWORK: HOST

Compared to the default bridge mode, the host mode gives significantly better networking performance since it uses the host’s native networking stack whereas the bridge has to go through one level of virtualization through the docker daemon.

It is recommended to run containers in this mode when their networking performance is critical, for example, a production Load Balancer or a High Performance Web Server.

Some tests with Flame Graphs follow:

When using the host’s native networking stack with --net=host, there are fewer system calls and this is clearly depicted in the following Flame Graphs. Details:

  • system wide captures for 30sec: sudo perf record -F 99 -a -g -- sleep 30
  • ab test from another physical machine: ab -n 50000 -c 1000 http://my-host-ip/ (takes place while capturing)

For more info on Flame Graphs, check Brendan Gregg's website: www.brendangregg.com/

Flame Graph when publishing port -p 80:80:

Full picture here

Zoomed to nginx part:

docker nginx flame graph publish port zoomed



Flame Graph when using --net=host:

Full picture here

Zoomed to nginx part:

docker nginx flame graph net host zoomed

Improve this answer
Sign up to request clarification or add additional context in comments.

Comments

9

How are you running the container? Is it using the default Docker bridge network? If so, try running the tests with --net=host and see what the results look like.

Improve this answer

6 Comments

Wow! It gains about 50% performance improvement! Time taken for tests: 16.944 seconds
Yeah, the bridge networking is really simple to get things working, but it's not good for efficiency. There's still a fairly large difference, it would be interesting to figure out where that comes from. Writing to log and cache files might also be slower due to the overlay fs.
I tried this but still get about only 60% of running nginx directly on the host (when I don't have the --net=host parameter, I get only 35%).
I'd also check if you are using the same library (glibc vs musl) and the effects of the container filesystem (try using volumes for any data/files).
Just a heads up, --net host unfortunately does not work on Docker for Mac - seems to only be supported on Linux
|

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.