1

My spring boot application is not accepting desired load. In order to isolate problem I created a very simple service that just sleeps for 500ms and returns success response back. Following is my configuration:

server: tomcat: protocol: org.apache.coyote.http11.Http11NioProtocol accesslog: enabled: false max-connections: 10000 threads: max: 1000 min-spare: 1000 accept-count: 1000

I connected Jconsole to the process and I see min-spare is taking effect as I see right number of threads there when I change that value.

My SoapUI load test configuration is below:

Thread: 5000 (5000 concurrent users) Strategy: Simple Test Delay: 0 Random: 0

Problem is this that application is not taking more that 400 TPS no matter how I change tomcat configuration or SoapUI load test configuration.

Result from SOAP UI test with above specified configuration is below:

min: 500 max: 959 avg: 510 tps: 386 err: 0
Improve this question
4
  • Are you performing this test on the same machine? soapui and your application? Commented Jun 6 at 17:42
  • Are you using seperate machines? Why are you tweakikng tomcat have you identified that that is the cause? How many DB connections are you allowed from your connection pool? Feels like you are just trying to juggle some numbers without having investigated what the actual problem is. Commented Jun 7 at 10:25
  • Yes both soapui and my service are running on the same machine. Also when I tried using JMeter, same setup gave 1200TPS. @M. Deinum The service that I am hitting does not connect to db does pool size still matter? Avg response time is 500ms as shown on results above. If I am using 1000/5000. I was expecting this basic service at least to go higher than 400 tps. Commented Jun 7 at 12:54
  • Never run the tests and the api on the same machine, you should run the tests and app on a different machine. Next to that your machine isn't production you should run against a production like environment not your local development laptop and expect it to have the same performance as production. Commented Jun 7 at 13:01

1 Answer 1

Reset to default
0

  1. Use at least three load generators. Two for primary load. One for a control set of one of each type of virtual user involves in your test. All users, control and global, should degrade at the same rate if the service is the problem

  2. You have a deliberate delay of 500ms before responding. This will impact scalability. That clock tick has to be addressed for every session and this is a hardware interrupt call to get the clock tick. So your service design could simply be hitting a limit on scalability

  3. Test design. No think time delays is a client-server model. The model anticipates that delays between requests on system A are used to handle requests from systems B, C, and so forth. Collapse time between requests to zero and you have collapsed the model. It's better to go with larger numbers of users that havew some delays in use

  4. Your ramp up/down model for 5000 users? hitting it all at once can create a system shock as resources are allocated and locked. in the performance testing universe we refer to this as the vodka problem. Take one user and one bottle of vodka, combine in seconds and you can get system shutdown fast. Take same user, add a thimble full of vodka at a time and you wil find the scaling limit in number of vodka thimbles when the system response begins to obvious degrade. On the ramp down side it's the junkie problem. Take all of the load away at once and you can also induce system shock as sessions are left dangling, resources locked waiting for cleanup that may never come because code execution was cut. Just as you want a clean ramp up, you want a clean ramp down, as this will help you understand how your system recovers resources.

  5. On the resource front. it is recommended that you consider monitoring the finite resource pool, CPU, DISK, MEMORY, and NETWORK, to see where you begin to hit limits during rampup and load. This is an area that is almost never addressed in self-trained performance testers.

For insight in performance test design, there are few documents better than the TPC-C benchmark test definition available at TPC.ORG. You'll find references to the above items in the benchmark as well as a few others that you might consider for your test design.

Improve this answer
Sign up to request clarification or add additional context in comments.

1 Comment

Thank you @James Pulley. I tried different Ramp up of 60 sec and 120 sec originally but since service was not accepting beyond 400 TPS I just set it to 0. I did not see any limits being breached for CPU around 12-15% DISK, or Memory. Also I ran this on Blazemeter running on different hosts and application running on another host originally where my actual service was not going beyond 1000 TPS which it should have gone far beyond 1000TPS (running on AWS ECS). In order to find if my tomcat settings are working correctly, I setup a dummy service with 500ms delay and hit it locally.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.