Using directly the Thread API is error prone because low level.
Using parallelStream() may be interesting but it may also be an issue because the processed stream could consume all CPU core available for your application.
Which means that other HTTP client requests of your application could be served very lately.
And note also that the number of threads used in parallelStream() is a JVM implementation detail and doesn't make part of the public API.
The ExecutorService API that allows to specify a number of threads available in the pool looks a better/more robust alternative.
Spring Boot provides a built-in feature that wraps it.
You could extract the individual task to invoke into a method such as :
@Async public Future<ResponseEntity<String>> getEmployee(long employeeId, HttpEntity<String> requestEntity) { String url = "http://dummy.restapiexample.com/api/v1/employee/" + employeeId; ResponseEntity<String> responseEntity = restTemplate.exchange(url, HttpMethod.GET, requestEntity, String.class); return new AsyncResult<ResponseEntity<String>>(responseEntity); }
Now call it :
private void callApi(List<Long> employeeList, HttpEntity<String> requestEntity) { // Do async calls and store futures in a List List<Future<ResponseEntity<String>>> futures = new ArrayList<>(); for (Long id : employeeList) { futures.add(getEmployee(id, requestEntity)); } // Then process list of future for (Future<ResponseEntity<String>> future : futures) try{ saveInDatabase(future.get()); } catch(Exception e){ //handle the exception } } }
As a side note, doing the saveInDatabase() operation into a loop is not a right approach.
Instead of, you want to batch the database insertions because you have many of them to do. Something like :
private void callApi(List<Long> employeeList, HttpEntity<String> requestEntity) { List<ResponseEntity<String>> responseEntities = employeeList.stream() .map(id -> getEmploye(id)) .map(future -> { try{return future.get();} catch(Exception e){ //handle the exception } } ) .collect(toList()); saveInDatabase(responseEntities); }
To make the @Asynch feature working, you have to add @EnableAsync on a @Configuration class of your application.
Optionally you can define a Executor bean with the pool/queue configuration that suit to your need. Beware : if you don't define an Executor bean, Spring will create a SimpleAsyncTaskExecutor and use that (it creates a Thread by task and don't reuse them).
For example :
@SpringBootApplication @EnableAsync public class Application { public static void main(String[] args) { SpringApplication.run(Application.class, args); } @Bean public Executor taskExecutor() { ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor(); executor.setCorePoolSize(2); executor.setMaxPoolSize(2); executor.setQueueCapacity(500); executor.setThreadNamePrefix("GithubLookup-"); executor.initialize(); return executor; } }
