Executor: Wait for specific tasks to finish

Question

The server application I am running gets multiple requests for tasks which I want to handle using a task system.

Each task is represented as a Runnable that will demand n number of threads from a thread pool where n is smaller or equal to the thread pool size. The thread pool of course is necessary in order to not overload the CPU with too many threads.

However, some of those tasks can be multi threaded and some can not. That is why it might be necessary for one task to wait for all its specific threads to finish in order to merge the results from those threads for the final result.

If one uses multiple Thread instances one might join those like this:

try { // Wait for all threads to finish their tasks for (Thread thread : threads) { thread.join(); } } catch (InterruptedException e) { e.printStackTrace(); } // Finish job here ..

but I'd need something like this using java.util.concurrent.Executor or anything similar that works with a thread pool.

I may be wrong here, but it feels like you're using Executor on a wrong level of abstraction. If you have a task that requires n threads to run, I suggest you let that task create it's own thread pool, or break it down into smaller 1-thread tasks. — aioobe
– aioobe, Commented Jan 15, 2015 at 10:05
@aioobe you may end up with too many executors/pools which is not efficient — bachr
– bachr, Commented Jan 15, 2015 at 10:08
One could let each task do someSharedSemaphore.acquire(n) before starting its work. — aioobe
– aioobe, Commented Jan 15, 2015 at 10:11
Arbi is right. I can not let a task run its own thread. It is the whole application that is limited to a specific number of threads. Imagine the client click 100 times on "run task". The server would execute to many threads causing each task to take forever. No, semaphore is too low-level — Stefan Falk
– Stefan Falk, Commented Jan 15, 2015 at 10:12

bachr · Accepted Answer · 2015-01-15 10:24:18Z

You can use ExecutorService along with a CyclicBarrier for each task as follows:

public class ThreadedTask implements Runnable { CyclicBarrier barrier; public ThreadedTask(CyclicBarrier barrier) { this.barrier = barrier; } @Override public void run() { // do something barrier.await(); } } ExecutorService executor = Executors.newFixedThreadPool(pool_size); ... CyclicBarrier barrier = new CyclicBarrier(n+1); for(int i=0; i<n; i++) { // launch all tasks executor.submit(new ThreadedTask(barrier)); } // waits for the tasks to finish or timeout barrier.await(seconds, TimeUnit.SECONDS);

This does not seem like what I need. I don't want to wait for all threads. I want to wait for a bundle of threads that were executed by a specific task. Your answer seems to wait for all tasks ever executed.
not really, the part after the ... is the body of a task (I suppose in run() as your tasks implements Runnable). It's just an alternative in case you want to use the powerful barrier concept.
Okay, now I get it! This is also a valid solution for my problem but I'll go for the other one. +1 anyway :)

DmitryKanunnikoff · Accepted Answer · 2015-01-15 10:16:04Z

If I understand you correctly, you will need something like this (but your architecture seems too complicated):

class MyTask implements Runnable { @Override public void run() { // some work } }

After that:

ExecutorService executorService = Executors.newFixedThreadPool(2000); ArrayList<Future> futures = new ArrayList<>(); futures.add(executorService.submit(new MyTask())); futures.add(executorService.submit(new MyTask())); futures.add(executorService.submit(new MyTask())); for (Future future: futures) { try { future.get(100, TimeUnit.SECONDS); } catch (Throwable cause) { // process cause } }

Each future.get() will wait for task ending (max 100 seconds in this example).

I think the task should implements Callable not Runnable in order the future
But if some tasks only require 1 thread, then it's pointless to wait for them to finish before launching the next.
@aioobe Yes, you are right. I suggested only idea "how to wait".
That is what I was looking for. Didn't know that this is called Future. What an odd name for an object ..

Collectives™ on Stack Overflow

Executor: Wait for specific tasks to finish

2 Answers 2

3 Comments

5 Comments

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

3 Comments

5 Comments

Related