425

I need to run multiple async tasks in a console application, and wait for them all to complete before further processing.

There's many articles out there, but I seem to get more confused the more I read. I've read and understand the basic principles of the Task library, but I'm clearly missing a link somewhere.

I understand that it's possible to chain tasks so that they start after another completes (which is pretty much the scenario for all the articles I've read), but I want all my Tasks running at the same time, and I want to know once they're all completed.

What's the simplest implementation for a scenario like this?

0

12 Answers 12

633

Both answers didn't mention the awaitable Task.WhenAll:

var task1 = DoWorkAsync(); var task2 = DoMoreWorkAsync(); await Task.WhenAll(task1, task2); 

The main difference between Task.WaitAll and Task.WhenAll is that the former will block (similar to using Wait on a single task) while the latter will not and can be awaited, yielding control back to the caller until all tasks finish.

More so, exception handling differs:

Task.WaitAll:

At least one of the Task instances was canceled -or- an exception was thrown during the execution of at least one of the Task instances. If a task was canceled, the AggregateException contains an OperationCanceledException in its InnerExceptions collection.

Task.WhenAll:

If any of the supplied tasks completes in a faulted state, the returned task will also complete in a Faulted state, where its exceptions will contain the aggregation of the set of unwrapped exceptions from each of the supplied tasks.

If none of the supplied tasks faulted but at least one of them was canceled, the returned task will end in the Canceled state.

If none of the tasks faulted and none of the tasks were canceled, the resulting task will end in the RanToCompletion state. If the supplied array/enumerable contains no tasks, the returned task will immediately transition to a RanToCompletion state before it's returned to the caller.

Sign up to request clarification or add additional context in comments.

13 Comments

When I try this my tasks run sequentially? Does one have to start each task individually before await Task.WhenAll(task1, task2); ?
@Zapnologica Task.WhenAll doesn't start the tasks for you. You have to provide them "hot", meaning already started.
Ok. That makes sense. So what will your example do? Because you have not started them?
@YuvalItzchakov thank you very much! It is so simple but it helped me a lot today! Is worth at least +1000 :)
@Pierre I'm not following. What does StartNew and spinning new tasks have to do with asynchronously waiting on them all?
|
139

You could create many tasks like:

List<Task> TaskList = new List<Task>(); foreach(...) { var LastTask = new Task(SomeFunction); LastTask.Start(); TaskList.Add(LastTask); } Task.WaitAll(TaskList.ToArray()); 

7 Comments

I would recommend WhenAll
Is it possible to start multiple new threads, at the same time, using the await keyword rather than .Start() ?
@MattW No, when you use await, it would wait for it to complete. In this case you would not be able to create a multi-threaded environment. This is the reason that all the tasks are waited at the end of the loop.
Downvote for future readers since it isn't made clear that this is a blocking call.
See the accepted answer for reasons why not to do this.
|
79

You can use WhenAll which will return an awaitable Task or WaitAll which has no return type and will block further code execution simular to Thread.Sleep until all tasks are completed, canceled or faulted.

WhenAll WaitAll
Any of the supplied tasks completes in a faulted state A task with the faulted state will be returned. The exceptions will contain the aggregation of the set of unwrapped exceptions from each of the supplied tasks. An AggregateException will be thrown.
None of the supplied tasks faulted but at least one of them was canceled The returned task will end in the TaskStatus.Canceled state An AggregateException will be thrown which contains an OperationCanceledException in its InnerExceptions collection
An empty list was given An ArgumentException will be thrown The returned task will immediately transition to a TaskStatus.RanToCompletion State before it's returned to the caller.
Doesn't block the current thread Blocks the current thread

Example

var tasks = new Task[] { TaskOperationOne(), TaskOperationTwo() }; Task.WaitAll(tasks); // or await Task.WhenAll(tasks); 

If you want to run the tasks in a particular/specific order you can get inspiration from this answer.

3 Comments

sorry for coming to the party late but, why do you have await for each operation and at the same time use WaitAll or WhenAll. Shouldn't tasks in Task[] initialization be without await?
@dee zg You are right. The await above defeats the purpose. I ll change my answer and remove them.
yes, that's it. thanks for clarification! (upvote for nice answer)
32

The best option I've seen is the following extension method:

public static Task ForEachAsync<T>(this IEnumerable<T> sequence, Func<T, Task> action) { return Task.WhenAll(sequence.Select(action)); } 

Call it like this:

await sequence.ForEachAsync(item => item.SomethingAsync(blah)); 

Or with an async lambda:

await sequence.ForEachAsync(async item => { var more = await GetMoreAsync(item); await more.FrobbleAsync(); }); 

1 Comment

I did this overload to get the return type: public static Task<R[]> ForEachAsync<T, R>(this IEnumerable<T> sequence, Func<T, Task<R>> action) { return Task.WhenAll(sequence.Select(action)); }
17

Yet another answer...but I usually find myself in a case, when I need to load data simultaneously and put it into variables, like:

var cats = new List<Cat>(); var dog = new Dog(); var loadDataTasks = new Task[] { Task.Run(async () => cats = await LoadCatsAsync()), Task.Run(async () => dog = await LoadDogAsync()) }; try { await Task.WhenAll(loadDataTasks); } catch (Exception ex) { // handle exception } 

3 Comments

If LoadCatsAsync() and LoadDogAsync() are just database calls they are IO-bound. Task.Run() is for CPU-bound work; it adds additional unnecessary overhead if all you are doing is waiting for a response from the database server. Yuval's accepted answer is the right way for IO-bound work.
@StephenKennedy could you please clarify what kind of overhead and how much it can impact performance? Thanks!
That would be quite hard to summarise in the comments box :) Instead I recommend reading Stephen Cleary's articles - he's an expert on this stuff. Start here: blog.stephencleary.com/2013/10/…
9

Do you want to chain the Tasks, or can they be invoked in a parallel manner?

For chaining
Just do something like

Task.Run(...).ContinueWith(...).ContinueWith(...).ContinueWith(...); Task.Factory.StartNew(...).ContinueWith(...).ContinueWith(...).ContinueWith(...); 

and don't forget to check the previous Task instance in each ContinueWith as it might be faulted.

For the parallel manner
The most simple method I came across: Parallel.Invoke Otherwise there's Task.WaitAll or you can even use WaitHandles for doing a countdown to zero actions left (wait, there's a new class: CountdownEvent), or ...

5 Comments

Appreciate the answer, but your suggestions could have been explained a little more.
@drminnaar which other explanation beside the links to msdn with examples do you need? you didn't even click on the links, did you?
I clicked on the links, and I read the content. I was going for the Invoke, but there were a lot of If's and But's about whether it runs asynchronously or not. You were editing your answer continuously. The WaitAll link you posted was perfect, but I went for the answer that demonstrated the same functionality in a quicker and easier to read way. Don't take offense, your answer still provides good alternatives to other approaches.
@drminnaar no offense taken here, i am just curious :)
Parallel.Invoke works like a charm... didn't know about it until I got to this answer... Thx!
6

This is how I do it with an array Func<>:

var tasks = new Func<Task>[] { () => myAsyncWork1(), () => myAsyncWork2(), () => myAsyncWork3() }; await Task.WhenAll(tasks.Select(task => task()).ToArray()); //Async Task.WaitAll(tasks.Select(task => task()).ToArray()); //Or use WaitAll for Sync 

2 Comments

Why don't you just keep it as Task Array?
If your not careful @talha-talip-açıkgöz your execute the Tasks when you didn't expect them to execute. Doing it as a Func delegate makes your intent clear.
2

might like to take a look at below link which explains which explains best practice for await async. https://learn.microsoft.com/en-us/dotnet/csharp/asynchronous-programming/async-scenarios

Use this... Instead of this... When wishing to do this... await Task.Wait Task.Result Retrieving the result of a background task await Task.WhenAny Task.WaitAny Waiting for any task to complete await Task.WhenAll Task.WaitAll Waiting for all tasks to complete await Task.Delay Thread.Sleep Waiting for a period of time 

Comments

1

Since there is another convenient option present - Task.WhenEach which allows to process the tasks upon completion:

Creates an IAsyncEnumerable<T> that will yield the supplied tasks as those tasks complete.

int[] delays = [100, 50, 0]; var tasksToRun = delays .Select(async (delay, index) => { await Task.Delay(delay); return (index, delay); }); var sw = Stopwatch.StartNew(); await foreach (var completed in Task.WhenEach(tasksToRun)) { Console.Write($"{completed.Status} "); var result = completed.Result; Console.WriteLine($"index: {result.index} delay: {result.delay} elapsed: {sw.ElapsedMilliseconds}ms"); } 

Which prints "on my machine":

RanToCompletion index: 2 delay: 0 elapsed: 27ms RanToCompletion index: 1 delay: 50 elapsed: 57ms RanToCompletion index: 0 delay: 100 elapsed: 102ms 

Comments

-1

I prepared a piece of code to show you how to use the task for some of these scenarios.

 // method to run tasks in a parallel public async Task RunMultipleTaskParallel(Task[] tasks) { await Task.WhenAll(tasks); } // methode to run task one by one public async Task RunMultipleTaskOneByOne(Task[] tasks) { for (int i = 0; i < tasks.Length - 1; i++) await tasks[i]; } // method to run i task in parallel public async Task RunMultipleTaskParallel(Task[] tasks, int i) { var countTask = tasks.Length; var remainTasks = 0; do { int toTake = (countTask < i) ? countTask : i; var limitedTasks = tasks.Skip(remainTasks) .Take(toTake); remainTasks += toTake; await RunMultipleTaskParallel(limitedTasks.ToArray()); } while (remainTasks < countTask); } 

1 Comment

how get the results of Tasks? For example, for merge "rows" (from N tasks in parallel) in a datatable and bind it to gridview asp.net ?
-1

There should be a more succinct solution than the accepted answer. It shouldn't take three steps to run multiple tasks simultaneously and get their results.

  1. Create tasks
  2. await Task.WhenAll(tasks)
  3. Get task results (e.g., task1.Result)

Here's a method that cuts this down to two steps:

 public async Task<Tuple<T1, T2>> WhenAllGeneric<T1, T2>(Task<T1> task1, Task<T2> task2) { await Task.WhenAll(task1, task2); return Tuple.Create(task1.Result, task2.Result); } 

You can use it like this:

var taskResults = await Task.WhenAll(DoWorkAsync(), DoMoreWorkAsync()); var DoWorkResult = taskResults.Result.Item1; var DoMoreWorkResult = taskResults.Result.Item2; 

This removes the need for the temporary task variables. The problem with using this is that while it works for two tasks, you'd need to update it for three tasks, or any other number of tasks. Also it doesn't work well if one of the tasks doesn't return anything. Really, the .Net library should provide something that can do this

1 Comment

it's not a bad idea, and would probably be given a better rating if you fix the usage example; you probably meant to call WhenAllGeneric instead of WhenAll.
-3

If you're using the async/await pattern, you can run several tasks in parallel like this:

public async Task DoSeveralThings() { // Start all the tasks Task first = DoFirstThingAsync(); Task second = DoSecondThingAsync(); // Then wait for them to complete var firstResult = await first; var secondResult = await second; } 

5 Comments

This approach introduces the risk of leaking a fire-and-forget task, in case the first task completes with failure before the completion of the second task. The correct way to await multiple tasks is the Task.WhenAll method: await Task.WhenAll(first, second);. Then you can await them individually to get their results, because you know that all have completed successfully.
@TheodorZoulias Is there a problem with leaking fire-and-forget tasks? It seems that for a console application at least, you don't get much benefit from waiting ten minutes on WhenAll to find out that you misspelled the input file name.
It depends on what this fire-and-forget task does. In the best case it just consumes resources, like network bandwidth, that are going to waste. In the worst case it modifies the state of the application, at a time when it's not expected to happen. Imagine that a user clicks a button, they get an error message, the button is re-enabled, and then the ProgressBar continues moving up and down by the ghost task... This never happens by any tool provided by Microsoft (Parallel, PLINQ, TPL Dataflow etc). All these APIs do not return before all internally initiated operations are completed.
If a failure of one task makes the result of another task irrelevant, then the correct course of action is to cancel the still-running task, and await it to complete as well. Awaiting each task sequentially, as your answer suggests, is rarely a good idea. If you decide that leaking fire-and-forget tasks is OK for your use case, then symmetrically a failure on second should also leak the first. Your code doesn't support that. Its leaking behavior is asymmetric.
this is just a horrible suggestion that doesn't include any supportive or explanatory documentation. It should be deleted as noobs will assume the wrong things from this

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.