Skip to main content
Formatting improvements
Source Link
Theodor Zoulias
  • 46.1k
  • 8
  • 112
  • 155

I'm actually facing a problem with parallelism on asynchronous tasks.

My goal is to have an async TaskTask which connects to multiple endpoints to retrieve the same kind of data. A list with all "connection strings" is provided. The data retrieval shall happen in parallel with all connections and the retrieved data of each connection shall be merged into a single container which will be returned, once all parallel async tasks finished.

My first idea was using AsParallel.ForAll()AsParallel.ForAll() - passing the configurations, and an async Task to fill a (Dataflow) BufferBlockBufferBlock with SendAsyncSendAsync. Now I got stuck on the problem, that ParallelParallel queries seem to only take Action<>Action<T> as parameter and not Task<>Task<T>. This will lead to an undesired behavior if an exception occurs.

Does anyone have a better approach to my problem?

Here is a snippet for better visualization:

configurations.AsParallel().ForAll(async config => await FetchAllData(config, resultBufferBlock, cancellationToken));

configurations.AsParallel().ForAll(async config => await FetchAllData(config, resultBufferBlock, cancellationToken)); 

After some comments: I basically need Parallel.ForEachAsyncParallel.ForEachAsync which is available in .NET 6. we will move to .NET 6 in a couple of months. This is already planned. Although I would appreciate a solution for .NET 5. Actually it looks like I will use the bad way using the task as Action and fix that later…

I'm actually facing a problem with parallelism on asynchronous tasks.

My goal is to have an async Task which connects to multiple endpoints to retrieve the same kind of data. A list with all "connection strings" is provided. The data retrieval shall happen in parallel with all connections and the retrieved data of each connection shall be merged into a single container which will be returned, once all parallel async tasks finished.

My first idea was using AsParallel.ForAll() - passing the configurations, and an async Task to fill a (Dataflow) BufferBlock with SendAsync. Now I got stuck on the problem, that Parallel queries seem to only take Action<> as parameter and not Task<>. This will lead to an undesired behavior if an exception occurs.

Does anyone have a better approach to my problem?

Here is a snippet for better visualization:

configurations.AsParallel().ForAll(async config => await FetchAllData(config, resultBufferBlock, cancellationToken));

After some comments: I basically need Parallel.ForEachAsync which is available in .NET 6. we will move to .NET 6 in a couple of months. This is already planned. Although I would appreciate a solution for .NET 5. Actually it looks like I will use the bad way using the task as Action and fix that later…

I'm actually facing a problem with parallelism on asynchronous tasks.

My goal is to have an async Task which connects to multiple endpoints to retrieve the same kind of data. A list with all "connection strings" is provided. The data retrieval shall happen in parallel with all connections and the retrieved data of each connection shall be merged into a single container which will be returned, once all parallel async tasks finished.

My first idea was using AsParallel.ForAll() - passing the configurations, and an async Task to fill a (Dataflow) BufferBlock with SendAsync. Now I got stuck on the problem, that Parallel queries seem to only take Action<T> as parameter and not Task<T>. This will lead to an undesired behavior if an exception occurs.

Does anyone have a better approach to my problem?

Here is a snippet for better visualization:

configurations.AsParallel().ForAll(async config => await FetchAllData(config, resultBufferBlock, cancellationToken)); 

After some comments: I basically need Parallel.ForEachAsync which is available in .NET 6. we will move to .NET 6 in a couple of months. This is already planned. Although I would appreciate a solution for .NET 5. Actually it looks like I will use the bad way using the task as Action and fix that later…

added 307 characters in body
Source Link
Otto V.
  • 179
  • 5
  • 11

I'm actually facing a problem with parallelism on asynchronous tasks.

My goal is to have an async Task which connects to multiple endpoints to retrieve the same kind of data. A list with all "connection strings" is provided. The data retrieval shall happen in parallel with all connections and the retrieved data of each connection shall be merged into a single container which will be returned, once all parallel async tasks finished.

My first idea was using AsParallel.ForAll() - passing the configurations, and an async Task to fill a (Dataflow) BufferBlock with SendAsync. Now I got stuck on the problem, that Parallel queries seem to only take Action<> as parameter and not Task<>. This will lead to an undesired behavior if an exception occurs.

Does anyone have a better approach to my problem?

Here is a snippet for better visualization:

configurations.AsParallel().ForAll(async config => await FetchAllData(config, resultBufferBlock, cancellationToken));

After some comments: I basically need Parallel.ForEachAsync which is available in .NET 6. we will move to .NET 6 in a couple of months. This is already planned. Although I would appreciate a solution for .NET 5. Actually it looks like I will use the bad way using the task as Action and fix that later…

I'm actually facing a problem with parallelism on asynchronous tasks.

My goal is to have an async Task which connects to multiple endpoints to retrieve the same kind of data. A list with all "connection strings" is provided. The data retrieval shall happen in parallel with all connections and the retrieved data of each connection shall be merged into a single container which will be returned, once all parallel async tasks finished.

My first idea was using AsParallel.ForAll() - passing the configurations, and an async Task to fill a (Dataflow) BufferBlock with SendAsync. Now I got stuck on the problem, that Parallel queries seem to only take Action<> as parameter and not Task<>. This will lead to an undesired behavior if an exception occurs.

Does anyone have a better approach to my problem?

Here is a snippet for better visualization:

configurations.AsParallel().ForAll(async config => await FetchAllData(config, resultBufferBlock, cancellationToken));

I'm actually facing a problem with parallelism on asynchronous tasks.

My goal is to have an async Task which connects to multiple endpoints to retrieve the same kind of data. A list with all "connection strings" is provided. The data retrieval shall happen in parallel with all connections and the retrieved data of each connection shall be merged into a single container which will be returned, once all parallel async tasks finished.

My first idea was using AsParallel.ForAll() - passing the configurations, and an async Task to fill a (Dataflow) BufferBlock with SendAsync. Now I got stuck on the problem, that Parallel queries seem to only take Action<> as parameter and not Task<>. This will lead to an undesired behavior if an exception occurs.

Does anyone have a better approach to my problem?

Here is a snippet for better visualization:

configurations.AsParallel().ForAll(async config => await FetchAllData(config, resultBufferBlock, cancellationToken));

After some comments: I basically need Parallel.ForEachAsync which is available in .NET 6. we will move to .NET 6 in a couple of months. This is already planned. Although I would appreciate a solution for .NET 5. Actually it looks like I will use the bad way using the task as Action and fix that later…

Source Link
Otto V.
  • 179
  • 5
  • 11

How can I gather results from parallel async tasks in C#?

I'm actually facing a problem with parallelism on asynchronous tasks.

My goal is to have an async Task which connects to multiple endpoints to retrieve the same kind of data. A list with all "connection strings" is provided. The data retrieval shall happen in parallel with all connections and the retrieved data of each connection shall be merged into a single container which will be returned, once all parallel async tasks finished.

My first idea was using AsParallel.ForAll() - passing the configurations, and an async Task to fill a (Dataflow) BufferBlock with SendAsync. Now I got stuck on the problem, that Parallel queries seem to only take Action<> as parameter and not Task<>. This will lead to an undesired behavior if an exception occurs.

Does anyone have a better approach to my problem?

Here is a snippet for better visualization:

configurations.AsParallel().ForAll(async config => await FetchAllData(config, resultBufferBlock, cancellationToken));