How can I gather results from parallel async tasks in C#?-CodePudding

I'm actually facing a problem with parallelism on asynchronous tasks.

My goal is to have an async Task which connects to multiple endpoints to retrieve the same kind of data. A list with all "connection strings" is provided. The data retrieval shall happen in parallel with all connections and the retrieved data of each connection shall be merged into a single container which will be returned, once all parallel async tasks finished.

My first idea was using AsParallel.ForAll() - passing the configurations, and an async Task to fill a (Dataflow) BufferBlock with SendAsync. Now I got stuck on the problem, that Parallel queries seem to only take Action<T> as parameter and not Task<T>. This will lead to an undesired behavior if an exception occurs.

Does anyone have a better approach to my problem?

Here is a snippet for better visualization:

configurations.AsParallel().ForAll(async config => await FetchAllData(config,
    resultBufferBlock, cancellationToken));

After some comments: I basically need Parallel.ForEachAsync which is available in .NET 6. we will move to .NET 6 in a couple of months. This is already planned. Although I would appreciate a solution for .NET 5. Actually it looks like I will use the bad way using the task as Action and fix that later…

CodePudding user response：

I'm actually facing a problem with parallelism on asynchronous tasks.

The problem you describe is concurrency (doing more than one thing at a time) and AFAICT does not need parallelism (multiple threads).

Does anyone have a better approach to my problem?

The way to do asynchronous concurrency is to start all the tasks (commonly using a LINQ select with an asynchronous lambda), and then use Task.WhenAll to join them.

var tasks = configurations.Select(async config => await FetchAllData(config, resultBufferBlock, cancellationToken));
await Task.WhenAll(tasks);

After some comments: I basically need Parallel.ForEachAsync which is available in .NET 6.

I disagree. Parallel.ForEachAsync is for when you have a complex mixture of CPU-bound and I/O-bound code and need to do both parallelism and asynchronous concurrency. In this case parallelism isn't necessary.