Implement parallel computation for fetching team's Data-CodePudding

I am creating an App where I fetch post and comments from Teams. Previously I was using for loop and fetching details each time i am member of one by one, As I am member of more than 40-50 teams, It's taking lot of time to fetch the comments and posts.

I am trying to apply some parallelism in the code:

 public async Task<Package> getTeamsPostData(Teams team,string token)
    {
        Package pkgTeam = new Package();
   
        //some computation to fetch post and comment of this team
        //Some await functions is also called from this section

        return pkgTeam;
    }

    public async Task<Package> GetTeamsCommentsAndChannelActivityByDate(string Date, string token,List<Teams> teams)
    {
   
            var taskArray = teams.Select(t => Task.Factory.StartNew(() => getTeamsPostData(t, token))).ToArray();

            //then wait for all tasks to complete asyncronously
            Task.WaitAll(taskArray);

            //then add the result of all the tasks to r in a treadsafe fashion
            var r = taskArray.Select(task => task.Result).ToList();


        return r;
    }

I have done above changes to implement parallel processing, but in result I am getting Status as Waiting for Activation and i am not getting any post and comment data in the result. Also it is not taking any time for executing WaitAll method

CodePudding user response：

You are using the Task.Factory.StartNew method to create tasks and then calling WaitAll on the array of tasks.

Task.Factory.StartNew is used to create tasks that execute an action that is represented by a delegate, rather than an asynchronous method.

To run your getTeamsPostData method asynchronously, you can use the Task.Run method instead. It runs a delegate asynchronously, and returns a task that represents the asynchronous operation.

Modify your code this way

public async Task<Package> GetTeamsCommentsAndChannelActivityByDate(string Date, string token,List<Teams> teams)
{
    var taskArray = teams.Select(t => Task.Run(() => getTeamsPostData(t, token))).ToArray();

    // Now wait for all tasks to complete asynchronously
    await Task.WhenAll(taskArray);

    var r = taskArray.Select(task => task.Result).ToList();
    return r;
}

this will run getTeamsPostData method asynchronously for each team.

CodePudding user response：

First of all, your code can't be compiled because GetTeamsCommentsAndChannelActivityByDate() method returns List<Task<Package>> but your method signature return type is Task<Package>.

Your problem probably occurs because your Select() returns IEnumerable<Task<Task<T>>> instead of IEnumerable<Task<T>>. Then in the Task.WaitAll(taskArray) line you are waiting for the Task<Task<T>> instead of each inner task.

You can use Unwrap() method to resolve your problem. Unwrap() allows to create a proxy Task that represents the entire asynchronous operation. Example of use:

var taskArray = Enumerable.Range(0, 10).Select(i => Task.Factory.StartNew(async () =>
{
    await Task.Delay(10000);
    return i;
}).Unwrap()).ToArray();


Task.WaitAll(taskArray);

In your case:

var taskArray = teams.Select(t => Task.Factory.StartNew(async () => await getTeamsPostData(t, token)).Unwrap()).ToArray();

You can also use Task.Run() instead of Task.Factory.StartNew() when you don't need configuration as @Jiya suggested. It unwraps asynchronous operation automatically under the hood.