Run asynchronous methods in parallel, but ensure the return have data-CodePudding

Can we call multiple asynchronous methods and ensure that the returned object must wait till all methods run are complete?

Here is the scenario. I have a main method which calls other several methods. Each sub method calls a separate ExecuteAsync API to return results. Some of the methods are dependent on the result returned from previous methods. Running them sequentially this way, it takes much time to complete a request. Can we make parallel call to each method and the final returned object should have all of the data?

Below is a sample code for what I am trying to achieve.

public Student GetStudentDetails()
{
    Student objStudent = new Student();

    objStudent.Name = Helpers.GetStudentName(); // Takes 1 second

    objStudent.CoursesIDs = Helpers.GetStudentCourses(); //Returns a list of string > Takes 1 second
    foreach (String courseID in objStudent.CoursesIDs)
    {
        string courseName = Helpers.GetCourseName(courseID); // Each Call takes 1 second. 10 courses X 10 seconds
        objStudent.CourseName.Add(new Courses { ID:courseID, Title:courseName });
    }

    objStudent.MarksIDs = Helpers.GetStudentMarks(); //Returns a list of string > Takes 1 second
    foreach (String MarksID in objStudent.MarksIDs)
    {
        string ActualMarks = Helpers.GetActualMarks(MarksID); // Each Call takes 1 second. 10 calls X 10 seconds
        objStudent.Marks.Add(new Marks { ID:MarksID, Title:ActualMarks });
    }

    return objStudent;
}

This is just sample code to get overall idea. I had to implement more bigger calls than this but I believe the idea should be same.

How I can make my function to run GetStudentName, GetCourseName and GetActualMarks simultaneously and the objStudent should have all the data?

I tried running the methods sequentially, this way it work fine but takes 30 to 40 seconds to return all data for a student.

I also tried running them parallel by splitting it into multiple tasks using below but most of the returned values are just null.

 Task.Run(() => mySubMethods );

P.S: I am using RestSharp and in each of my method which returns me student related data I am using ExecuteAsync. For example.

var client = new RestClient(APIURL);
RestRequest request = new RestRequest();
RestResponse response = client.ExecuteAsync(request).Result;

I appreciate any helpful approach.

CodePudding user response：

Actually, async calls won't help you much here. These will help you avoid that the parent process (e.g. the UI) doesn't freeze, but won't improve the speed of the whole call. You could split request and await, but that would increase the complexity of the code. I think the easiest thing here would be to use the Parallel class instead.

Something along the lines of:

object updateLock = new object();
objStudent.CoursesIDs = Helpers.GetStudentCourses();
Parallel.Foreach(objStudent.CoursesIDs, courseID => 
    {
        string courseName = Helpers.GetCourseName(courseID); 
        lock (updateLock)
        {
            objStudent.CourseName.Add(new Courses { ID:courseID, Title:courseName  } );
        }
   });

This queries all the courses in parallel. Overall this should be faster (assuming the server is capable of answering all requests simultaneously). Note the required lock, because the CourseName array now needs to be thread safe. This shouldn't have a performance impact since adding an entry to a list is several orders of magnitude faster than the query. Note that the CourseName list will be in random order now.

CodePudding user response：

First step: get rid of all .Result calls. Everywhere that you have a .Result replace it with await, add the async keyword in the method definition, and change the return type form X to Task<X>. You can also (optionally) append the prefix Async to the name of the method, to signify that the method is asynchronous. For example:

async Task<string> GetCourseNameAsync(int id)
{
    //...
    var client = new RestClient(url);
    RestRequest request = new RestRequest();
    RestResponse response = await client.ExecuteAsync(request);
    //...
    return courseName;
}

Second step: parallelize the GetCourseNameAsync method for each student by using the Parallel.ForEachAsync method (available from .NET 6 and later), configured with an appropriate MaxDegreeOfParallelism:

ParallelOptions options = new() { MaxDegreeOfParallelism = 4 };

Parallel.ForEachAsync(objStudent.CoursesIDs, options, async (courseID, ct) =>
{
    string courseName = await Helpers.GetCourseNameAsync(courseID);
    lock (objStudent.CourseName)
    {
        objStudent.CourseName.Add(new Courses { ID:courseID, Title:courseName });
    }
}).Wait();

The courseNames will be added in the objStudent.CourseName collection is the order of completion of the asynchronous operations, not in the order of the ids in originating list objStudent.CoursesIDs. In case this is a problem, you can find solutions at the bottom of this answer.

In the above example the Parallel.ForEachAsync is Waited synchronously, because the container method GetStudentDetails is not asynchronous. So it violates the async-all-the-way principle. This might not be a problem, but it is something that you should have in mind if you care about improving your application, and making it as efficient as possible.

The MaxDegreeOfParallelism = 4 is a random configuration. It's up to you to find the optimal value for this setting, by experimenting with your API.