Multithreading in C# increasing CPU usage-CodePudding

I am using multiple thread for invoking a function as below. At table level there is queue number. According to the queue

Number the MAX_Download_Thread will be set. So that much thread will be created and work it continuously. When I put MAX_Download_Thread as 4 it consuming 30% of CPU out of 8 processors. When make it as 10 it almost consuming 70%. Just want to know whether any better method to reduce this or this is normal.

protected override void OnStart(string[] args)
{
    for (int i = 1; i <= MAX_Download_THREAD; i  )
    {
        int j = i;
        var fileDownloadTread = new Thread(() =>
        {
            new Repo.FILE_DOWNLOAD().PIC_FILE_DOWNLOAD(j);
        });
        fileDownloadTread.Start();
    }
}

CodePudding user response：

The point of using multiple Threads is to achieve a higher utilization of multicore CPUs to complete a Task in a shorter amount of time. The higher usage is to be expected, nay desired.

If the work your threads are doing is I/O-Bound (e.g. writing large amounts of Data to a Disk) it might end up taking longer than with fewer threads as data can only be written to a disk in series.

CodePudding user response：

Using multiple threads does use more CPU. In this case though, that's simply wasted. IO operations like file operations and downloads are carried out by the OS itself, not application code. The application threads just wait for the OS to retrieve data over the network.

To execute multiple downloads concurrently you need asynchronous operations. One way to do this is to use Parallel.ForeachAsync. The following code will download files in parallel from a list and save them locally:

HttpClient _client=new HttpClient();

async Task DownloadUrl(Uri url,DirectoryInfo folder)
{
    var fileName=url.Segments.Last();
    var newPath=Path.Combine(folder.FullName,fileName);

    var stream=await _client.GetStreamAsync(url);
    using var localStream=File.Create(newPath);
    await stream.CopyToAsync(localStream);
}

async Task DonwloadConcurrently(IEnumerable<Uri> urls,DirectoryInfo folder)
{    
    await Parallel.ForEachAsync(urls,async (url,_)=>{
        await DownloadUrl(url,folder);
    });
}

This will executeDownloadUrl multiple times in parallel. The default number of concurrent tasks is equal to Environment.ProcessorCount but can be adjusted

ParallelOptions parallelOptions = new()
{
    MaxDegreeOfParallelism = 3
};

await Parallel.ForEachAsync(urls, parallelOptions, async (url,_) =>{
    await DownloadUrl(url,folder);
});