How to handle broken connection during active HTTP client operation?-CodePudding

This is C# code example that downloads big file (1Gb) and saves it as "download.bin" file. Operation timeout is setup to 20 seconds.

async Task RunHttpDownload()
{
    var url = "https://speed.hetzner.de/1GB.bin";

    Console.WriteLine($"Start download {DateTime.Now}");

    try
    {
        using (var httpClient = new HttpClient() { Timeout = TimeSpan.FromSeconds(20) })
        using (var httpResp = await httpClient.GetAsync(url, HttpCompletionOption.ResponseHeadersRead))
        {
            httpResp.EnsureSuccessStatusCode();

            using (var fileStream = new FileStream("download.bin", FileMode.OpenOrCreate, FileAccess.Write, FileShare.Read, bufferSize: 4096, useAsync: true))
            using (var httpStream = await httpResp.Content.ReadAsStreamAsync())
            {
                await httpStream.CopyToAsync(fileStream);
            }
        }
    }
    catch (Exception e)
    {
        Console.WriteLine($"Exception: {e.Message}");
    }

    Console.WriteLine($"End download {DateTime.Now}");
}

HTTP Get method completion is HttpCompletionOption.ResponseHeadersRead to avoid loading entire 1Gb input file in memory. If user lost Internet connection (WiFi is switched off) at line await httpStream.CopyToAsync(fileStream); (method spends 99% of time here), operation will never finished! even if timeout is 20 seconds! For some reason C# HTTP client can detect case when connection is lost (broken) and throw some exception about it.

How to fix it?

CodePudding user response：

For some reason C# HTTP client can[not] detect case when connection is lost (broken) and throw some exception about it.

The reason is actually the HTTP protocol. It was designed a long time ago before people knew much about protocol design, and it forces a one-way communication from client to server followed by one-way communication from server to client. The problem with one-way communication over TCP/IP is that the receiving end has no notification for when the connection is dropped. In the future, this will be less of an issue, since I believe HTTP/QUIC has proper heartbeat support.

For now, HTTP clients and servers work around this problem by using timers, which is the only reliable approach for a protocol you can't change, as I explain on my blog. Servers give clients a certain amount of time to finish sending their request. Client support for timeouts is a bit different; most clients use one timeout value for the whole round trip time. For common REST requests (which have relatively small request and response bodies), this works well enough that most developers aren't even aware there's a timeout to work around a limitation in the HTTP protocol.

So, let's talk solutions. The reason your existing timeout isn't working is because HttpClient.Timeout only applies to the time spent in SendAsync. Due to the ResponseHeadersRead, SendAsync does not cover the entire response - just until the headers are read. So the HttpClient.Timeout is just until the response body starts as opposed to when it ends.

There's nothing in HttpClient that provides timeouts on reading the response body if you use ResponseHeadersRead. So you'll have to use your own timeouts. Here's one that gives the body 20 seconds to complete:

using (var fileStream = new FileStream("download.bin", FileMode.OpenOrCreate, FileAccess.Write, FileShare.Read, bufferSize: 4096, useAsync: true))
using (var httpStream = await httpResp.Content.ReadAsStreamAsync())
using (var cts = new CancellationTokenSource(TimeSpan.FromSeconds(20)))
{
  await httpStream.CopyToAsync(fileStream, cts.Token);
}