Polly does not retry even though an exception is caught-CodePudding

I have the following configuration of my HTTP service:

services
   .AddHttpClient<ISomeHttpService, SomeHttpService>((Action<HttpClient>) (client =>
      {
        client.BaseAddress = new Uri(configuration.BaseAddress);
        client.Timeout = TimeSpan.FromSeconds(10);
      }))
  .AddPolicyHandler(GetPollyPolicy())
  .AddHttpMessageHandler<DistributedCachedHttpClientHandler>();

// .....


private static IAsyncPolicy<HttpResponseMessage> GetPollyPolicy()
{
    return HttpPolicyExtensions
        .HandleTransientHttpError()
        .OrInner<Exception>() // I will probably use TimeoutException, for now I'm checking all possible options
        .WaitAndRetryAsync(2, _ => TimeSpan.FromMilliseconds(100), 
        (exception, timeSpan) =>
        {
           _ = "breakpoint";
        });
}

It works really well, if the request made by SomeHttpService returns 500. Polly tries to retry 2 more times.

However, if the request takes more time than 10 seconds (which is more than the configured timeout), retries do not occur. I even set a breakpoint within the Polly's action and I see that the breakpoint is hit when the timeout occurs! However, when I continue the program, no retries are executed and my web API fails with 500 directly after.

What's wrong?

I also tried using Or<Exception>() instead of OrInner<Exception>().

CodePudding user response：

TL;DR: The observed behaviour is the way how HttpClient's Timeout overarches all retry attempts.

The root cause of your problem is that HttpClient's Timeout is considered as global rather than local timeout:

Global means overarching, includes all the retry attempts
Local means per retry attempt

If you want to define a local timeout then you have to combine Retry and Timeout policies:

var timeoutPolicy = Policy.TimeoutAsync<HttpResponseMessage>(5);
var retryPolicy = HttpPolicyExtensions
    .HandleTransientHttpError()
    .OrInner<TimeoutRejectedException>()
    .WaitAndRetryAsync(2, _ => TimeSpan.FromMilliseconds(10));

var combinedPolicy = Policy.WrapAsync(retryPolicy, timeoutPolicy);

Please note that the order of the parameters of the Policy.WrapAsync matters.

The above example defines a per request timeout policy (inner) and a retry policy (outer), which is aware of the potential timeout (TimeoutRejectedException).

You can define two timeout policies, where one acts locally whereas the other acts globally:

var combinedPolicy = Policy.WrapAsync(globalTimeout, retryPolicy, localTimeout);