NullReferenceException Error while using Polly package for retries-CodePudding

We have .net 6 service where we are using polly package (Polly, Version=7.0.0.0) for retrying transient api calls. We are seeing System.NullReferenceException: Object reference not set to an instance of an object error a lot. Though, the retry seems to be working, I think we are missing something from the code.

we are able to see logs for retries:

NotExtended delaying for 1000 ms, then making retry 1 {...data..}
NotExtended delaying for 1000 ms, then making retry 2 {...data..}
NotExtended delaying for 1000 ms, then making retry 3 {...data..}

Since I am new to how Polly works, not sure what exactly we are missing from the below code. any suggestions would be much helpful. Thanks in advance.

P.S.: I am pasting snippet of code where we use polly package but not the lines of code NOT related to polly implementation to avoid confusion.

Not sure if this error is occurring for few api calls or for all of them.

using Polly;
using Polly.Extensions.Http;
using Polly.Wrap;

services.AddHttpClient<IApiService, ApiService>()
        .AddPolicyHandler(GetPollyRetryPolicy(builder.services));


private static AsyncPolicyWrap<HttpResponseMessage> GetPollyRetryPolicy(IServiceCollection services)
{
    var asyncPolicy = HttpPolicyExtensions
                        .HandleTransientHttpError()
                        .WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(retryAttempt),
                            onRetryAsync: async (outcome, timespan, retryAttempt, context) => 
                            {
                                var serviceProvider = services.BuildServiceProvider();
                                var data = new                                                      
                                {
                                    url = outcome.Result?.RequestMessage?.RequestUri?.ToString(),
                                    method = outcome.Result?.RequestMessage?.Method?.Method,
                                    resultContent = await outcome.Result?.Content?.ReadAsStringAsync()
                                };
                                serviceProvider.GetService<ILogger>()
                                ?.Warn($"{outcome.Result.StatusCode} delaying for {timespan.TotalMilliseconds} ms, "  
                                   $"then making retry {retryAttempt}", data: data, ex: outcome.Exception);
                            });

    return Policy.TimeoutAsync(60).WrapAsync(asyncPolicy);
}

Error we are receiving is below:

System.NullReferenceException: Object reference not set to an instance of an object.

--- End of stack trace from previous location ---
   at Polly.Retry.AsyncRetryEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates`1 shouldRetryResultPredicates, Func`5 onRetryAsync, Int32 permittedRetryCount, IEnumerable`1 sleepDurationsEnumerable, Func`4 sleepDurationProvider, Boolean continueOnCapturedContext)
   at Polly.AsyncPolicy`1.ExecuteAsync(Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
   at Polly.Wrap.AsyncPolicyWrapEngine.<>c__DisplayClass2_0`1.<<ImplementationAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Polly.Timeout.AsyncTimeoutEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Func`2 timeoutProvider, TimeoutStrategy timeoutStrategy, Func`5 onTimeoutAsync, Boolean continueOnCapturedContext)
   at Polly.Timeout.AsyncTimeoutEngine.ImplementationAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Func`2 timeoutProvider, TimeoutStrategy timeoutStrategy, Func`5 onTimeoutAsync, Boolean continueOnCapturedContext)
   at Polly.AsyncPolicy.ExecuteAsync[TResult](Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
   at Polly.Wrap.AsyncPolicyWrapEngine.ImplementationAsync[TResult](Func`3 func, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext, IAsyncPolicy outerPolicy, IAsyncPolicy`1 innerPolicy)
   at Polly.AsyncPolicy`1.ExecuteAsync(Func`3 action, Context context, CancellationToken cancellationToken, Boolean continueOnCapturedContext)
   at Microsoft.Extensions.Http.PolicyHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Extensions.Http.Logging.LoggingScopeHttpMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
   at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)

CodePudding user response：

As Stephen Cleary has mentioned it in his comment: Building a whole new service provider within each retry is very odd. It is not just odd, but not necessary at all.

The AddPolicyHandler has an overload which provides access to the IServiceProvider:

services.AddHttpClient<IApiService, ApiService>()
        .AddPolicyHandler((provider, _) => GetPollyRetryPolicy(provider))

With this technique in our hand you can simply ask for an ILogger like this inside the onRetryAsync delegate:

var logger = provider.GetRequiredService<ILogger>();

I would also suggest to prefer the static Wrap method over the instance method:

Policy.WrapAsync(Policy.TimeoutAsync(60), asyncPolicy);

Here I have detailed the differences between the alternatives.

Please also note that the ordering matters:

Timeout outer, Retry inner: Timeout act a global, over-aching constraint
Retry outer, Timeout inner: Timeout act a local constraint, so it resets itself for each retry attempt