Error handling in .NET 6 BackgroundServices hosted in a Windows Service-CodePudding

I've been reading through the MS documentation regarding the deployment of a Windows service to run a Worker App.

The MS code sample talks about the need to add an Environment.Exit(1) inside the exception handler so that the Windows Service Management can leverage the configured recovery options.

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    try
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            string joke = _jokeService.GetJoke();
            _logger.LogWarning("{Joke}", joke);

            await Task.Delay(TimeSpan.FromMinutes(1), stoppingToken);
        }
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "{Message}", ex.Message);

        // Terminates this process and returns an exit code to the operating system.
        // This is required to avoid the 'BackgroundServiceExceptionBehavior', which
        // performs one of two scenarios:
        // 1. When set to "Ignore": will do nothing at all, errors cause zombie services.
        // 2. When set to "StopHost": will cleanly stop the host, and log errors.
        //
        // In order for the Windows Service Management system to leverage configured
        // recovery options, we need to terminate the process with a non-zero exit code.
        Environment.Exit(1);
    }
}

There are a few concepts that are not clear to me and hoping someone can advise:

In my own project, my background services include various classes and operations such as connection managemenet for Azure IOT Hubs Device Client. In some cases, i simply do no want to force the environment i.e. the whole app to exit on every catch/exception scenario, but the docs are not clear on whether we are expceted to do this? I mean why catch an exception if we're going to simply wipe out the running of the application every time? doesnt make sense to me...

The next point ref the following statement "To correctly allow the service to be restarted, you can call Environment.Exit with a non-zero exit code" but then earlier in the article, it also talks about the two options available for 'BackgroundServiceExceptionBehavior':

Ignore - Ignore exceptions thrown in BackgroundService. StopHost
The IHost will be stopped when an unhandled exception is thrown.

An unhandled exception in my mind means the app has gone fowl on something that hasnt been appropriately caught in the right place i.e. where no try/catch block exists. So how does one provision an 'Environment.Exit(1)' to something they havent yet accounted for? And what happens in thie scenario?

The way the article reads to me suggests that the only way we can ensure the Windows Service will manage the re-starting of the app succesfully is from any exception that we knowingly caught, but that equally that doesnt tie up with what the general article is suggesting will happen.

Totally confused :(

CodePudding user response：

As described in the article pre .NET 6 an unhandled exception in background service has not affected application in any way - for example if you had an app which only job was to process some queue in background service and this service would fail then the app has continued running as if nothing happened, which obviously is wrong in such cases. That's why it was fixed in .NET 6.

As service recovery options and .NET BackgroundService instances paragraph states new default behavior for unhandled exceptions is StopHost which behaves like as if application (Windows service) would exit normally (with exit code 0):

But it stops cleanly, meaning that the Windows Service management system will not restart the service

If you want your app to be restarted automatically by the Windows Service management system (if it is setup to do so) you need to "handle" the exception and end the application with non-zero exit code which indicates failure.

Obviously if some concrete exception is recoverable you need to just recover:

while (!stoppingToken.IsCancellationRequested)
{
    try
    {
        // some job
    }
    catch (SomeRecoverableException e)
    {
        // handle it
    }
    catch (Exception ex)
    {
        _logger.LogError(ex, "{Message}", ex.Message);
        Environment.Exit(1);
    }
}

The Ignore option exists for backward compatibility so it's possible to revert to the previous behaviour (can't think off the top of my head why it is possibly can be needed but still).

CodePudding user response：

tl;dr:

From .NET Core 2 up through .NET 5 (or rather, .NET Platform Extensions 2 through 5), a BackgroundService throwing an exception kept your host application running, but without that particular background service (but with any other tasks it was running, if any).
Starting with .NET (Platform Extensions) 6, a BackgroundService throwing an exception will bring down the host application, with an exit code of 0.
An exit code of 0 will not trigger the Service Control Manager's failure options, i.e. Windows won't know your service crashed, and won't restart it.
Ergo: you'll have to tell the SCM yourself that the service crashed, and exiting with a nonzero exit code is one way to do so.

A .NET BackgroundService is a platform-agnostic construct to represent a long-running tasks in a .NET application through the IHostedService mechanism (services.AddHostedService()).

You can have a hosted service in an ASP.NET application, as well as in a WinForms application or a plain old console application.

These hosted background services have no connection to the Windows-specific thing that are Windows Services.

You can have a Windows Service that does absolutely nothing, or one that runs a custom task of yours, or one that runs one or more BackgroundServices (or other IHostedServices), or a combination of the latter. You can even have one application that hosts multiple Windows Services, each running zero or more IHostedServices, but let's ignore that.

Now what do you want your service to do when it starts with zero hosted services? It should probably continue doing whatever else it was supposed to do. What if the Windows Service runs multiple BackgroundServices and one stops? Probably continue. What if one throws an exception?

That's probably not good, and if your process does other things besides hosting that one now crashed service, you probably don't want those jobs to continue. So since the .NET Platform Extension 6 BackgroundService lifetime change, the runtime pulls the rug from under the entire application, so everything that your Windows Service was doing stops happening.

But that doesn't work entirely. Yes, it kills your application and logs an event or two (9, 10) to the Application log, but it doesn't set the exit code nor does it communicate back to the Service Control Manager (SCM) that the service failed.

It says right so in the comments in that code block, but it isn't made clear enough: that is a problem. When you don't set the exit code and don't report an error to the SCM, the SCM won't run the service's recovery actions. So your service will stay down until you reboot the machine (assuming the service starts automatically) or you start the service manually.

Setting the process exit code and exiting the application from your exception handler is one way to let the Service Control Manager know your service crashed, and that's what that code block from the docs does:

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    try
    {
        // Do your long-running work
    }
    catch (Exception ex)
    {
        // In order for the Windows Service Management system to leverage configured
        // recovery options, we need to terminate the process with a non-zero exit code.
        Environment.Exit(1);
    }
}

But then you'll have to do so in all your BackgroundServices, and there are more issues with the .NET Platform Extensions (which contain these Windows Service helper classes) involving error handing, so I've written a wrapper library to solve that: CodeCaster.WindowsServiceExtensions.

Using my library, you don't have to bother with this Windows Service-specific error handling (it's done for you):

public class MyCoolBackgroundService : WindowsServiceBackgroundService
{
    public MyCoolBackgroundService(
        ILogger<MyCoolBackgroundService> logger,
        IHostLifetime hostLifetime
    )
        : base(logger, hostLifetime)
    {
    }

    protected override async Task TryExecuteAsync(CancellationToken stoppingToken)
    {
        // Do your continuous or periodic background work.
        await SomeLongRunningTaskAsync();

        // This will report to the SCM that your service failed.
        throw new Exception("Foo");
    }
}