How to add 2 minutes delay between jobs in a queue?-CodePudding

I am using Hangfire in ASP.NET Core with a server that has 20 workers, which means 20 jobs can be enqueued at the same time.

What I need is to enqueue them one by one with 2 minutes delay between each one and another. Each job can take 1-45 minutes, but I don't have a problem running jobs concurrently, but I do have a problem starting 20 jobs at the same time. That's why changing the worker count to 1 is not practical for me (this will slow the process a lot).

The idea is that I just don't want 2 jobs to run at the same second since this may make some conflicts in my logic, but if the second job started 2 minutes after the first one, then I am good.

How can I achieve that?

CodePudding user response：

You can use BackgroundJob.Schedule() to run your job run at a specific time:

BackgroundJob.Schedule(() => Console.WriteLine("Hello"), dateTimeToExecute);

Based on that set a date for the first job to execute, and then increase this date to 2 minutes for each new job.

Something like this:

var dateStartDate = DateTime.Now;
foreach (var j in listOfjobsToExecute)
{
    BackgroundJob.Schedule(() => j.Run(), dateStartDate);
    dateStartDate = dateStartDate.AddMinutes(2);
}

See more here: https://docs.hangfire.io/en/latest/background-methods/calling-methods-with-delay.html?highlight=delay

CodePudding user response：

One way to achieve this would be to use Hangfire's batching feature. With this feature, you can group your jobs into batches and set the maximum number of jobs that can be processed concurrently within each batch. This way, you can ensure that only a certain number of jobs are started at the same time, while still allowing for concurrent processing of individual jobs. Here :

var batch = new BatchOptions
{
    BatchSize = 20,
    MaxConcurrencyLevel = 1
};

Next, enqueue your jobs using the EnqueueInBatch method, specifying the batch you created in the previous step. This method also takes a delay parameter, which you can use to specify the delay between each job. For example:

// Enqueue the first job with no delay
BackgroundJob.EnqueueInBatch(() => ProcessJob(), batch);

// Enqueue the remaining jobs with a two-minute delay
for (int i = 1; i < 20; i  )
{
    BackgroundJob.EnqueueInBatch(() => ProcessJob(), batch, TimeSpan.FromMinutes(2));
}

This will enqueue your jobs in the specified batch, with a two-minute delay between each one. The maximum concurrency level for the batch will ensure that only one job is started at a time, while still allowing for concurrent processing of individual jobs.