I'm using Rails 6 in my app with Sidekiq on board. I've got FetchAllProductsWorker like below:

module Imports
  class FetchAllProductsWorker
    include Sidekiq::Worker
    sidekiq_options queue: 'imports_fetch_all', retry: 0

    def perform
      (...)
    end
  end
end

I want to check when FetchAllProductsWorker was finished successfully last time and display this info in my front-end. This job will be fired sporadically but the user must have feedback when the last database sync (FetchAllProductsWorker is responsible for that) succeeded.

I want to have such info only for this one worker. I saw a lot of useful things inside the sidekiq API docs but none of them relate to the history of completed jobs.

CodePudding user response：

You could use the Sidekiq Batches API that provides an on_success callback but that is mostly used for tracking batch work which is an overkill for your problem. I suggest writing your code at the end of the perform function.

def perform
  (...) # Run the already implemented code
  (notify/log for success) # If it is successful notify/log.
end

CodePudding user response：

The simplified default lifecycle of a Sidekiq looks like this:

• If there is an error then the job will be retried a couple of times (read about Retries in the Sidekiq docs). During that time you can see the failing job and the error on the Sidekiq Web UI if configured. – If the job is finished successfully the job is removed from Redis and there is no information about this specific job available to the application.

That means Sidekiq does not really support running queries about jobs that run successfully in the past. If you need information about past jobs then you need to build this on its own. I basically see three options to allow monitoring Sidekiq Jobs:

1. Write useful information to your application's log. Most logging tools support monitoring for specific messages, sending messages, or creating views for specific events. This might be enough if you just need this information for debugging reasons.

   def perform
     Rails.logger.info("#{self.class.name}" started)
     begin
       # job code
     rescue => exception
       Rails.logger.error("#{self.class.name} failed: #{exception.message}")
       raise # re-raise the exception to trigger Sidekiq's default retry behavior
     else
       Rails.logger.info("#{self.class.name}" was finished successfully)  
     end
   end
   

2. If you are mostly interested in getting informed when there is a problem then I suggest looking at a tool like Dead Man's Snitch. how those tools are working is that you ping their API as the last step of a job which will only reach when there was no error. Then configure that tool to notify you if its API hasn't been pinged in the expected timeframe, for example, if you have a daily import job, then Dead Man's Snitch would send you a message only if there wasn't a successful import Job in the last 24 hours. If the job was successful it will not spam you every single day.

   require 'open-uri'
   def perform
     # job code
   
     open("https://nosnch.in/#{TOKEN}")
   end
   

3. If you want to allow your application's users to see job return statuses on a dashboard in the application. Then it makes sense to store that information in the database. You could, for example, just create a JobStatus ActiveRecord model with columns like job_name, status, payload, and a created_at and then create records in that table whenever it feels useful. Once the data is in the database you can present that data like every other model's data to the user.

   def perform
     begin
       # job code
     rescue => exception
       JobStatus.create(job_name: self.class.name, status: 'failed', payload: exception.to_json)            
       raise # re-raise the exception to trigger Sidekiq's default retry behavior
     else
       JobStatus.create(job_name: self.class.name, status: 'success')            
     end
   end
   

And, of course, you can combine all those technics and tools for different use-cases. 1. for history and statistics, 2. for admins and people being on-call, 3. for users of your application.