We have a Spring Batch application that runs millions of batches every month. The vast majority of the jobs are successful and only a handful of them fail and require attention.
We have no actual need of tracking and keep data on successful jobs. In order to keep the size of the database small, we don't want to keep all the successful jobs data.
Is there an "out-of-the-box" option in Spring Batch to:
- Automatically delete the job entries from the DB once a successful job is done
OR
- Create a "time-to-live" policy that will clean successful jobs after X time
We found this Tasklet to delete old records from Spring Batch but we are looking for something that is more "official".
CodePudding user response：
Spring Batch is not (and IMO should not be) concerned with this kind of database housekeeping tasks, because the archiving strategy and the retention policy can vary from case to case. That is why there is no official support for that, and I don't see how this can be an out-of-the-box feature from an API point of view in the framework itself (meaning where would the injection/extension point be defined for such a task?).
The Tasklet you referred to is the most common answer if you want to create a Spring Batch job for this task. You can also take a look at the script here if you want to do it without a Spring Batch job.
Please take a look at the appendix here for more details about the matter.
CodePudding user response：
Here's a variation of the spring-batch-toolkit RemoveSpringBatchHistory tasklet that deletes old spring batch jobs using chunk based processing...