I want to execute some code to every user in my database. For this I created a script and run it with rails runner myscript.rb

User.all.each do |user|
  # code on user
end

If I have 500 user in my database this works but if I have for example 4000 users I get some errors given some of the code inside the .each given the amount of users. (This numbers are fictional)

For this reason I want to execute this script in intervals of 500 users.

How can I execute every 500 users until there are no more users?

CodePudding user response：

Use find_each:

Looping through a collection of records from the database (using the Scoping::Named::ClassMethods.all method, for example) is very inefficient since it will try to instantiate all the objects at once.

In that case, batch processing methods allow you to work with the records in batches, thereby greatly reducing memory consumption.

You want to say:

User.find_each do |user|
  # code on user
end

That will load the users 1000 at a time and hand them one by one to your "code on user" block. You can specify different chunk sizes if you'd like.

CodePudding user response：

You can do it under while do loop and the user object can be set to User.all.sample. By doing this, until there are no users left, your script will work then quit. But you should add a flag to your table.

Let's imagine that you are trying to send e-mail to all users, and you should have a field to use as flag similar to email_sent.

And now your user object is User.all.where(email_sent: true).sample

Another aproach may be using querying jobs by user. Create a loop thorugh your users and have the index of the loop. Set indexamount_of_time seconds to start the query. So in case you have 1000 users and you set index2 seconds of buffer in between users. It will take 2000 secs. to handle all jobs queried.

There are another solutions. If you can better explain what you are looking for and what practise you need; I can suggest you a better one.