celery beat - how to run multiple tasks for list of users?-CodePudding

I have three tasks those need to be executed for 100s of users. Eaxample users lsit:

['user1', 'user2', 'user3','user4', 'user5', 'user6']

Here is the code snippet for celery beat:

for user in users:
    APP.conf.beat_schedule = {
        "trigger-sync_books": {
            "task": "call_sync_books",
            "schedule": crontab(minute=0, hour="*/24"),
            'args': (user, bo, gg,),
        },
        "trigger-sync_stock": {
            "task": "call_sync_stock",
            "schedule": crontab(minute="*/10"),
            'args': (user, bo, gg,),
        },
        "trigger-sync_customer": {
            "task": "call_sync_customer",
            "schedule": crontab(minute=0, hour="*/24"),
            'args': (user, bo, gg,),
        },
    }

But everytime the celery beat sends these three tasks for the last user from the list. i.e., for user6 from the example list.

How can I make sure these three tasks will be excuted for all the users?

CodePudding user response：

This will not work. You are redefining APP.conf.beat_schedule as many times as you have users, so basically all previous 5 configs get overwritten by the last one in that loop.

Try the following:

for user in users:
    us_sched = {
        f"trigger-sync_books-{user}": {
            "task": "call_sync_books",
            "schedule": crontab(minute=0, hour="*/24"),
            'args': (user, bo, gg,),
        },
        f"trigger-sync_stock-{user}": {
            "task": "call_sync_stock",
            "schedule": crontab(minute="*/10"),
            'args': (user, bo, gg,),
        },
        f"trigger-sync_customer-{user}": {
            "task": "call_sync_customer",
            "schedule": crontab(minute=0, hour="*/24"),
            'args': (user, bo, gg,),
        }
    }
    # Python >= 3.5:
    APP.conf.beat_schedule = {**APP.conf.beat_schedule, **us_sched}
    # Python 3.9: APP.conf.beat_schedule |= us_sched

By the time the loop above ends you wil have all schedules for all users in the APP.conf.beat_schedule dictionary.