Home > Enterprise >  django how can I use multiprocessing in a management command
django how can I use multiprocessing in a management command

Time:08-10

how can I use multiprocessing in a django management command? I can't figure out how to start the process inside of my websites django context

management command

users = users.objects.filter(is_active=True)

with ProcessPoolExecutor(max_workers=5) as executor:
   processes = []
   for user in users:
       p = executor.submit(complex_user_calculation,user)
       processes.append(p)

   for f in concurrent.futures.as_completed(processes):
       result = f.result()
       if result:
           print(result)

when I run this management command I get this error

    raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
...
  File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 390, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

CodePudding user response:

You need to initialise your Django application in each subprocess to ensure your apps, models, etc are all loaded. Pass django.setup as your pool initialiser

with ProcessPoolExecutor(max_workers=5, initializer=django.setup) as executor:
    ...
  • Related