how can I use multiprocessing in a django management command? I can't figure out how to start the process inside of my websites django context
management command
users = users.objects.filter(is_active=True)
with ProcessPoolExecutor(max_workers=5) as executor:
processes = []
for user in users:
p = executor.submit(complex_user_calculation,user)
processes.append(p)
for f in concurrent.futures.as_completed(processes):
result = f.result()
if result:
print(result)
when I run this management command I get this error
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
...
File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\concurrent\futures\_base.py", line 390, in __get_result
raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.
CodePudding user response:
You need to initialise your Django application in each subprocess to ensure your apps, models, etc are all loaded. Pass django.setup
as your pool initialiser
with ProcessPoolExecutor(max_workers=5, initializer=django.setup) as executor:
...