How to run multiple asyncio loops inside syncrhonous sub-processes inside a main asyncio loop?-CodePudding

I have a main function which I run with asyncio and then inside it I used event_loop.run_in_executor() to run mutliple processes of a blocking function.

What I wish to do is inside each of these processes to run a new asyncio loop for each one in order to execute async code.

So I have a main async function where I create multiple processes and I want to create a new asyncio loop in each of them. How can this be done?

CodePudding user response：

You can just call asyncio.run within your subprocess with a coroutine with the async work you want to do. A minimal example that spins up 5 processes each with their own event loops:

import asyncio
from concurrent.futures import ProcessPoolExecutor


def run_loop_in_process():
    async def subprocess_async_work():
        await asyncio.sleep(5) #whatever async code you need
    asyncio.run(subprocess_async_work())

async def main():
    loop = asyncio.get_running_loop()
    pool = ProcessPoolExecutor()
    tasks = [loop.run_in_executor(pool, run_loop_in_process) for _ in range(5)]
    await asyncio.gather(*tasks)

if __name__ == "__main__":
    asyncio.run(main())