How to add tasks to asyncio at runtime?-CodePudding

Basically, my goal is to collect data and send it at runtime, and I want to do it asynchronously - one co-routine per message. I've tried to build a small example of that, but it doesn't exceed my expectations. I've assumed I need two endless loops and a queue for them to communicate with each other - one loop to get user input and another to listen to the queue and output data if queue has something.

Here is my example:

import asyncio


async def output(queue):
    while True:
        data = await queue.get()
        if data:
            await asyncio.sleep(5)
            print(data)

async def main():
    queue = asyncio.Queue()
    loop = asyncio.get_event_loop()
    loop.create_task(output(queue))
    while True:
        data = await loop.run_in_executor(None, input)
        await queue.put(data)


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.create_task(main())
    loop.run_forever()

Expected result is something like this (>>> means input and <<< is output):

>>> 1
--- 0.0001 seconds delay before I enter my next input 
>>> 2
--- 5 seconds delay
<<< 1
--- ~0.0001 seconds delay
<<< 2

But the actual result is something like this:

>>> 1
--- 0.0001 seconds delay before I enter my next input
>>> 2
--- 5 seconds delay
<<< 1
--- 5 seconds delay
<<< 2

How do I achieve that?

CodePudding user response：

You said that you wanted "one co-routine per message." So just do that - and then you have no need for the Queue. I added some time stamps to your print statements so you could see that the two 5-second waits do, in fact, run in parallel. I also replaced the code in your main() function with a simple call to the convenience function asyncio.run().

import time
import asyncio

async def output(data):
    await asyncio.sleep(5)
    print("<<<", data, time.ctime())

async def main():
    loop = asyncio.get_event_loop()
    while True:
        data = await loop.run_in_executor(None, input)
        print(">>>", data, time.ctime())
        asyncio.create_task(output(data))

if __name__ == "__main__":
    asyncio.run(main())
    
>>> 1 Tue Oct  5 21:44:18 2021
>>> 2 Tue Oct  5 21:44:18 2021
<<< 1 Tue Oct  5 21:44:23 2021
<<< 2 Tue Oct  5 21:44:23 2021