Problem

It's very common for beginners to solve IO waiting while concurrent processing in an similar way like here:

#!/usr/bin/env python3

"""Loop example."""

from time import sleep

WAITING: bool = True
COUNTER: int = 10


def process() -> None:
    """Non-blocking routine, that needs to be invoked periodically."""
    global COUNTER  # pylint: disable=global-statement
    print(f"Done in {COUNTER}.")
    COUNTER -= 1
    sleep(1)
    # Mimicking incoming IO callback
    if COUNTER <= 0:
        event()


def event() -> None:
    """Incoming IO callback routine."""
    global WAITING  # pylint: disable=global-statement
    WAITING = False


try:
    while WAITING:
        process()
except KeyboardInterrupt:
    print("Canceled.")

Possible applications might be servers, what are listening for incomming messages, while still processing some other internal stuff.

Possible Solution 1

Threading might in some cases a good solution. But after some research it seems that threading adds a lot of overheading for the communcation between the threads. One example for this might be the 'Warning' in the osc4py3 package documentation below the headline 'No thread'. Also i have read somewhere the thumb rule, that 'Threading suits not for slow IO' (sorry, lost the source of this rule).

Possible Solution 2

Asynchronous processing (with the asyncio package) might be another solution. Especially because the ominous thumb rule also says that 'For slow IO is asyncio efficient'.

What i tried

So i tried to rewrite this example with asyncio but failed completely, even after reading about Tasks, Futures and Awaitables in general in the Python asyncio documentation. My problem was to solve the perodically (instead of one time) call while waiting. Of course there are infinite loops possible, but all examples i found in the internet are still using 'While-True'-Loops what does not look like an improvement to me. For example this snippet:

import asyncio

async def work():
    while True:
        await asyncio.sleep(1)
        print("Task Executed")

loop = asyncio.get_event_loop()
try:
    asyncio.ensure_future(work())
    loop.run_forever()
except KeyboardInterrupt:
    pass
finally:
    print("Closing Loop")
    loop.close()

Source: https://tutorialedge.net/python/concurrency/asyncio-event-loops-tutorial/#the-run_forever-method

What i want

To know the most elegant and efficient way of rewriting these stupid general 'While-True'-Loop from my first example code. If my 'While-True'-Loop is still the best way to solve it (beside my global variables), then it's also okay to me. I just want to improve my code, if possible.

CodePudding user response：

Solution with asyncio:

#!/usr/bin/env python3

"""Asyncronous loop example."""

from typing import Callable
from asyncio import Event, get_event_loop


DONE = Event()


def callback():
    """Incoming IO callback routine."""
    DONE.set()


def process():
    """Non-blocking routine, that needs to be invoked periodically."""
    print('Test.')


try:
    loop = get_event_loop()
    run: Callable = lambda loop, processing: (
        processing(),
        loop.call_soon(run, loop, processing)
    )
    loop.call_soon(run, loop, process)
    loop.call_later(1, callback)  # Mimicking incoming IO callback after 1 sec
    loop.run_until_complete(DONE.wait())
except KeyboardInterrupt:
    print("Canceled.")
finally:
    loop.close()
    print("Bye.")

CodePudding user response：

What you describe is some kind of polling operation and is similar to busy waiting. You should rarely rely on those methods as they can incur a serious performance penalty if used incorrectly. Instead, you should rely on concurrency primitives provided by the OS of a concurrency library.

As said in a comment, you could rely on a condition or an event (and more broadly on mutexes) to schedule some come to run after an event occurs. For I/O operations you can also rely on low-level OS facilities such as select, poll and signals/interruptions.

Possible applications might be servers, what are listening for incomming messages, while still processing some other internal stuff.

For such use cases you should really use a dedicated library to do that efficiently. For instance, here is an example of a minimal server developed with AsyncIO's low-level socket operations. Internally, AsyncIO probably uses the select system call and exposes a friendly interface with async-await.