asyncio.as_completed() supposedly accepting `Iterable`, but crashes if input is `Generator`?-CodePudding

So, essentially, in Python 3.7 (as far as I know) if you try to do this,

import asyncio


async def sleep():
    asyncio.sleep(1)


async def main():
    tasks = (sleep() for _ in range(5))
    for task in asyncio.as_completed(tasks):
        result = await task


if __name__ == "__main__":
    asyncio.run(main())

It crashes with

TypeError: expect a list of futures, not generator

But the type hints clearly specify that it accepts an Iterable, which a Generator is.

If you turn tasks into a list, it works, of course, but... what am I missing?

And why would it be subjected to lists? I don't see why it should not allow generators.

CodePudding user response：

You are right. The documentation here is not consistent with the actual behavior.

The official documentation refers to the first argument as an "iterable". And typeshed as of today also annotates the first argument with Iterable[...].

However, in the CPython code for as_completed the first argument is passed to coroutines.iscoroutine, which checks, if it is an instance of types.GeneratorType. Obviously, that is what it is, which has it return True and cause the TypeError.

And of course a generator is also an iterable. Which means the function does not in fact accept an iterable as the docs claim, but only a non-generator iterable.

Maybe someone else here can shine additional light on the background or thought process here. In any case, I would argue this is worth opening an issue over, if one addressing this does not exist yet.

EDIT: Apparently (and unsurprisingly) we were not the first ones to notice this. Thanks to @KellyBundy for pointing it out.