Tagging tasks in asyncio requests used with gather-CodePudding

I have a get_tasks function below:

def get_tasks(session):
    tasks = []
    topics, params = get_params()
    url = "http://test_site/scrolls"
    for topic, param in zip(topics, params):
        tasks.append(asyncio.create_task(session.post(url, json=param)))

    return tasks

Where I then call the function below to gather tasks and get the responses I need.

async def get_topic_urls():
    results = []
    async with aiohttp.ClientSession() as session:
        tasks = get_tasks(session)
        responses = await asyncio.gather(*tasks)
        for response in responses:
            #await asyncio.sleep(0)
            res = await response.json()
            if response.ok:
                results.append(res)
            else:
                raise res.raise_for_status()

        return results

get_params() in the get_tasks function returns a list of topics (e.g. [cars, food, soccer, gold, movies]), and its corresponding params to post to the API.

The main problem I have is tracking the json responses that belong to each topic. Since the tasks are being run asynchronously (hence not really in order), I can't really tell which topic the json responses belong to. Is there any way to tag which topic the response being returned belongs to? It was straight forward tagging the responses when I used requests instead of asyncio, but there are a lot of topics that needs to be run.

CodePudding user response：

Simply return topics from get_tasks as well.

def get_tasks(session):
    tasks = []
    topics, params = get_params()
    # ... as before ...
    return topics, tasks

responses returned by asyncio.gather will be in the same order as the tasks,

If all awaitables are completed successfully, the result is an aggregate list of returned values. The order of result values corresponds to the order of awaitables in aws.

so by extension will be in the same order as topics and you can associate each result with a topic:

async def get_topic_urls():
    results = []
    async with aiohttp.ClientSession() as session:
        topics, tasks = get_tasks(session)
        responses = await asyncio.gather(*tasks)
        for topic, response in zip(topics, responses):
            # ...