Home > Software design >  How to combine a dynamic amount (150-200) of async functions for calculation purposes
How to combine a dynamic amount (150-200) of async functions for calculation purposes

Time:10-07

Currently working on a project that uses Binance Websockets to fetch data of the coin prices and makes calculations with the data.

The number of coins that I calculate is dynamic (varies from 150 to ~250).

The ASYNC function is called calculate(client, queue, coin)

It takes the following arguments:

  1. client: The Binance Async client (python-binance library)
  2. queue: A queue where opportunities get passed in if the calculation has a positive outcome
  3. coin: The coin e.g. BTC

The calculate function contains a while 1 (infinitive) loop, so the calculations keep on running.

At the moment the main function uses asyncio.gather to combine the calculate functions for the different coins (150 - ~250)

async def main(queue):

await asyncio.gather(
    calculate(client, queue, 'BTC'),
    calculate(client, queue, 'ETH'),
    calculate(client, queue, 'BNB'),
    calculate(client, queue, 'LINK'),
    calculate(client, queue, 'SHIB'),
    calculate(client, queue, 'ADA'),
    calculate(client, queue, 'XRP'),
    calculate(client, queue, 'VET')
    # This continues for the the other  100 coins
    )


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main(queue))

This is not a pretty way of writing the code and it is very inefficient if I want to change the coins. Using an API I fetch the coins that need to be calculated. So I need to manually update the 100 lines which call the function for each coin and update some other files to ensure that everything runs smooth.

What would be a good solution for this issue?

I've seen threading/ multithreading / more complex uses of asyncio, but I couldn't see how to apply it to this issue.

Help would be much appreciated.

CodePudding user response:

You want to create a list of the coins to call calculate() on, use that list to create a list of calculations, and then use a starred expression unpack that list as arguments to asyncio.gather():

async def main(queue):
    coins = ['BTC', 'ETH', 'BNB']  # Add more coins as desired.
    calculations = [calculate(client, queue, coin) for coin in coins]

    await asyncio.gather(*calculations)  # Use a starred expression to unpack the list.


if __name__ == "__main__":
    loop = asyncio.get_event_loop()
    loop.run_until_complete(main(queue))
  • Related