Regarding: https://github.com/gvasquez95/binance-trader/issues/8

I'm concerned that the actual "inner" await (in every single invocation of getPriceChange) isn't blocking asynchronous invocation of the about 1.5k calls this method has on the outer loop (getPriceChanges). As if it was blocking, then the loop would be serial processing.

From the performance view and timing, I'd say it is working as expected, as it takes just some seconds to complete the whole process but, I'd like a programming ACK of the assumptions, if possible, please.

Block codes for those that don't want to get into GitHub and, prefer the code more visible here:

Outer loop:

async function getPriceChanges (since, breakpoint) {
  const changes = []
  for (const [i, ticker] of tickers.entries()) {
    if (!ticker.includes('BIDR') && !ticker.includes('BIFI')) {
      changes.push(getPriceChange(ticker, since, breakpoint))
    }
  }
  const res = await Promise.all(changes)
  let symbol
  let maxChange = 0
  let slope = 0
  for (const ticker of res) {
    if (ticker.percentChange > maxChange && ticker.slope > Math.pow(10, -1 * NUM_DECIMALS)) {
      maxChange = ticker.percentChange
      symbol = ticker.symbol
      slope = ticker.slope
    }
  }
  return { symbol, maxChange: Math.round(100 * maxChange) / 100, slope: round(slope, NUM_DECIMALS) }
}

Inner getPriceChange function:

async function getPriceChange (symbol, since, breakpoint) {
  const params = {
    TableName: DYNAMODB_TABLE,
    ProjectionExpression: '#timestamp, price',
    KeyConditionExpression: 'symbol = :symbol and #timestamp > :timestamp',
    ExpressionAttributeNames: {
      '#timestamp': 'timestamp'
    },
    ExpressionAttributeValues: {
      ':symbol': symbol,
      ':timestamp': since
    }
  }
  const res = await documentClient.query(params).promise()
  const prev = []
  const recent = []
  const trendData = []
  for (const data of res.Items) {
    if (data.timestamp < breakpoint) {
      prev.push(data.price)
    } else {
      trendData.push({ x: trendData.length, y: data.price })
      recent.push(data.price)
    }
  }
  let sumPrev = 0
  let sumRecent = 0
  for (const price of prev) { sumPrev  = price }
  for (const price of recent) { sumRecent  = price }
  const avgPrev = sumPrev / prev.length
  const avgRecent = sumRecent / recent.length
  const trend = createTrend(trendData, 'x', 'y')
  return { symbol, percentChange: (100 * (avgRecent - avgPrev)) / avgRecent, slope: trend.slope }
}

Update: Function execution takes about 30 seconds in a Node.js 14.x AWS Lambda environment, configured with 624 MB RAM and, from a cost perspective the both the DynamoDB query (ReadRequests) costs and Lambda executions are 1/50th of the actual DynamoDB Batch Put operations (WriteRequest), so optimizing the query should be a needed goal.

CodePudding user response：

Your loop is correctly asynchronous. And it seems you're concerned about how many queries will be in flight at the same time. In your case, you intended for many concurrent queries, and you have done as such.

Explanation

In the simple example of ordering 10 pizzas, consider these arrays:

concurrently

This is an array of pizza orders. We can await all 10 orders to get an array of 10 pizzas. Assuming there are at least 10 delivery people working that night, there's a good chance you'll get all your pizzas within 30 minutes.

for (let i = 0; i < 10;   i) {
   orders.push(orderPizza());
}
pizzas = await Promise.all(orders);

sequentially

This is an array of pizzas. Each order is awaited as it is issued. If each pizza is delivered in 30 minutes or it's free this is likely to result in about 5 hours (10 * ~30 minutes) of waiting for that last pizza because the next pizza isn't ordered until the previous one arrives. (No need for a final Promise.all in this case as each promise is already resolved before continuing with the loop.)

for (let i = 0; i < 10;   i) {
   pizzas.push(await orderPizza());
}

CodePudding user response：

Not sure I understand what your question is here exactly. Your assumptions about your code seem to be correct – I (and many others I assume) would be highly surprised if 1500 promises changed anything in their execution model.

As a general note, if you need more control over promise execution, I can recommend checking out Sindre Sorhus' collection of promise helpers. You'll find tools for limiting concurrency, allowing for promise failures, async mapping … and much more.