How to design goroutines program to handle api limit error-CodePudding

Just started learning about the power of goroutines.

I have ~100 accounts and ~10 regions, looping through them to create ~ 1000 goroutines with golang to increase the reading speed. It worked too fast that it hit the API return limit of 20/ sec.

How do I ensure that all the goroutines can maintain at the maximum call rate of (20/s)? Im unsure of which golang concurrency methods works best together to handle the error.

eg:

func readInstance(acc string, region string, wg *sync.WaitGroup) {
    defer wg.Done()
    response, err := client.DescribeInstances(acc, region)
    if err != nil {
        log.Println(err) // API limit exceeding 20
        return
    }
    .
    .
    .
    .

}

func main() {
    accounts := []string{"g", "h", "i", ...}
    regions := []string{"g", "h", "i", ...}
    for _, region := range regions {
        for i := 0; i < len(accounts); i   {
            wg.Add(1)
            go readInstance(accounts[i], region, &wg)
        }
    }
    wg.Wait()
}

CodePudding user response：

If you have a fixed upper limit on how many requests you can do in a particular amount of real time, you can use a time.NewTicker() to space things out.

c := time.NewTicker(50 * time.Millisecond)
defer c.Stop()

Now, when you want to make a server request, just insert

<- c.C

prior to the actual request.