Using golang.org/x/time/rate to allow N requests per minute-CodePudding

What I want:
A limiter which allows n requests per minute.

What I tried:

(somewhere during init procedure)

limiter = rate.NewLimiter(rate.Every(1*time.Minute/2), 2)

Then in the middleware of my HTTP server:

func (self *Router) limiterMiddleware(next http.Handler) http.Handler {
    return http.HandlerFunc(func(responseWriter http.ResponseWriter, request *http.Request) {
        if !limiter.Allow(){
            http.Error(responseWriter, "Too many requests", http.StatusTooManyRequests)
            return
        }

        next.ServeHTTP(responseWriter, request)
    })
}

This should, to my understanding, allow 2 requests per minute. Then I tried to send several requests, with the first two having 15s of pause in between, then roughly 1 request per second.

Expectation:

first two requests work
subsequent requests get HTTP429 until 1 minute has passed, and the first succeeding one is "cleared"
one more request works
subsequent requests get HTTP429 until 15s have passend, and the initial second request is"cleared"

Actual result:

22/10/31 14:02:31 access: 200 POST /some/path
22/10/31 14:02:46 access: 200 POST /some/path
22/10/31 14:02:47 access: 429 POST /some/path
22/10/31 14:02:48 access: 429 POST /some/path
22/10/31 14:02:49 access: 429 POST /some/path
22/10/31 14:02:50 access: 429 POST /some/path
22/10/31 14:02:51 access: 429 POST /some/path
22/10/31 14:02:52 access: 429 POST /some/path
22/10/31 14:02:53 access: 429 POST /some/path
22/10/31 14:02:54 access: 429 POST /some/path
22/10/31 14:02:55 access: 429 POST /some/path
22/10/31 14:02:56 access: 429 POST /some/path
22/10/31 14:02:57 access: 429 POST /some/path
22/10/31 14:02:58 access: 429 POST /some/path
22/10/31 14:02:59 access: 429 POST /some/path
22/10/31 14:03:00 access: 429 POST /some/path
22/10/31 14:03:01 access: 200 POST /some/path
22/10/31 14:03:02 access: 429 POST /some/path
22/10/31 14:03:03 access: 429 POST /some/path
22/10/31 14:03:04 access: 429 POST /some/path
22/10/31 14:03:05 access: 429 POST /some/path
22/10/31 14:03:06 access: 429 POST /some/path
22/10/31 14:03:07 access: 429 POST /some/path
22/10/31 14:03:08 access: 429 POST /some/path
22/10/31 14:03:09 access: 429 POST /some/path
22/10/31 14:03:10 access: 429 POST /some/path
22/10/31 14:03:11 access: 429 POST /some/path
22/10/31 14:03:12 access: 429 POST /some/path
22/10/31 14:03:13 access: 429 POST /some/path
22/10/31 14:03:14 access: 429 POST /some/path
22/10/31 14:03:15 access: 429 POST /some/path
22/10/31 14:03:16 access: 429 POST /some/path
22/10/31 14:03:16 access: 429 POST /some/path
22/10/31 14:03:17 access: 429 POST /some/path
22/10/31 14:03:18 access: 429 POST /some/path
22/10/31 14:03:19 access: 429 POST /some/path
22/10/31 14:03:20 access: 429 POST /some/path
22/10/31 14:03:21 access: 429 POST /some/path
22/10/31 14:03:22 access: 429 POST /some/path
22/10/31 14:03:23 access: 429 POST /some/path
22/10/31 14:03:24 access: 429 POST /some/path
22/10/31 14:03:25 access: 429 POST /some/path
22/10/31 14:03:26 access: 429 POST /some/path
22/10/31 14:03:27 access: 429 POST /some/path
22/10/31 14:03:28 access: 429 POST /some/path
22/10/31 14:03:29 access: 429 POST /some/path
22/10/31 14:03:30 access: 429 POST /some/path
22/10/31 14:03:32 access: 200 POST /some/path
22/10/31 14:03:33 access: 429 POST /some/path

You can see from the log that after the initial 2 requests, the third successful request happens after 30s. This means, that during 30s of uptime, 3 requests have succeeded, while it should only be 2. The fourth successful request happens again after 30s (60s in total).

If I set the bursts to 1, then initially only 1 request succeeds, and every 30s one more request can succeed.

So I am not sure how I should configure the limiter to achieve what I want (plain n requests per minute).

What am I doing wrong? Can this even be achieved using the built-in limiter, or do I need a different library for this task?

CodePudding user response：

Limiter implements a token bucket algorithm, which in essence feeds tokens at the stipulated interval.

You mentioned sliding window in a comment: I don't believe a "standard" implementations of sliding window for rate limiting will handle this for you either. The approach described under https://konghq.com/blog/how-to-design-a-scalable-rate-limiting-algorithm indicates weighting over fixed windows should be used, and I've seen this type approach implemented.

What you're describing seems to better match what is described as "sliding log" in the linked article, which you could look to implement.

Considering the constraints you're working with (2 req/minute), there might also be an option to just use two token buckets side-by-side to achieve your goal. I wouldn't really recommend it for use cases which require higher throughput.

func NewMultiLimiter(interval time.Duration, r int) MultiLimiter {
    limiter := make(MultiLimiter, r)
    for i := 0; i < r; i   {
        limiter[i] = rate.NewLimiter(rate.Every(interval), 1)
    }
    return limiter
}

type MultiLimiter []*rate.Limiter

func (ls MultiLimiter) Allow() bool {
    for _, l := range ls {
        if l.Allow() {
            return true
        }
    }
    return false
}

Then using

limiter := NewMultiLimiter(time.Minute, 2)

With the above, two token buckets will essentially exist independently, so the effective addition of new tokens is always relative to 2 separate points in time.

2022/10/31 17:51:15 allow: true
2022/10/31 17:51:30 allow: true
2022/10/31 17:51:31 allow: false
2022/10/31 17:51:32 allow: false
2022/10/31 17:51:33 allow: false
2022/10/31 17:51:34 allow: false
2022/10/31 17:51:35 allow: false
2022/10/31 17:51:36 allow: false
2022/10/31 17:51:37 allow: false
2022/10/31 17:51:38 allow: false
2022/10/31 17:51:39 allow: false
2022/10/31 17:51:40 allow: false
2022/10/31 17:51:41 allow: false
2022/10/31 17:51:42 allow: false
2022/10/31 17:51:43 allow: false
2022/10/31 17:51:44 allow: false
2022/10/31 17:51:45 allow: false
2022/10/31 17:51:46 allow: false
2022/10/31 17:51:47 allow: false
2022/10/31 17:51:48 allow: false
2022/10/31 17:51:49 allow: false
2022/10/31 17:51:50 allow: false
2022/10/31 17:51:51 allow: false
2022/10/31 17:51:52 allow: false
2022/10/31 17:51:53 allow: false
2022/10/31 17:51:54 allow: false
2022/10/31 17:51:55 allow: false
2022/10/31 17:51:56 allow: false
2022/10/31 17:51:57 allow: false
2022/10/31 17:51:58 allow: false
2022/10/31 17:51:59 allow: false
2022/10/31 17:52:00 allow: false
2022/10/31 17:52:01 allow: false
2022/10/31 17:52:02 allow: false
2022/10/31 17:52:03 allow: false
2022/10/31 17:52:04 allow: false
2022/10/31 17:52:05 allow: false
2022/10/31 17:52:06 allow: false
2022/10/31 17:52:07 allow: false
2022/10/31 17:52:08 allow: false
2022/10/31 17:52:09 allow: false
2022/10/31 17:52:10 allow: false
2022/10/31 17:52:11 allow: false
2022/10/31 17:52:12 allow: false
2022/10/31 17:52:13 allow: false
2022/10/31 17:52:14 allow: false
2022/10/31 17:52:15 allow: true
2022/10/31 17:52:16 allow: false
2022/10/31 17:52:17 allow: false
2022/10/31 17:52:18 allow: false
2022/10/31 17:52:19 allow: false
2022/10/31 17:52:20 allow: false
2022/10/31 17:52:21 allow: false
2022/10/31 17:52:22 allow: false
2022/10/31 17:52:23 allow: false
2022/10/31 17:52:24 allow: false
2022/10/31 17:52:25 allow: false
2022/10/31 17:52:26 allow: false
2022/10/31 17:52:27 allow: false
2022/10/31 17:52:28 allow: false
2022/10/31 17:52:29 allow: false
2022/10/31 17:52:30 allow: true
2022/10/31 17:52:31 allow: false
2022/10/31 17:52:32 allow: false
2022/10/31 17:52:33 allow: false