Count API Requests to external API with Laravel-CodePudding

I've been trying to solve a problem with external service that has a rate limiter. My Laravel app will be doing regular requests via jobs. Users also can interact with the external service. This means that jobs requests users requests can easily exceed the rate limit of the service and result in a X minute timeout(ban). To prevent that, Laravel app has to know the number of request for the last X minutes/seconds to allow certain method to be executed without exceeding the API rate. Has anyone ran into the same problem and how did you solve it?

CodePudding user response：

Probably a solution would be to use a table to store the number of requests for every url (or service provider) with the reset time. When you want to make a query you can refer to the static Eloquent Model to retrieve the number of requests to the url and check of you did not reach the maximum. Also, you can use a service or task to reset the timer on this specific table when the reset time is reached.

Does it help you ?

CodePudding user response：

For this type of scenario, you can create a custom middleware for handling different types of requests.

Create a middleware by command => php artisan make:middleware requestCountMiddleware
For new log table you can create a migration and model of RequestLog::class with columns of id (autoincrement) , count(big integer), timestamps().
After that inside App/Http/Middleware/requestCountMiddleware.php create logic for storing or updating the entry of each request based on the date inside the RequestLog table.

4. Sample logic as per your request =>

public function handle($request, Closure $next)
{
    $checkRequest = App\Models\RequestLog::whereDate('created_at', Carbon::now()->format('Y-m-d'))->first();
    if(!$checkRequest){
        $checkRequest = new App\Models\RequestLog();
    }
    $checkRequest->count  = 1;
    $checkRequest->save();
    return $next($request);
}

Also from Job methods you can do the same logic for adding or updating the count of requests.

6. So whenever you need a count by the date you can get it from that table with this type query => $date = Any Date what you want data from table ( 2022-09-27 )

App\Models\RequestLog::whereDate('created_at', $date)->value('count');

7. Use this middleware in the user request route by =>

Route::get('/details', function () {
    //
})->middleware('requestCountMiddleware');

Also if you want to handle multiple requests like more than 60 or 120 request per minute than also you can set RateLimiter for handle maximum number of request accept per IP or per user in Laravel app.

CodePudding user response：

If you only want to count outcomes API requests you can set event and listener that populate something like requests table in DB.

So, register event and listener in EventServiceProvider:

protected $listen = [
    'App\Events\RequestSent' => [
        'App\Listeners\IncrementCounterIntoRequestsDBTable',
    ],
];

and then: php artisan event:generate
see documentation

In this scenario, each request dispatch an event that increments a counter inside a requests table.

*******************************************************************

But if you want to solve the problem of 3rd party API rate limits:

Optimize and make something like a cache: if you don't need "strict" realtime you can:

schedules API requests each XX minutes, via commands or further jobs
store fresh data into your internal DB
then you can serves this data to user requests.

In this manner you can have a "pre-defined" update each XX minutes, depending on rate limits.

For example, if you have rate limit about 1.500 req/day you can potentially update your data each minute.