Checking the throttle before using a GraphServiceClient-CodePudding

I have several GraphServiceClients and I'm using them to retrieve information from Microsoft Graph API. There's a throttle on the GraphServiceClient calls. As far as I understood from this documentation, you can't call APIs more than 10,000 times in a 10-minute time frame and you can only use 4 concurrent requests at the same time. What's a thread-safe and efficient way to check if I have reached the maximum limit?

My implementation

I came up with this but I'm not sure if it's actually how the Microsoft Graph is checking for the limits.

public class ThrottledClient
{
    private readonly TimeSpan _throttlePeriod;
    private readonly int _throttleLimit;

    public ThrottledClient(int throttleLimit, TimeSpan throttlePeriod)
    {
        _throttleLimit = throttleLimit;
        _throttlePeriod = throttlePeriod;
    }


    private readonly ConcurrentQueue<DateTime> _requestTimes = new();

    public required GraphServiceClient GraphClient { get; set; }

    public async Task CheckThrottleAsync(CancellationToken cancellationToken)
    {
        _requestTimes.Enqueue(DateTime.UtcNow);

        if(_requestTimes.Count > _throttleLimit)
        {
            Console.WriteLine($"Count limit, {DateTime.Now:HH:mm:ss}");
            _requestTimes.TryDequeue(out var oldestRequestTime);

            var timeRemaining = oldestRequestTime   _throttlePeriod - DateTime.UtcNow;
            if(timeRemaining > TimeSpan.Zero)
            {
                Console.WriteLine($"Sleeping for {timeRemaining}");

                await Task.Delay(timeRemaining, cancellationToken).ConfigureAwait(false);
                Console.WriteLine($"Woke up, {DateTime.Now:HH:mm:ss}");
            }
        }
    }
}

public class Engine
{
    public async Task RunAsync()
    {
        var client = GetClient();
        await client.CheckThrottleAsync(_cts.Token).ConfigureAwait(false);
        
        await DoSomethingAsync(client.GraphClient).ConfigureAwait(false);
    }
}

I can think of other ways to use like lock or Semaphore but again, I'm not sure if I'm thinking about this correctly.

CodePudding user response：

I believe you can use a Graph Developer proxy to test these.

Microsoft Graph Developer Proxy aims to provide a better way to test applications that use Microsoft Graph. Using the proxy to simulate errors, mock responses, and demonstrate behaviors like throttling, developers can identify and fix issues in their code early in the development cycle before they reach production.

More details can be found here https://github.com/microsoftgraph/msgraph-developer-proxy

CodePudding user response：

We use Polly to automatically retry failed http calls. It also has support for exponentially backing of.

So why not handle the error in a way that works for your application, instead of trying to figure out what the limit it before hand (and doing an extra call counting to the limit). You can test those scenarios with the Graph Developer proxy from the other answer.

We also use a circuit breaker to fail quick without extra call to graph and retry later.

The 4 concurrent requests you’re mentioning are for Outlook resources (I’ve written many user voices, GitHub issues and escalations on it). Since Q4 2022, you can do 20 requests in a batch for all resources (in the same tenant). Batching reduces the http overhead and might help you to overcome throttling limits, by combining requests in a smart way.