Enumerating large Azure BLOB containers using Azure.Storage.Blobs-CodePudding

With Microsoft.WindowsAzure.Storage.Blob one could enumerate a large BLOB container using the BlobContinuationToken in the ListBlobsSegmentedAsync method, and the BlobResultSegment would include the ContinuationToken property if a subsequent read would be required to retrieve more from the container.

With Azure.Storage.Blobs it appears the enumeration API is BlobContainerClient.GetBlobsAsync but the method signature does not provide a means of supplying a continuation token, and the results as Azure.AsyncPageable<BlobItem> do not appear to provide a property that is a continuation token.

Does anyone have a code snippet for enumerating large BLOB containers on Azure, using continuation tokens, via the Azure.Storage.Blobs package?

CodePudding user response：

If you really want to use a continuation token you can do so:

    string continuationToken = null;
    var container = new BlobContainerClient("xxx", "test");
    var blobPages = container.GetBlobsAsync().AsPages();
        
    await foreach (var page in blobPages)
    {
        continuationToken = page.ContinuationToken;
        foreach(var blob in page.Values)
        {
            ...
        }
    }

You can pass the token to AsPages to continue iteration:

blobPages = container.GetBlobsAsync().AsPages(continuationToken: continuationToken);

or you can just iterate over the blobs, let the SDK do the job of providing all the data using a stream of blobs:

    var container = new BlobContainerClient("xxx", "test");
    var blobs = container.GetBlobsAsync();
        
    await foreach (var blob in blobs)
    {
        //
    }

Iterating over the blobs can make multiple calls to the blob service.

For more about pagination in the SDK see the docs.