Python program crashing when printing blob names for large blob container-CodePudding

Recently I've been writing a Python program that prints out the names of all the blobs within a blob container. The program seems to work fine for blob containers that have a decent amount of files.

However, one of the blob containers has roughly 1,100,000 items. When I try to print all of the blobs in this container, my Python program crashes. I'm fairly new to working with Azure in Python, and I was wondering if there was any way to print all of the names of the blobs within the container without crashing (e.g. print a certain number of blob names per a given time).

This is the code I am using to print the names of the blobs in a given container:

generator = container_client.list_blobs()
for item in generator:
  print("Container: {0}, Blob:
  {1}\n".format(container_client.container_name, item.name))

Thank you!

CodePudding user response：

The List-Blobs has a limit to list maximum 5000 results. If you want to get more than 5000 results you have to use the NextMarker to get the continuous token and the results of all lists of blobs.

List the overall blob names

nextMarker = None
blobsArray = []

while True:
generator = container_client.list_blobs('<Your_Container>', nextMarker=nextMarker)
    blobsArray.extend(generator)

# Using next_market to get Contiuous Token and the rest of the blob result
    if not generator.next_marker:
        break
    nextMarker = generator.next_marker
    
for item in blobsArray:
    print("Container: {0}, Blob:{1}\n".format(container_client.container_name, item.name))