listing s3 buckets using boto3 and python-CodePudding

I am using the below code and referred to many SO answers for listing files under a folder using boto3 and python but was unable to do so. Below is my code:

s3 = boto3.client('s3')
        object_listing = s3.list_objects_v2(Bucket='maxValue',
                                    Prefix='madl-temp/')

My s3 path is "s3://madl-temp/maxValue/" where I want to find if there are any parquet files under the maxValue bucket based on which I have to do something like below:

If len(maxValue)>0:
 maxValue=true
else:
 maxValue=false

I am running it via Glue jobs and I am getting the below error:

botocore.errorfactory.NoSuchBucket: An error occurred (NoSuchBucket) when calling the ListObjectsV2 operation: The specified bucket does not exist

CodePudding user response：

Your bucket name is madl-temp and prefix is maxValue. But in boto3, you have the opposite. So it should be:

s3 = boto3.client('s3')
object_listing = s3.list_objects_v2(Bucket='madl-temp',
                                    Prefix='maxValue/')

To get the number of files you have to do:

len(object_listing['Contents']) - 1

where -1 accounts for a prefix maxValue/.