aws s3 ls listing a file that cannot be found in console-CodePudding

I have a s3 bucket in AWS, and I have tried to list all the files in the bucket by:

aws s3 ls s3://bucket-1 --recursive | awk '{$1=$2=$3=""; print $0}' | sed 's/^[ \t]*//' | sort > bucket_1_files

This works for most files, but there are some files that are listed in bucket_1_files, but when I search for the file in the s3 bucket in the console, I cannot find the files (no matches returned when I search for the name) - would anyone know of possible reasons this could be the case? The file is a .png file, and there are other .png files listed that I can find within the console.

CodePudding user response：

I think there is something wrong with the command I'm using, when I just do

aws s3 ls s3://bucket-1 --recursive

I am finding that a lot of these missing files actually have a "t" in front of them

CodePudding user response：

Rather than playing with aws, sed and sort, you can list objects with:

aws s3api list-objects --bucket bucket-1 --query 'Contents[].[Key]' --output text

However, it will only return 1000 objects.