How to download the current version of a file from an S3 versioned bucket-CodePudding

I have objects with multiple versions and I am trying to compare which versions I can delete. I basically want to delete any version that has the same size of the current version. The problem that I am having is that I can't find out which of the returned versions is the latest/current.

If I use the aws cli, it returns a field called 'IsLatest' but apparently, the boto3 version doesn't. The aws cli also always returns the StorageClass while boto3 doesn't in some scenarios apparently.

Return from boto3:

{'ResponseMetadata': {'RequestId': 'PHQFMDCF3AHQM6R1', 'HostId': 'b7PmgsVm6y30wfA9GExS Rc659cu1DI4YFec3i7tvDBew8ob5tY0Mtz6q yC9nTwdmAoykdV7Lo=', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amz-id-2': 'b7PmgsVm6y30wfA9GExS Rc659cu1DI4YFeR3i7tVDBeu8ab5tY0Mtz6X yC9nTwdmAoykdV7Lo=', 'x-amz-request-id': 'PHQFMDTB32HQM6R1', 'date': 'Sat, 19 Feb 2022 22:42:14 GMT', 'last-modified': 'Thu, 17 Feb 2022 17:02:54 GMT', 'etag': '"55f146382684970d4970ae31b3d4b310"', 'x-amz-server-side-encryption': 'AES256', 'x-amz-version-id': 'gHm2D2uuosJQS6GpmuySU9uNSXN84cq9', 'accept-ranges': 'bytes', 'content-type': 'text/plain', 'server': 'AmazonS3', 'content-length': '969'}, 'RetryAttempts': 0}, 'AcceptRanges': 'bytes', 'LastModified': datetime.datetime(2022, 2, 17, 17, 2, 54, tzinfo=tzutc()), 'ContentLength': 969, 'ETag': '"55f141382684970d4970ae31b3d4b310"', 'VersionId': 'gHa2D2uuosJQS6GpmuySU9uNSXN84cR9', 'ContentType': 'text/plain', 'ServerSideEncryption': 'AES256', 'Metadata': {}, 'Body': <botocore.response.StreamingBody object at 0x10f29e1c0>}
Versioning_Test/file1.txt

Response from aws cli:

        {
            "ETag": "\"55f141382684970d4970ae31b3d4b310\"",
            "Size": 969,
            "StorageClass": "STANDARD",
            "Key": "Versioning_Test/file1.txt",
            "VersionId": "gHa2D2uuosJQS6GpmuySU9uNSXN84cR9",
            "IsLatest": true,
            "LastModified": "2022-02-17T17:02:54 00:00",
            "Owner": {
                "ID": "1e5bc34834bec07ae1bc55a5d07adab10d7d58da04ae761769339a914d1ab472"
            }
        },

Here is my python script:


bucket_name = 'bucket-name'
profile_name = 'aws-profile-name'

key = ''
session = boto3.session.Session(profile_name=profile_name)
s3 = session.resource('s3')
versions = s3.Bucket(bucket_name).object_versions.filter()

for version in versions:
    print(version.object_key)
    obj = version.get()
    print(obj)
    #print("\t"   obj.get('VersionId'), obj.get('ContentLength'), obj.get('LastModified'), obj.get('IsLatest'), obj.get('StorageClass'))

I am missing something?

CodePudding user response：

You can list your object versions from a bucket using list_object_versions API:

import boto3

bucket_name = 'bucket-name'
profile_name = 'aws-profile-name'

if __name__ == "__main__":
    session = boto3.Session(profile_name=profile_name)
    client = session.client('s3')
    response = client.list_object_versions(Bucket=bucket_name)
    for version in response['Versions']:
        print(f'Key: {version["Key"]}, Size: {version["Size"]} bytes, Latest: {version["IsLatest"]}'
              f' LastModified: {version["IsLatest"]}, StorageClass: {version["StorageClass"]}')

You can notice that the response from AWS contains an IsLatest property as well.