S3 replication: delete files in the source bucket after they've been replicated-CodePudding

I enabled S3 replication for a bucket in another account. Is there any way I can delete the objects from the source bucket once they've been replicated to the destination one?

I can't seem to find an easy way to do this.

CodePudding user response：

It appears that your actual goal is to 'move' objects from one bucket to another.

This could be accomplished with an AWS Lambda function that is triggered by Amazon S3 whenever a new object is created in the source bucket. The Lambda function should then:

Copy the object to the destination bucket
Delete the object from the source bucket

This avoids the need to configure S3 Replication.

Important note: When copying an object to a bucket owned by a different AWS Account, be sure to use ACL=bucket-owner-full-control so that the destination account 'owns' the object after the copy operation.

The code would be something like:

import boto3
import urllib

DESTINATION_BUCKET = 'bucket2'

def lambda_handler(event, context):
    
    s3_client = boto3.client('s3')

    # Get the bucket and object key from the Event
    for record in event['Records']:
        source_bucket = record['s3']['bucket']['name']
        key = urllib.parse.unquote_plus(record['s3']['object']['key'])
        
        # Copy object
        s3_client.copy_object(
            Bucket = DESTINATION_BUCKET,
            Key = key,
            CopySource= {'Bucket': source_bucket, 'Key': key},
            ACL = 'bucket-owner-full-control'
        )
        
        # Delete source object
        s3_client.delete_object(
            Bucket = source_bucket,
            Key = key
        )