How can I continuously copy one S3 bucket to another? I want to copy the files every time a new file has been added.

I've tried using the boto3 copy_object however I require the key each time which won't work if I'm getting a new file each time.

CodePudding user response：

Please look at this: https://aws.amazon.com/premiumsupport/knowledge-center/move-objects-s3-bucket/

You can use the aws cli s3 sync command to achieve this.

CodePudding user response：

From Replicating objects - Amazon Simple Storage Service:

To automatically replicate new objects as they are written to the bucket use live replication, such as Same-Region Replication (SRR) or Cross-Region Replication (CRR).

S3 Replication will automatically create new objects in another bucket as soon as they are created. (Well, it can take a few seconds.)

Alternatively, you could configure the S3 bucket to trigger an AWS Lambda function that uses the CopyObject() command to copy the object to another location. This method is useful if you want to selectively copy files, by having the Lambda function perform some logic before performing the copy (such as checking the file type).