S3 cross account file transfer, file not accessible-CodePudding

I am pushing a s3 file from accountA to accountB but the pushed file is not accessible from accountB. I checked the pushed file and the Owner of the pushed file appears to be accountA.

Here is what I have done.

The IAM role in accountA has this policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:*",
                "s3-object-lambda:*"
            ],
            "Resource": "*"
        }
    ]
}

The bucket policy in accountB looks like this:

{
            "Sid": "S3AllowPutFromDataLake",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::accountId:role/roleNameAccountA"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::bucketName/*"
        }

How to fix this?

CodePudding user response：

This is a common problem when copying S3 objects between AWS Accounts. Here are several options to avoid it happening. Pick whichever one you prefer:

Pull instead of Push

The problem occurs when Account A copies an object to Account B. Ownership stays with Account A.

This can be avoided by having Account B trigger the copy. It is, in effect, 'pulling' the object into Account B rather than 'pushing' the object. Ownership will stay with Account B, since Account B requested the copy.

Disable ACLs

The concept of object-level ACLs pre-dates Bucket Policies and causes many problems like the one you are experiencing.

Amazon S3 has now introduced the ability to disable ACLs on a bucket and this is the recommended option when creating new buckets. Disabling the ACLs will also remove this 'ownership' concept that is causing problems. In your situation, it is the Target bucket in Account B that should have ACLs disabled.

See: Disabling ACLs for all new buckets and enforcing Object Ownership - Amazon Simple Storage Service

Specify ownership while copying

When copying the file, it is possible to specify that ownership should be transferred by setting the ACL to bucket-owner-full-control.

Using the AWS CLI:

aws s3 cp s3://bucket-a/foo.txt s3://bucket-b/foo.txt --acl bucket-owner-full-control

Using boto3:

    s3_client.copy_object(
                        ACL = 'bucket-owner-full-control',
                        Bucket = DESTINATION_BUCKET,
                        Key = KEY,
                        CopySource = {'Bucket':SOURCE_BUCKET, 'Key':KEY}
                    )

CodePudding user response：

Was able to fix this by modifying the bucket policy as below:

        {
            "Sid": "S3AllowPutFromDataLake",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::accountId:role/roleNameAccountA"
            },
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl"
            ]
            "Resource": "arn:aws:s3:::bucketName/*",
            "Condition": {
                "StringEquals": {
                    "s3:x-amz-acl": "bucket-owner-full-control"
                }
            }
        }

And adding this parameter while pushing the file:

'ACL': 'bucket-owner-full-control'

The owner is still accountA but now I am able to access the file from accountB.