I am stucking trying to upload images to aws s3 using the boto3 library from my flask app running on an ec2 instance with the following s3 policy attached to the Instance role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketTagging",
                "s3:ListBucket",
                "s3:GetBucketAcl",
                "s3:GetBucketPolicy",
                "s3:PutObject",
                "s3:GetObjectAcl",
                "s3:GetObject",
                "s3:PutObjectVersionAcl",
                "s3:GetObjectVersionAcl",
                "s3:PutBucketAcl",
                "s3:PutBucketPolicy",
                "s3:DeleteObject",
                "s3:GetBucketLocation",
                "s3:PutObjectAcl",
                "s3:AbortMultipartUpload",
                "s3:ListBucketMultipartUploads",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-name",
                "arn:aws:s3:::bucket-name/*",
                "arn:aws:s3:::bucket-name/users/*",
                "arn:aws:s3:::*/*"
            ]
        }
    ]
}

This works from my computer but fails when uploaded to the flask app running on the ec2 instance. I get the following error:

An error occurred (AccessDenied) when calling the PutObject operation: Access Denied

This is the flask code used to carryout this upload:

class S3ImageBucketManagement:
    S3_LOCATION = 'http://{}.s3.amazonaws.com/'.format('bucket-name')

    def __init__(self, bucket_name: str, bucket_region: str, _id: str, image_type: str, acl: str = "public-read"):
        self._client = self.__s3_resource().Bucket(bucket_name)
        self._image_type = image_type
        self._bucket_name = bucket_name
        self._bucket_region = bucket_region
        self._id = _id
        self._acl = acl

    def __s3_resource(self):
        """
        Creates the boto3 client
        :return: boto3.client
        """
        session = boto3.Session()
        res = None
        try:
            res = session.resource(
                service_name='s3',
                region_name='eu-west-2'
            )
        except Exception as e:
            print(f'Error: {e}')
        return res

    def upload_file_to_s3(self, file: FileStorage, upload_folder: str):
        try:
            res = self._client
            res.put_object(
                Body=file,
                Bucket=self._bucket_name,
                Key=f"{self._image_type}/{file.filename}",
                ACL=self._acl,
                ContentType=file.content_type,
            )

            # res.upload_file(
            #     Filename=f"/tmp/{file.filename}",
            #     Key=f"{self._image_type}/{file.filename}",
            #     ExtraArgs={
            #         "ACL": self._acl,
            #         "ContentType": file.content_type
            #     }
            # )
         except Exception as e:
            current_app.logger.error(f'upload_file_to_s3 function error: {e}')
            return False, 'File upload failed', 500
         return True, "{}{}/{}".format(self.S3_LOCATION, self._image_type, file.filename), 200

I don't know what I am missing in the permissions, and why this works on my system but not on the server.

CodePudding user response：

Your policy looks fine. You have both of the required actions

s3:PutObject
s3:PutObjectACL

and listing the buckets in the correct way. However, python implementation is a bit problematic. AWS python SDK provides two interfaces: a client and a resource.

Client interface is very close to API and you receive back a json as a response if the action returns one.

Resource interface is pythonic, it is higher level and returns an object that you can work with.

In the code above, you initiate a resource from Session

res = session.resource(
            service_name='s3',
            region_name='eu-west-2'
        )

However, when trying to upload, syntax is using the client syntax instead of resource syntax. If you wanted to use client's put_object method, then you should initiate a client from session.