What's the most secure way of accessing s3 resource?-CodePudding

The question heading is broad but my question is not. I just want clarification on my approach. I have an s3 bucket with blocked public access. The bucket policy is set to http-referer. This is how it looks.

{
    "Version": "2008-10-17",
    "Id": "http referer policy example",
    "Statement": [
        {
            "Sid": "Allow get requests referred by www.mysite.com and mysite.com",
            "Effect": "Allow",
            "Principal": "*",
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::storage/*",
            "Condition": {
                "StringLike": {
                    "aws:Referer": [
                        "https://www.example.com/*",
                        "https://example.com/*",
                    ]
                }
            }
        }
    ]
}

I still get an error if my frontend (on my tld) tries to access the s3 resource through the URL. (URL example - https://storage.s3.amazonaws.com/path/to/my/file.png).

I dropped the approach of hitting the s3 URL directly and decided to build a backend utility on my TLD that'd fetch the s3 resource in question and send it back to the frontend. So the URL would look something like this, https://<tld>/fetch-s3-resource/path/to/file.png.

I wanna know if this approach is correct or if there's a better one out there. In my mind even setting a http-referer policy doesn't make sense cause anyone can make a call to my bucket with http-referer manually set to my TLD.

UPDATE - I found out about signed URL's which should supposedly allow user's to publically access a resourec through the URL. This should solve my problem but I still have set public access to "off" and I don't really know which switch to toggle in order to allow for the user's with signed urls to be able to access the resource.

Here's the sample of s3 signed URL. Here's the link to the doc

import logging
import boto3
from botocore.exceptions import ClientError


def create_presigned_url(bucket_name, object_name, expiration=3600):
    """Generate a presigned URL to share an S3 object

    :param bucket_name: string
    :param object_name: string
    :param expiration: Time in seconds for the presigned URL to remain valid
    :return: Presigned URL as string. If error, returns None.
    """

    # Generate a presigned URL for the S3 object
    s3_client = boto3.client('s3')
    try:
        response = s3_client.generate_presigned_url('get_object',
                                                    Params={'Bucket': bucket_name,
                                                            'Key': object_name},
                                                    ExpiresIn=expiration)
    except ClientError as e:
        logging.error(e)
        return None

    # The response contains the presigned URL
    return response

UPDATE UPDATE - Since the question isn't already clear enough, and it seems like I took some liberties with my "loose" language, let me clarify some things.

1 - What am I actually trying to do? I want to keep my s3 bucket secure in such a way that only user's with presigned URL's generated by "me" can access whatever resource is there. 2 - When I ask if "my approach" is better or if there's any other approach what do I mean by that? I wanna know if there's a "native" / aws provided way of accessing the bucket without having to write a backend endpoint that'd fetch the resource and throw it back on the frontend. 3 - How do I measure one approach against another ? I think this one is quite obvious, you don't try to write an authentication flow from scratch if there's one provided by your framework*. This logic applies here too, if there's a way to access the objects that's listed by AWS then I probably shouldn't go about writing my own "hack"

CodePudding user response：

Presigned URLs came through. You can have all public access blocked and still be able to generate signed URL's and serve it the frontend. I've already linked the official documentation in my question, here's the final piece of code I ended up with.

def create_presigned_url(bucket_name, bucket_key, expiration=3600, signature_version='s3v4'):
    """Generate a presigned URL for the S3 object
    :param bucket_name: string
    :param bucket_key: string
    :param expiration: Time in seconds for the presigned URL to remain valid
    :param signature_version: string
    :return: Presigned URL as string. If error, returns None.
    """
    s3_client = boto3.client('s3',
                             aws_access_key_id=settings.AWS_ACCESS_KEY_ID,
                             aws_secret_access_key=settings.AWS_SECRET_ACCESS_KEY,
                             config=Config(signature_version=signature_version),
                             region_name='us-east-1'
                             )   
    try:
        response = s3_client.generate_presigned_url('get_object',
                                                    Params={'Bucket': bucket_name,
                                                            'Key': bucket_key},
                                                    ExpiresIn=expiration)
    except ClientError as e:
        logging.error(e)
        return None
    # The response contains the pre-signed URL
    return response