Error deleting AWS S3 objects from Django Web App-CodePudding

I'm having the following error when I try to delete S3 files from my Python/Django Webapp.

ClientError at /eq83/deletephoto
An error occurred (AccessDenied) when calling the DeleteObject operation: Access Denied

I have no problem viewing or uploading files.

Some details on the bucket permissions

All block public access boxes are unchecked.

Bucket policy is empty and says No policy to display.

Access control List(ACL) has all the boxes checked for bucket owner and the rest are unchecked.

This is the Cross-origin resource sharing (CORS)

[
    {
        "AllowedHeaders": [
            "*"
        ],
        "AllowedMethods": [
            "GET",
            "POST",
            "PUT",
            "DELETE"
        ],
        "AllowedOrigins": [
            "*"
        ],
        "ExposeHeaders": []
    }
]

Some details on my application

Below is an excerpt from my settings.py file. I have blocked out the secret key and omitted TEMPLATES, MIDDLEWARE, and some INSTALLED_APPS


import os
import dotenv
dotenv.read_dotenv()

BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))

# SECURITY WARNING: keep the secret key used in production secret!
SECRET_KEY = 'XXXXXXXXXXXXXXXXXXXXXXXXXX'

DEBUG = True

ALLOWED_HOSTS = ["*"]


INSTALLED_APPS = [
'tracker.apps.TrackerConfig',
...
'storages'
]

WSGI_APPLICATION = 'tracker_django.wsgi.application'


DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
    }
}

STATIC_URL = '/static/'

STATICFILES_DIRS = [os.path.join(BASE_DIR,'tracker', 'static')]

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY', '')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_KEY', '')

AWS_STORAGE_BUCKET_NAME = 'tracker-django-files'

AWS_S3_FILE_OVERWRITE = False
AWS_DEFAULT_ACL = "public-read"

AWS_QUERYSTRING_AUTH = False

And here is a sample of how I'm deleting the files in my views.py.

def remove_eq_files(request, file_id):
    try:
        eq_file = EqFile.objects.get(pk=file_id)
        eq_id=eq_file.eq.id
    except KeyError:
        return render(request, "jobs/error.html", {"message": "No Selection"})
    except EqFile.DoesNotExist:
        return render(request, "jobs/error.html", {"message": "Invalid File Selection. Contact Admin"})
    #delete file in S3
    eq_file.eq_file.delete(save=False) 
    #delete file in django
    eq_file.delete() 
    return HttpResponseRedirect(request.META.get('HTTP_REFERER'))

And finally this is the clip of code from the models.py file


class Eq(models.Model):
    name = models.CharField(max_length=128, blank=False)

def eq_file_path(instance, filename):
    try:
        eq=instance.eq
        path= "eq/" eq.name  "_" str(eq.pk) "/eq_folder/" filename
        return path

class EqFile(models.Model):
    eq_file=models.FileField(max_length=500, null=True, blank = True, upload_to = eq_file_path)
    file_url=models.URLField(max_length=500, null=True, blank=True)
    file_name = models.CharField(max_length=256, null=True, blank=True)
    eq= models.ForeignKey(Eq, null=True, on_delete=models.CASCADE, blank=True, related_name="eq_file")
    def filename(self):
        return os.path.basename(self.model_file.name)

I had someone more experienced set up the s3 and help me integrate it, and has all been working well for over a year, including deleting objects, I think that last time I tested was around 4 weeks ago. I find it peculiar that the delete seems to no longer be working and thought perhaps Amazon changed something in the way deleting s3 objects works. Or perhaps I've unwittingly changed some code on the Python side and messed it up.

(Please note that this software is not yet used by customers and I am aware that these settings might not be appropriate for deployment. )

Any help would be greatly appreciated!

CodePudding user response：

One solution to this which I ended up implementing was to set up an IAM user, give that IAM permissions to edit the bucket, and then add the IAM credentials to program.

Set up IAM

open the AWS console, use the searchbar to open 'IAM'
select 'Users' on the left under 'Access management' and create a new IAM user. Important: save the security credentials (access key and secret key) somewhere and save the User ARN
click ' add inline policy' on the right and create a new policy with the code below except replace BUCKETS!!! with the name of your bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:ListBucketVersions",
                "s3:GetBucketLocation",
                "s3:Get*",
                "s3:Put*",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKETS!!!",
                "arn:aws:s3:::BUCKETS!!!/*"
            ]
        }
    ]
}

Give IAM permissions to edit bucket

In AWS open S3 (via searchbar or the menu) and click Buckets on the left
click the bucket you want to modify and click 'Permissions' tab
scroll down to 'Bucket policy', hit Edit, and paste in the code below except replace USERARN!!! with the User ARN you saved before in the IAM Management Console and replace the BUCKETS!!! with the name of your bucket.

{
    "Version": "2012-10-17",
    "Id": "Policy1488494182833",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "USERARN!!!"
            },
            "Action": [
                "s3:ListBucket",
                "s3:ListBucketVersions",
                "s3:GetBucketLocation",
                "s3:Get*",
                "s3:Put*",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::BUCKETS!!!",
                "arn:aws:s3:::BUCKETS!!!/*"
            ]
        }
    ]
}

Now anyone with the IAM login credentials can manipulate the bucket.

Add the IAM credentials to program.

Since my project is a Django project, I am using boto3 and I entered the credentials in the settings.py by simply adding the data below except replace AK!!! with the Access Key you saved earlier and SK!!! with the Secret Access Key you saved earlier.

AWS_ACCESS_KEY_ID = 'AK!!!'
AWS_SECRET_ACCESS_KEY = 'SK!!!'

I don't know how you would do this if you're not using a Django framework.

Thanks to the prompting by luk2302 for looking into setting up the IAM user.