Access Denied when trying to access S3 from AWS Lambda-CodePudding

I'm trying to access S3 from AWS lambda and I keep getting an access denied. The lambda code has been deployed, and I run an application on my PC that invokes said lambda. The lambda is meant to read 1 files from S3, modify it, and write it to a different location in S3.

I have run this code without the lambda i.e. the logic that goes into the lambda can reside on my PC. That works i.e. I run the code on my PC. The code fetches the file from S3, modifies it, and saves it, successfully. However, the problem occurs when this same logic is deployed as a lambda. I therefore believe it to be some configuration problem, but I have no idea what.

Here's my inline s3 policy for the lambda execution role:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::MY_BUCKET/*"
        }
    ]
}

Here's the policty configuration for the S3 bucket:

{
    "Version": "2012-10-17",
    "Id": "PolicyNumber",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT_ID:role/web_functions"
            },
            "Action": "s3:*",
            "Resource": "arn:aws:s3:::MY_BUCKET/*"
        }
    ]
}

Note that I'm using "s3:*" for now. I've been trying various combinations and s3:* is only being used until I can get something to work.

Additionally (if it matters):

The runtime is Go 1.x
Arch: x86_64
There is no VPC configured
For the lambda execution role, there are 2 other policies. 1 is the default policy added at function creation time for logging. The other was added when I enabled X-Ray monitoring.
My S3 bucket is not publicly accessible.
The app on my PC runs using an IAM role with admin permissions. This is different from the lambda execution role.

I've looked at other posts answering this question and I think I've covered everything, but something is still wrong. Please let me know if you find any problem I missed.

Any help will be greatly appreciated

Edit: As requested, I'm sharing a portion of the code and the error message:

func getS3Client(region string) *s3.Client {
    //region is a string (not aws.String()): "us-east-2"
    client := s3.New(s3.Options{
        Region: region,
    })
    return client
}

func getFile(bucket string, filename string) {
        client := getS3Client(region)
        object, err := client.GetObject(context.TODO(), &s3.GetObjectInput{
            Bucket: aws.String(bucket),
            Key:    aws.String(filename),
        })
        if err != nil {
            return nil, errors.New(fmt.Sprintf("Error reading s3 object: %v (bucket=%s, filename=%s)\n", err, bucket, filename))
    }   
}

And here's the error as reported by golang:

{"errorMessage":"Error reading s3 object: operation error S3: GetObject, https response error StatusCode: 403, RequestID: QEWV1F12YHCT862X, HostID: knxgns9O oCTJ9G2p hXNEyZJqjeTup/25Ab6Ih6jOrN/FAZvH7gjxKb0JEp3rGQMVyzqdgu30s=, api error AccessDenied: Access Denied (bucket=MY_BUCKET, filename=DCIM/061121/DSC01033.JPG)\n","errorType":"errorString"}

And here's a sample log from testing the lambda within the AWS console:

START RequestId: 33fe5480-bbfc-42e6-bce8-e5d3589d1187 Version: $LATEST
bucket: MY_BUCKET, filename: DCIM/061121/DSC01423.JPG, region: us-east-2
Error reading s3 object: operation error S3: GetObject, https response error StatusCode: 403, RequestID: KDCZ6XBVSNHX6HE1, HostID: FG1nTNI9MAwh2HhwBfRCE0Xj355RefzsSy1ludhM52rls6yzu /ElgCeB27Q4i6enjPlX8HaGQ8=, api error AccessDenied: Access Denied (bucket=MY_BUCKET, filename=DCIM/061121/DSC01423.JPG)
: errorString
null
END RequestId: 33fe5480-bbfc-42e6-bce8-e5d3589d1187
REPORT RequestId: 33fe5480-bbfc-42e6-bce8-e5d3589d1187  Duration: 132.27 ms Billed Duration: 133 ms Memory Size: 512 MB Max Memory Used: 39 MB  Init Duration: 98.39 ms

CodePudding user response：

Solved

The permissions were fine. I didn't even need the permissions on the S3 bucket (it works with/without those permissions). The error is with the following line of code:

client := s3.New()

Apparently this call doesn't inherit things like service role values. However, the following works:

cfg, _ := config.LoadDefaultConfig(context.TODO())
client := s3.NewFromConfig(cfg)

Since I was calling the service from a lamba and not my local PC, I used s3.New() since I didn't have a credentials file, and I wasn't explicitly setting any environment variables. I wrongly assumed that s3.New() will use the role configured for the lambda. s3.NewFromConfig() does that correctly.

Hope this saves someone else a bunch of time.

CodePudding user response：

I am assuming that role attached to lambda function is arn:aws:iam::ACCOUNT_ID:role/web_functions According to me Lambda function is missing s3:ListBucket permission & that's the reaseon it showing 403 error. I would suggest to try below policy. Its having same permission you gave along with s3:ListBucket permission.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::MY_BUCKET/*"
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": "s3:ListBucket"
      "Resource": "*"
    }
  ]
}