Issue getting access_key and secret_access_key from Lambda function-CodePudding

I am running a python script in a lambda function that retrieves both the access_key and secret_access_key. However when I use them later (to copy s3 data to redshift) I get the following error:

"errorMessage": "S3ServiceException:The AWS Access Key Id you provided does not exist in our records.,Status 403,Error InvalidAccessKeyId

I have the same lambda role in other functions and I can write in s3, in redshift, etc.

On the other hand, when I run this script in a ec2 instance (with aws credentials configured) and I add a default session: boto3.setup_default_session(profile_name='myUserName'), it works perfect because uses specific credentials.

Script

import boto3
import psycopg2 as sql

def lambda_handler(event, context):
    # retrieve aws credentials
    client_iamCredentials = boto3.Session()
    credentials = client_iamCredentials.get_credentials()
    credentials = credentials.get_frozen_credentials()
    access_key = credentials.access_key
    secret_key = credentials.secret_key

    # psycopg2 connection
    conn = sql.connect('connection details')
    cur = conn.cursor()

    # copy data from s3 to temp_table
    s3_copy = f"""COPY table_name
                    FROM 's3://bucketname/file.csv'
                    CREDENTIALS 'aws_access_key_id={access_key};aws_secret_access_key={secret_key}' csv 
                    DELIMITER AS ','
                    FILLRECORD
                    emptyasnull
                    blanksasnull
                    IGNOREHEADER 1 
                    NULL 'NaN' 
                    ACCEPTINVCHARS;"""

    cur.execute(s3_copy)
    conn.commit()
    cur.close()
    conn.close()

The question is, how can I retriever proper credentials that work in Lambda? I.e. my specific user credentials. Or maybe there is other way to execute the s3_copy variable in credentials section?

Thanks.

CodePudding user response：

You do not need to call boto3 get_credentials(). In Lambda, you will have environment variables already available that you can use: https://docs.aws.amazon.com/lambda/latest/dg/configuration-envvars.html#configuration-envvars-runtime

It contains: AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN

you will need all three in order to make it work.