I inserted my AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in ~/.aws/credentials and they seem to work. However, my java application (running on that EC2 instance and using AWS Java v2 SDK) is unable to see them. I got this exception while uploading a file to S3 using java code.

Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Either the environment variable AWS_WEB_IDENTITY_TOKEN_FILE or the javaproperty aws.webIdentityTokenFile must be set., ProfileCredentialsProvider(): Profile file contained no credentials for profile 'default': ProfileFile(profiles=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): The requested metadata is not found at http://169.254.169.254/latest/meta-data/iam/security-credentials/]

FYI, uploading the file using aws cli works fine while running this command in my ec2 instance: aws s3 cp foo.txt s3://mybucket-name

FYI, the same application runs correctly when I run it on my local machine with an STS client as a credential provider

According to AWS documentation,

Using the Default Credential Provider Chain When you initialize a new service client without supplying any arguments, the AWS SDK for Java attempts to find AWS credentials by using the default credential provider chain implemented by the DefaultAWSCredentialsProviderChain class. The default credential provider chain looks for credentials in this order:

1. Environment variables-AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.

2. Java system properties-aws.accessKeyId and aws.secretKey.

3. Web Identity Token credentials from the environment or container.

4. The default credential profiles file- typically located at ~/.aws/credentials (location can vary per platform), and shared by many of the AWS SDKs and by the AWS CLI.

5. Amazon ECS container credentials- loaded from the Amazon ECS if the environment variable AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is set.

6. Instance profile credentials- used on EC2 instances, and delivered through the Amazon EC2 metadata service.

If I understand correctly, the default credential provider chain will do everything for me and no credential provider needs to be mentioned. Is that correct?

CodePudding user response：

Instead of using a Credential file under .aws, try setting EC2 environment variables. This is a valid cred provider and is described as Environment variables provider in the AWS SDK for Java V2 DEV Guide.

To use this provider with a AWS Java v2 Service client, you use:

Region region = Region.US_EAST_1;
     RdsDataClient dataClient = RdsDataClient.builder()
               .credentialsProvider(EnvironmentVariableCredentialsProvider.create())
                .region(region)
                .build(); 

This solution works nicely.

CodePudding user response：

Don't use IAM User credentials on EC2 unless you absolutely have to. Instead, launch the EC2 instance with an appropriate IAM Role (or edit the instance later to apply an IAM Role).

The SDK will then automatically retrieve credentials and they will be auto-rotated for you (unlike IAM User credentials, which are fixed and hence your exposure is greater). You don't need to indicate which credentials provider to use. The SDK does this for you.

And, it should go without saying, always give the IAM Role the minimum set of permissions that it actually needs.