How can I avoid data transfer costs between S3 and EC2 across accounts?-CodePudding

I have some EC2 instances in a private subnet (as part of a larger EKS cluster) in Account A. There is a containerized application running on these EC2 instances that reaches out to S3 for some data to process. The S3 bucket is in Account B. Currently the application just uses plain Access Key and Secret Key (since its a legacy app we don't want to modify).

Currently I have a NAT gateway configured that allows the EC2 instances to access the internet. When the applications (within the Kubernetes pods) reaches out for files in S3, it goes through the NAT gateway, over the internet.

This incurs a significant cost for data transfer via the NAT gateway.

I've heard that VPC Gateway endpoints for S3 can help alleviate the problem by allowing access to go through the Amazon network, without using the internet. This can save money when it comes to data transfer costs. However, most blog posts seem to assume that the EC2 instances and the S3 buckets exist in the same account and region. In my case, they are in different AWS accounts.

I've also heard that VPC Interface Endpoints are an option. The documentation mentions they can be used across regions (but doesn't mention across accounts). More importantly, there is a cost associated with the data transfer, unlike with VPC Gateway endpoints.

Is this the appropriate tool to use for this case or is there an alternate way that I can access an S3 bucket from a separate AWS account while avoiding too many data transfer costs?

My question is similar to this one, but that question focuses on public IP addresses, and in my case all my EC2 instances are in a private subnet.

CodePudding user response：

An S3 vpc gateway endpoint is definitely what you want to be using. It doesn't matter if the S3 bucket you want to access is in another account, it will work. The gateway endpoint gives connectivity to the S3 service, so it doesn't matter where the bucket you want to access lives.

Do note that the gateway endpoint should be created in the same region as that where the S3 bucket lives.

CodePudding user response：

If the cost can be decreased if the EC2 instances and the S3 bucket is in the same account (I have not checked, but based on the question this seems to be the case), one option worth investigating is to setup S3 replication between the two accounts so that all S3 objects are copied to the EC2 account. The S3 pricing page has a replication tab that can be used as a guidance when comparing costs.