I have a AWS project that contains a S3 bucket, RDS database and Lambda functions. I want Lambda to have access to both the S3 bucket and the RDS database. The Lambda functions connects to the RDS database correctly but it times out when trying to retrieve an object from the S3 bucket:

Event needs-retry.s3.GetObject: calling handler <bound method S3RegionRedirectorv2.redirect_from_error of <botocore.utils.S3RegionRedirectorv2 object at 0x7f473a4ae910>>
...
(some more error lines)
...
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "https://{bucket name}.s3.eu-west-3.amazonaws.com/{file name}.tar.gz"

So I understand that the reason would be that Lambda doesn't have internet access and therefor my options are:

1. VPC endpoint (privatelink): https://aws.amazon.com/privatelink
2. NAT gateway for Lambda

But both go over the cloud (in same region), which doesn't make any sense as they are both in the same project. It's just a redundant cost for such a detail and there must be a better solution right?

CodePudding user response：

Maybe it helps you think of the S3 bucket "in the same project" as having permission to use an object system that resides in a different network outside your own. Your lambda is in VPC but S3 objects are not in your VPC. You access them using either public end-points (over the internet) or privately by establishing S3 Gateway endpoint or VPC Interface Endpoint. Neither uses public internet.

As long as you are staying in the same region, S3 gateway endpoint actually does not cost you money but if you need to cross regions, you will need to use VPC Interface endpoint. The differences are documented here: https://docs.aws.amazon.com/AmazonS3/latest/userguide/privatelink-interface-endpoints.html

If you are trying to avoid costs, S3 gateway might work for you, however, you will need to update your route tables that's associated with the gateway. The process is documented here: https://docs.aws.amazon.com/vpc/latest/privatelink/vpc-endpoints-s3.html