Home > Enterprise >  How to uppload a large file using Lambda, when presigned URL's are not an option
How to uppload a large file using Lambda, when presigned URL's are not an option

Time:04-06

I need to develop a public API for our application. It's running in AWS Lambda and uses S3 as storage. Now the problem is, that this public API should be able to handle file upploads. Files can be bigger then 6mb, so out of Lambda limits.

Using presigned URL's with direct file uppload to S3 is a good option for our internal use, but in case of public API, it may be confusing for the consumers.

It's not so clear for services, that will use the API, why should they do a call to receive presigned url, uppload the file to this url and do one more confirmation request to our API.

I'd like to make it as clear as possible for the consumers, but have no idea how to handle it.

The only one thing I could invent is creating a proxy EC2 instance. The API url will lead to this instance and it will handle the request:

  1. If it's a simple request to store json data, it will directly call the endpoint in our application to store it in DB
  2. If it's a file uppload request, it will uppload the file to S3 and then call an application endpoint, so application can handle the file uppload request and return a response

But that's very tricky way, so I'm thinking on some simpler options

CodePudding user response:

Actually, you can use Lambda with up to 10GB of Ephemeral Storage now: https://docs.aws.amazon.com/lambda/latest/dg/configuration-function-common.html#configuration-ephemeral-storage

CodePudding user response:

Direct Api Gateway to S3 integration is one way how to do it:

https://aws.amazon.com/premiumsupport/knowledge-center/api-gateway-upload-image-s3/

This is the part of the document which mentions the integration.

Now, Select Integration Request and select AWS Service under the Integration type category. For AWS Region, choose us-west-2. For AWS Service, choose Simple Storage Service (S3).

  • Related