S3 object creation trigger event to lambda-CodePudding

I have created an AWS lambda trigger from S3 - create object notification. When lambda gets object creation notification it also launches the ec2 instance. For the next step, I want ec2 to access the object/data form s3 and do some processing on that data. How can I access that data from ec2?

CodePudding user response：

When the Lambda function launches the EC2 instance, I would have it pass the S3 object key as part of the EC2 user-data.

CodePudding user response：

It appears that you are wanting to use an existing Amazon EC2 instance to process objects that are uploaded to an Amazon S3 bucket.

The standard method for doing this is:

Create an Amazon SQS queue
Configure the Amazon S3 bucket to send a message to the SQS queue when a new object is created
Write code on the Amazon EC2 instance that continuously polls the Amazon SQS queue, asking for a message (use WaitTimeSeconds=20 to reduce the number of calls, it will return immediately if a message is available)
Once a message is received, the code can download the S3 object (the Bucket and Key are provided in the message) and then process the file. Once the file is processed, your code should delete the SQS message

If there are long time-periods between object uploads and you wish to save money, you could add some logic to your code that does a Shutdown of the instance when no messages have been received for a given period of time. You could then create an Amazon CloudWatch alarm based on queue size that can trigger an AWS Lambda function to Start the instance if it is not currently Running.

To run a script every time that an Amazon EC2 instance starts, you can place it in /var/lib/cloud/scripts/per-boot/.