I am having a python code in AWS Lambda which is triggered based on sqs event generated. The criteria for generating sqs is if a new file comes into a particular S3 location, then sqs will be created and which in turn calls lambda. Right now, lambda is processing the files one after the other in a serial mode. But I would like to know if we can process multiple files at the same time.
Example: If 5 files comes to s3 location, all the 5 files should be processed parallely at the same time.
CodePudding user response：
I think you might miss observed the behavior of your system. If you using the Native SQS Standar Queue with Lambda integration, the Lambdas will consume the queus in batches, you can see a detailed explanation here:
CodePudding user response：
No need to add the SQS.
Enable triggering from the S3 PutObject action to the Lambda. With this, you can ensure invocations per object and also parallelism.
CodePudding user response：
Also, check your Reserve concurrency value
Doc:https://docs.aws.amazon.com/lambda/latest/dg/configuration-concurrency.html
Check the concurrency dashboard in the monitoring sub section, it is already running in parallel mode.
File may look
Hope this helps!!!