I have a lambda function which triggers a glue job to start running whenever a file is uploaded to s3. The glue job then processes the said file.

This works perfectly, but i'm wondering what will happen if another file is uploaded when the glue job is still processing the first one. Will it cause an error, will it be ignored, or will it just wait for the first one to finish, then move onto the second one?

CodePudding user response：

It depends on GLue job settings that you have in place. If you have set concurrency value by setting Max concurrency, then the lambda will trigger "another version" of glue job for that new file. You can read about it here.