Home > Software engineering >  How to schedule ECS tasks in fan out architecture
How to schedule ECS tasks in fan out architecture

Time:07-05

I have a video which I want to upload to S3. After uploading the video I want to run three ECS tasks A,B,C (each tasks get the same video_id). Those ECS tasks upload many rows to Dynamodb tables (ECS task A uploads to Table A, B to B, C to C). Each row in the tables contains the video_id.

After tasks A,B,C finish, I want that task Z would run (task Z takes data from Tables A,B,C).

How do I trigger those three ECS tasks after uploading file to S3 in the simplest way, and how to trigger ECS task Z after A,B,C finishing uploading rows for video to tables A,B,C?

P.S.I prefer event driven solution.

enter image description here

CodePudding user response:

Create an AWS Step Function that has a parallel state, with three branches, with one state each, that invoke the ECS tasks A,B and C respectively.

Configure a final state in the step function to run task Z, and configure this state to depend on the previous parallel state.

Then, use S3 event notification to invoke the Step Function on S3 object uploads.

  • Related