Home > Software design >  AWS: inconsistency between SQS and lambda
AWS: inconsistency between SQS and lambda

Time:03-26

I want to trigger lambda with a websocket. I have deployed a EC2 instance of websocket producer which is throwing all its data through SQS FIFO and SQS triggering lambda with same messageGroupId. But sometimes lambda is executing concurrently, I am expecting lambda to be executed sequentially. Because data is coming in a Queue. Since it is a cryptocurrency exchange websocket, data frequency is really high. And I checked one message from the websocket takes 3ms in lambda to get processed. I was expecting lambda to run only 1 process not concurrently (which is causing wrong data calculation). Can anyone tell me what config in Queue should I configure or is there any other method to achieve this goal. Thanks

Edit: Attaching config for fifo enter image description here

CodePudding user response:

There are two types of Amazon SQS queues: first-in, first-out (FIFO) and standard queues.

In FIFO queues, message strings remain in the same order in which the original messages were sent and received. FIFO queues support up to 300 send, receive or delete messages per second.

Standard queues attempt to keep message strings in the same order in which the messages were originally sent, but processing requirements may change the original order or sequence of messages. For example, standard queues can be used to batch messages for future processing or allocate tasks to multiple worker nodes.

The frequency of message delivery differs between standard and FIFO queues, as FIFO messages are delivered exactly once, while in standard queues, messages are delivered at least once.

Suggestion : check your que type and change it to FIFO.

CodePudding user response:

You need to set the maximum lambda concurrency to 1.

concurrency

  • Related