In a use case that precludes the use of SQS FIFO queues, duplicate messages are a possibility.

If a consumer receives a duplicate message that it has already deleted, does it need to delete the duplicate?

I'm guessing that the answer is no, since duplicates are a function of the distributed implementation of SQS and assume that the delete action is just in the process of propagating through the AWS network, but is there any explicit documentation confirming this?

CodePudding user response：

If a consumer receive such kind of message which is duplicate of a deleted message,it will not be deleted. Because it already get out from the queue.

For the fifo queue you have to provide MessageDeduplicationId in the send message parameter. And this id have to be unique.

So when a message with MessageDeduplicationId is already preserved (e.g. your consumer can return the message for any kind of error occured, one probability for visibility timeout) in your whole queue, the duplicate copy will be automatically deleted.

For sending message you can read this

For deduplication, learn more

CodePudding user response：

AWS gives you tools to do data deduplication [1], but it doesn't do it for you. There are scenarios within SQS where duplication is expected [2], so your consumer should be able to treat it as such.

[1] AWS has a resource for data deduplication in SQS: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/using-messagededuplicationid-property.html

[2] There is also a long conversation on this topic in stack overflow: How to prevent duplicate SQS Messages?