I am trying to implement a fanout architecture using AWS' SNS and SQS. There will be a FIFO topic and several FIFO queues subscribed to it each having its own filtering policies just like in this article https://betterprogramming.pub/how-to-fan-out-to-different-sqs-queues-using-sns-message-filtering-84cd23ed9d07
In the article above he has a catch-all queue which I want to avoid. Lets assume I have a topic T and two queues Q1 and Q2 subscribed to it. Q1 accepts "dogs" and Q2 accepts "cats".
My questions are:
- If someone publishes a message with a horse what would happen?
- How can I handle a message that is not being picked up by any of the queues? Will SNS DLQ get the horse message eventually or will it be discarded?
Thank you
CodePudding user response：
When a message is sent to an Amazon SNS Topic, it is then sent to all subscribers. If there are no subscribers on the queue, then nobody will receive the message.
From Amazon SNS subscription filter policies - Amazon Simple Notification Service:
A subscription accepts a message under the following conditions:
Each attribute name in a filter policy matches an attribute name assigned to the message.
For each matching attribute name, at least one match exists between the following:
- The values of the attribute name in the filter policy
- The message attributes
Thus, if no subscription accepts the message, then it is the same as having no subscribers -- nobody will receive the message.
Please note that messages are not "picked up by the SQS queues". Rather, the SNS Topic forwards a copy of the message to the subscribers. Whether or not one subscriber is sent the message has no impact on whether other subscribers are sent the message.
From Amazon SNS dead-letter queues (DLQs) - Amazon Simple Notification Service:
A dead-letter queue is an Amazon SQS queue that an Amazon SNS subscription can target for messages that can't be delivered to subscribers successfully. Messages that can't be delivered due to client errors or server errors are held in the dead-letter queue for further analysis or reprocessing.
Thus, Amazon SNS Dead Letter Queues are only used for messages that have a delivery failure. They are not used for 'messages that no subscriber wanted'.