I know SQS has the maxReceiveCount and once the message has been consumed and failed this amount of times, then the message is moved to the DLQ. That's working currently.

However, there are certain errors which I would want to go back in my source queue indefinitely e.g. if we get a 5XX error from an API we are invoking. This is very likely to be temporary, and so we would want to retry these until they are successful.

Now if these 5XX failed messages do go to the DLQ, it's not the end of the world, but it requires a manual redrive.

Is there a known solution or aws feature which allows for conditionally moving messages to the DLQ depending on the error received?

I'm not sure this is possible. I've read the documentation and can't find anything that hints at how to achieve this.

CodePudding user response：

Since you didn't provide much detail about the way you process the message, I'm assuming you are using a lambda that would make the call against that API you mentioned.

If that's the case, based on the response code, you could decide to put back the message in the queue (as if it is a new message) and return success so that the message is not duplicated.

I'd also recommend to have a custom counter as part of your message to not retry infinite number of time and be stuck in a situation where your lambda process the same message over and over.

A preferable approach would be to set Maximum receives to 1000 for DLQ. You could throw an error based on the response code in your lambda and let SQS moving the message to the DLQ once it reached 1000. 1,000 it is quite a lot of retries..