We have a Lambda event source that polls SQS for messages and passes them on to a Lambda function.

The SQS has a Visibility Timeout that dictates how long after a consumer (lambda) picks up a message it will re-appear in the queue if not successfully completed.

Is there a way from the Lambda function to force this to happen as soon as we detect an error?

For example, say Lambda has a timeout of 10 mins. The SQS Visibility Timeout will need to be longer than that.

But if the Lambda function detects a problem early on, and throws an error - this will fail the message, but we have to wait for the SQS Visibility Timeout before the message is available for other consumers to try again.

Is there a way for the Lambda function to tell SQS that the message has failed and it should go back on the queue immediately ?

CodePudding user response：

If you're using the built-in AWS Lambda SQS integration, then simply throwing an exception (returning a non-success status from the Lambda function) will result in all the SQS messages in the event of that invocation being placed back in the queue immediately.

From the documentation:

When Lambda reads a batch, the messages stay in the queue but become hidden for the length of the queue's visibility timeout. If your function successfully processes the batch, Lambda deletes the messages from the queue. If your function is throttled, returns an error, or doesn't respond, the message becomes visible again. All messages in a failed batch return to the queue, so your function code must be able to process the same message multiple times without side effects.

CodePudding user response：

Call the ChangeMessageVisibility API and set the visibility timeout of each failed message to 0 seconds. Then throw an exception.

If you received a batch of SQS messages and were able to successfully process some of them, then explicitly delete those (successful) messages from the underlying SQS queue, then set the visibility timeout of each remaining failed message to 0 seconds, then throw an exception.

Doing this will make the failed messages immediately visible to other consumers.