Imagine I have a message queue on to which I place messages saying that I want to send a user an email.

If I’m using a message broker that provides “at least once” delivery guarantees, then from all the resources I’ve been reading, they say “you must make sure your processing is idempotent”.

However, in the case of a side effect like sending an email, I don’t see how this is possible.

I have two possible choices:

• Store the message ID in my database, then send the email.
• Send the email, then store the message ID in the database.

When a message is received, I would then check the database to see if the message ID exists. If it does, I would skip the message as a duplicate.

However, the first case leaves me with “at most once” semantics on sending my email (if sending the email fails, it will be skipped next time the message is seen), and the second case gives me “at least once” semantics (if storing the ID in the database fails, I’ll end up sending multiple emails).

Some things I have read say “you need an email API that supports idempotency”, but as far as I can tell that just pushes the problem on to their servers - they still have the same dilemma.

Am I missing something here? Or is it just not possible to have idempotent message processing when that processing has external side effects?

CodePudding user response：

I am going to claim that email inherently does not support exactly-once semantics. When SMTP protocol is used, your side can always crash in between the far side confirms and you persist the confirmation. Your best bet is to use at-least once semantics.

But you may get exactly-once UX experience. If every time you send/resend a message you use the same Message ID, then the client side (mail client) may de duplicate those messages. See https://en.wikipedia.org/wiki/Message-ID for details.