In AWS IoT I can create a rule to route incoming mqtt payload to be processed by a lambda. Similarly I can write a lambda to send an mqtt message to a device (outgoing). My questions:
- Is there a risk that the IoT Core may never be able to hand over the incoming message to the lambda for any reason?
- Similarly is there a risk that a lambda was never able to send the outgoing mqtt message to the device?
In other words, how do I guarantee (or architecturally come up with solutions that at least increase delivery reliability without going overboard) a lambda receiving/sending mqtt messages? Is that why one would use SQS? We put the outgoing message we want to send in as SQS topic. Lambda receives it and sends the message with at least once delivery option in the mqtt protocol. If for whatever reason I do not receive the ACK I rollback the dequeue from SQS or put it in a different 'to be processed later' queue (or DLQ)?
CodePudding user response：
AWS dropping messages between services should not be keeping you up at night if we are talking about a non-hyperscale IoT application. An IoT Core -> Lambda setup is perfectly fine. AWS services are very reliable. The architecture gods will not frown on you for this choice.*
Before we shout YAGNI and go home, though, we should acknowledge that there good reasons for normal-sized applications to queue messages (IoT Core -> SQS -> Lambda).
SQS queues have lots of bells and whistles, but for mere mortals probably not must-haves. Instead, the case for normal-sized MVP applications to hop on the SQS train early boil down to two reasons:
- Error Handling Convenience: Let's face it, we should probably worry more about our own screw-ups causing lambda errors than AWS errors. Queues give us pause and replay capability in case our lambda code fails. Errors show up in the DLQ, not in logs.
- Cheap: SQS is not brain surgery, it is neither complex or costly. So why not?
* If you are keen to worry about reliability, backups and cross-zone redundancy are likely better things to invest in.