I am looking into dynamo db streaming options. There are 2:
- Dynamo streams
- Kinesis streams
Dynamo streams can have a lambda trigger that will consume the stream.
Kinesis streams can also have the lambda trigger that will consume the stream.
In both above cases, in the lambda settings we can choose the batch size.
In dynamodb, what is the difference between using dynamo streams lambda trigger versus kinesis streams lambda trigger?
CodePudding user response：
Below is some of the key differences between Kinesis Data Streams and DynamoDB Streams. In regards to your question around a Lambda consumer, the important details to you is data retention, ordering, de-duplication and number of allowed consumers.
| Properties | Kinesis Data Streams for DynamoDB | DynamoDB Streams |
|---|---|---|
| Data retention | Up to 1 year. | 24 hours. |
| Kinesis Client Library (KCL) support | Supports KCL versions 1.X and 2.X. | Supports KCL version 1.X. |
| Number of consumers | Up to 5 simultaneous consumers per shard, or up to 20 simultaneous consumers per shard with enhanced fan-out. | Up to 2 simultaneous consumers per shard. |
| Throughput quotas | Unlimited. | Subject to throughput quotas by DynamoDB table and AWS Region. |
| Record delivery model | Pull model over HTTP using GetRecords and with enhanced fan-out, Kinesis Data Streams pushes the records over HTTP/2 by using SubscribeToShard. | Pull model over HTTP using GetRecords. |
| Ordering of records | The timestamp attribute on each stream record can be used to identify the actual order in which changes occurred in the DynamoDB table. | For each item that is modified in a DynamoDB table, the stream records appear in the same sequence as the actual modifications to the item. |
| Duplicate records | Duplicate records might occasionally appear in the stream. | No duplicate records appear in the stream. |
| Stream processing options | Process stream records using AWS Lambda, Kinesis Data Analytics, Kinesis data firehose , or AWS Glue streaming ETL. | Process stream records using AWS Lambda or DynamoDB Streams Kinesis adapter. |
| Durability level | Availability zones to provide automatic failover without interruption. | Availability zones to provide automatic failover without interruption. |