I have a lambda configured to be triggered by a Kinesis stream at a batch size of 100 and batch window of 60s. The Kinesis stream has about 2 shards and 2K total records per minute. I'd imagine this would translate into about 20 lambda runs. However, every minutes, I'm seeing only an average of 5-6 runs.

I also see an iterationAge of 86K ms (approximately 24 Hours, my Kinesis stream holds records up to 24 hours)

The lambda function is not a fast one, runs approximately 25s on average.

I don't see any error count, or throttles or async delivery failures.

Appreciate any advice on how to troubleshoot this.

CodePudding user response：

I'd imagine this would translate into about 20 lambda runs

It does not work like that. If you have 2 shards, you will have only two concurrent lambda functions (one shard = one function), otherwise your records would be processed out of order.

runs approximately 25s on average

That's why you have about 5-8 lambda invocations per minute:.

2 shards x 4 invocations for each function within 1 minute

You would have to increase parallelization factor for your function. But this requires your records to have different partition keys within a shard. If they have same partition key, parallelization will not work.