I'm working on an iiot based monitoring solution and working with aws services. I'm batching the data received from iot core in this flow: -

iot core -> rules (to firehose delivery stream) -> kinesis firehose (900 second buffer) -> s3 bucket

the s3 prefix is as follows:-

partitionKey={partitionKeyFromQuery:device_name}/year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/hour=!{timestamp:hh}

but the issue with this flow is it makes the folder structure in s3 bucket as follows: -

partitionKey=wind-mill-01/year=2023/month=01/day=08/hour=08 (logs hour in utc time)

I want to log the time in ist format for the "hour" field is there any way possible to do it?

Any help will be greatly appreciated.

CodePudding user response：

Firehose only support UTC as mentioned in the documentation. If you want to have it in IST for some reason, you can have a lambda (or glue job) moving object into another path with the hour in IST.