I am trying to determine the best way to access the CloudWatch data I am able to display in a dashboard on an EC2 instance and pull it into a Rails app. I've been told that one approach is to define a Lambda function to parse the data from CloudWatch logs and then to access the Lambda function in an API. Most of what I read in the AWS documentation relates to CloudWatch accessing Lambda and not vice-versa.
What is the best solution here? Does Lambda really add value here?
Thank you in advance
CodePudding user response：
My recommendation is to use cloudwatch get-metric-statistics and get the statistics you need on demand. This will provide aggregated results and not actual data points, but this is probably what you need.
If you definitely need the data points, you could export the Cloud Watch logs to an S3 bucket. You will need to set up a lambda function to do this periodically. You would probably need another lambda function to do the some processing. Data in S3 could be queried using Amazon Athena or ingested into a Redshift Database.
If you need real time data, you could set up a Kinesis Stream out of your CloudWatch logs.
Unless you have very specific requirements, both these options look an over-design to me.
CodePudding user response：
The main idea of using AWS lambda in one of the scenarios is to run custom analysis on the extracted cloudwatch logs and use output as per your requirement.
If an application requires some sort of analysis on the logs and you don’t want to run on a dedicated system (like EC2 or similar), then lambda is the best fit for it.