I am looking for suggestions on how to validate the glue job and lambda function status, like succeeded or failed in the last run, programmatically using BOTO3 Python. We have 10 lambda function and glue jobs that run once a day; how can this be achieved using the Python script? Any lead would be highly appreciated.

I thought of using cloud watch logs to trigger the alert, but I have been tasked with doing this programmatically.

CodePudding user response：

For Glue you would need to first loop over all of the Job Runs by calling GetJobRuns and parse out which ones you want to know more info about, then call GetJobRun:

1. get_job_runs | Obtain JobName and RunId to pass to next API call
2. get_job_run | Check status of JobRunState

The same thing goes of Lambda, first call ListFunctions then call GetFunction.

However, I would strongly advise you to use a DynamoDB table to record the status of all your runs. When your Glue or Lambda runs complete, create an item for each run with the status of the job. Then all you need to do is get that days data from DynamoDB.