How to send all the results from a lambda function in one email using aws sns-CodePudding

I have a lambda script that extract all the filenames in a bucket that were uploaded more than one hour before and send an email via sns. My lambda function works well. However instead of receiving one mail containing all the files, I receive one email for each file. This means that when I have 100 files found by the lambda, I receive 100 emails. Is there a way to group all the files and just receive one email from sns ?

Here is my lambda sample script:

import boto3
import json 
from datetime import datetime, timedelta, timezone 


def lambda_handler(event, context):
     AWS_REGION = "aws-region"
     sns_resource = boto3.resource('sns', region_name=AWS_REGION)
     TOPIC_ARN = 'sns_topic_arn'
     sns_topic = sns_resource.Topic(TOPIC_ARN)
     s3_client = boto3.client('s3')
     paginator = s3_client.get_paginator('list_objects_v2')
     page_iterator = paginator.paginate(
        Bucket = 'bucket',
        Prefix = 'data/raw/mdf/'
    )
     
     for page in page_iterator:
         for object in page['Contents']:
             if object['LastModified'] < datetime.now(tz=timezone.utc) - timedelta(hours=2):             
                 message =  object['Key']
                 
             sns_topic.publish(Message=message)

any help is welcome

Best regards

CodePudding user response：

If you do not want to get a message for every file found, you should not publish a message for every iteration in your for loop.

So the following line should be outside of the for loop:

sns_topic.publish(Message=message)

Within the for loop you only create the message. After the for loop you publish it.

Example:

keys = []

for page in page_iterator:
    for object in page['Contents']:
        if object['LastModified'] < datetime.now(tz=timezone.utc) - timedelta(hours=2):             
            keys.append(object['Key'])
                
message = ",".join(keys)
sns_topic.publish(Message=message)

This will create a list of keys and then create a message that is just those keys separated by a comma (,).

You could create a message where every key is on a new line. This is up to you.

Beware: this does not scale. If you have thousands of files that have been created, the email won't help much.

You might be better off using AWS SES and sending an email with a file as attachment. The file contains a list of new keys.