Rows not being found in Lambda function reading CSV file-CodePudding

I have a Lambda function that is reading a CSV file, and each row is added to a DynamoDB table. I am using a print statement to print every row in the CSV to logs in CloudWatch. There is a problem here as only 51 of the 129 rows are being printed.

Also, only a small amount of the rows that are actually found are being added to the DynamoDB tables.

Lambda Function:

# ChronojumpDataProcessor Lambda function
#
# This function is triggered by an object being created in an Amazon S3 bucket.
# The file is downloaded and each line is inserted into DynamoDB tables.

from __future__ import print_function
import json, urllib, boto3, csv

# Connect to S3 and DynamoDB
s3 = boto3.resource('s3')
dynamodb = boto3.resource('dynamodb')

# Connect to the DynamoDB tables
athleteTable = dynamodb.Table('Athlete');
countermovementTable = dynamodb.Table('CMJ');
depthTable = dynamodb.Table('DepthJump');

# This handler is executed every time the Lambda function is triggered
def lambda_handler(event, context):

    # Show the incoming event in the debug log
    #print("Event received by Lambda function: "   json.dumps(event, indent=2))

    # Get the bucket and object key from the Event
    bucket = event['Records'][0]['s3']['bucket']['name']
    key = urllib.parse.unquote_plus(event['Records'][0]['s3']['object']['key'], encoding='utf-8')
    localFilename = '/tmp/session.csv'

    # Download the file from S3 to the local filesystem
    try:
        s3.meta.client.download_file(bucket, key, localFilename)
    except Exception as e:
        print(e)
        print('Error getting object {} from bucket {}. Make sure they exist and your bucket is in the same region as this function.'.format(key, bucket))
        raise e

    # Read the Session CSV file. Delimiter is the ',' character
    with open(localFilename) as csvfile:
        reader = csv.DictReader(csvfile, delimiter=',')

        # Read each row in the file
        rowCount = 0
        for row in reader:
            rowCount  = 1

            # Show the row in the debug log
            print(row['athlete_id'], row['athlete_name'], row['jump_id'], row['date_time'], row['jump_type'], row['jump_tc'], row['jump_height'], row['jump_RSI'])

            # Insert Athlete ID and Name into Athlete DynamoDB table
            athleteTable.put_item(
                Item={
                    'AthleteID':       row['athlete_id'],
                    'AthleteName':     row['athlete_name']})

            # Insert CMJ details into Countermovement Jump DynamoDB table
            if ((row['jump_type'] == "CMJ") | (row['jump_type'] == "Free")) :
                countermovementTable.put_item(
                    Item={
                        'AthleteID':           row['athlete_id'],
                        'AthleteName':         row['athlete_name'],
                        'DateTime':            row['date_time'],
                        'JumpType':            row['jump_type'],
                        'JumpID':               row['jump_id'],
                        'Height':              row['jump_height']})
            else :
                # Insert Depth Jump details into Depth Jump DynamoDB table
                depthTable.put_item(
                    Item={
                        'AthleteID':            row['athlete_id'],
                        'AthleteName':          row['athlete_name'],
                        'DateTime':             row['date_time'],
                        'JumpType':             row['jump_type'],
                        'JumpID':               row['jump_id'],
                        'ContactTime':          row['jump_tc'],
                        'Height':               row['jump_height'],
                        'RSI':                  row['jump_RSI']})

                # Finished!
                return "%d data inserted" % rowCount

I added a Timeout to the Lambda function of 2 minutes as I thought that maybe there wasn't enough time provided for the function to read each row, but that did not fix the problem.

CodePudding user response：

Your return statement is indented under the else, which means that the function will exit as soon as the if evaluates to False.

It should be indented to match the indent use on the with line.