Home > database >  AWS Lambda using Python, get .json file data from S3 Bucket and put it into DynamoDB
AWS Lambda using Python, get .json file data from S3 Bucket and put it into DynamoDB

Time:06-18

This is the code I've made, and seemingly got the best result so far. I still get errors, but I got my best result so far with this.

I need to read three things from the .json file, which is ID, Name and Country using Python.

import json
import boto3

s3_client = boto3.client('s3')
dynamodb = boto3.resource('dynamodb')

table = dynamodb.Table('valid_cities')



def lambda_handler(event, context):
    bucket = event['Records'][0]['s3']['bucket']['name']
    print(bucket)
    
    json_file_name = event['Records'][0]['s3']['object']['key']
    print(json_file_name)
    
    json_object = s3_client.get_object(Bucket=bucket,Key=json_file_name)
    print(json_object)
    
    file_reader = json_object['Body'].read().decode("utf-8")
    print(file_reader)
    
    jsonDict = json.loads(file_reader)
    print(jsonDict)
    

    table.put_item(Item={'id':{'N':id},'name':{'S':name},'country':{'S':country}})

I've gotten this far, it reads the json file in the S3 bucket, and processes it. Now I'm being met with an error that states the following:

{
  "errorMessage": "name 'name' is not defined",
  "errorType": "NameError",
  "requestId": "c2f72151-9591-4f35-909b-6b7876b4253a",
  "stackTrace": [
    "  File \"/var/task/lambda_function.py\", line 28, in lambda_handler\n    table.put_item(Item={'id':{'N':id},'name':{'S':name},'country':{'S':country}})\n"
  ]

This is a piece of the sample data in the citylist.json file:

{
    "city_list":[
        {
            "id": 6951112,
            "name": "Somerset West",
            "state": "",
            "country": "ZA",
            "coord": {
                "lon": 18.821131,
                "lat": -34.084011
            }
        },
        {
            "id": 3358975,
            "name": "Yzerfontein",
            "state": "",
            "country": "ZA",
            "coord": {
                "lon": 18.16157,
                "lat": -33.344379
            }
        }
]
}

CodePudding user response:

Like Anon Coward suggested but last line can be left plain:

import json
import boto3

s3_client = boto3.client('s3')
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('valid_cities')

def lambda_handler(event, context):
    bucket = event['Records'][0]['s3']['bucket']['name']
    json_file_name = event['Records'][0]['s3']['object']['key']
    json_object = s3_client.get_object(Bucket=bucket,Key=json_file_name)
    jsonDict = json.load(json_object['Body'])
    
    for city in jsonDict['city_list']:
        table.put_item(Item={'id': city['id'],'name': city['name'],'country':city['country']})
  • Related