File in s3 bucket comes with no values when running python script through lambda in aws-CodePudding

I have the following script but when the file lands in s3 it comes blank.

import json
import urllib.parse
from datetime import datetime
import boto3
import botocore.session as bc

print('Loading function')
client_s3=boto3.client('s3')

def run_athena(BUCKET_NAME, PREFIX):
    client=boto3.client('athena')
    database = 'data_lake'
    query ="SELECT buid, property_id, checkin_date, count(*) number_of_direct_bookings FROM data_lake.vw_qs_bookings_v1 where source_logic in ('sources_phone', 'sources_email', 'sources_website','sources_front_desk') and cancelled_flag=0 and checkin_date= date_add('day', -1, current_date) group by 1,2,3";
    s3_output = 's3://gainsight-file/gainsight/'
    response = client.start_query_execution(QueryString=query, QueryExecutionContext={'Database': database}, ResultConfiguration={'OutputLocation': s3_output})
   
  
    today = datetime.today().strftime('%Y-%m-%d')

    response_s3 = client_s3.list_objects(
        Bucket=BUCKET_NAME,
        Prefix=PREFIX,
    )
    name = response_s3["Contents"][0]["Key"]
    client_s3.copy_object(Bucket=BUCKET_NAME, CopySource=BUCKET_NAME '/' name, Key=PREFIX "vw_qs_bookings_v1_" today)
    client_s3.delete_object(Bucket=BUCKET_NAME, Key=name)
    

def lambda_handler(event, context):
    print("Entered lambda_handler")
    run_athena('gainsight-file', 'gainsight/')

I assume there is something to do with the query variable.

Thanks for the help.

CodePudding user response：

client.start_query_execution(…) only starts a query execution, it does not wait for it to complete (because that could be up to 30 minutes). You need to call client.get_query_execution(…) until you get a response that says the execution is successful (or failed). Only then is the output available.