Home > Enterprise >  Athena Output files are Missing in S3 Landing Bucket Even Though PutObject Permissions are Set
Athena Output files are Missing in S3 Landing Bucket Even Though PutObject Permissions are Set

Time:10-07

After checking my permissions, roles, and policies I would suspect that I have the permissions to write to the Athena output locations in S3 but for some reason whenever a file lands on an input S3 bucket to trigger the Athena query to run a large query within my Lambda python integration, I notice that:

  • I don't get any type of HTTP return code from Athena in AWS Cloud Watch although the code runs without errors.
  • I don't get any CSV files located in the Athena S3 output bucket.
  • When I test the query inside of the Athena console, it displays the correct output.

I am not sure why. I did do an Alter Table in Athena to make sure that the tables point to the correct output location too. Here are the following screen scrapes of the code, permissions, and policies: (Note that in the last two screen scrapes the client_name variable cotains one of 2 different tables that Athena will use as a query. The variable athena_output_bucket is a global variable which has been previously set as default (i.e. athena_output_bucket = "s3://prod-cog-kahala-test-output/baja/"). Thus it toggels based on the name of the input file that was dropped on S3 input bucket. Thanks much for all the help. Policies Policies

enter image description here

enter image description here

enter image description here

CodePudding user response:

OK I found out why there was no data being sent to the output S3 locations. I accidentally Altered the Table to be set to the S3 Output location instead of the S3 data input location by mistake and didn't realize it.

  • Related