I had a DynamoDB with 6 attributes (integers and strings), created an Athena data source through federated queries and everything was fine.

Recently, I added a List-type attribute to the DynamoDB table. The 7 attributes appear successfully when I use DynamoDB's built-in query. However, when I try to query DynamoDB from Athena, I find the original 6 columns but not the new List attribute (tried fetching the last few rows that contain the new attribute, but still).

I tried creating another data source and another lambda function, but I still can't get the new attribute when I select *, and select "new_attribute_name" returns a COLUMN_NOT_FOUND.

I considered dropping the entire table and starting a fresh one with the 7 attributes. But I wanted to ask here first before doing it. How do I get Athena to see the new DynamoDB attribute?

CodePudding user response：

When the Athena connector does a discover for the schema it only samples 5 items AFAIR. So if you've added an extra attribute to only some items but not all, then it will not appear in your queries as it's not considered a part of the schema. To overcome it, you can manually create your table in the Glue Catalog and use that in Athena.