In BigQuery I have a table storing 237 GB data. I don't have any columns on which i can create partition as it does not store any date fields When I am using it in the query the processing says 77 GB data will be processed but in bytes shuffled i see 7 GB data. what is the actual GB of data processed here? is there any way i could restructure this table ?
CodePudding user response:
BigQuery operates column-wise. If you only choose the columns you really need in a query then you're optimizing cost already. Traditionally databases operate row-wise, so this can be a bit counter-intuitive.
There's also this great blog article on optimizing for costs.