Home > Enterprise >  Is there a way to access nested json objects in BigQuery when the object name changes?
Is there a way to access nested json objects in BigQuery when the object name changes?

Time:07-02

If I have a json object and I wanted to access to most nested elements, but I don't know the name of a preceding object, is there a function that can do this?

CodePudding user response:

Consider below approach

create temp function  extract_keys(input string) returns array<string> language js as """
  return Object.keys(JSON.parse(input));
  """;
create temp function  extract_values(input string) returns array<string> language js as """
  return Object.values(JSON.parse(input));
  """;
create temp function extract_all_leaves(input string) returns string language js as '''
  function flattenObj(obj, parent = '', res = {}){
    for(let key in obj){
        let propName = parent ? parent   '.'   key : key;
        if(typeof obj[key] == 'object'){
            flattenObj(obj[key], propName, res);
        } else {
            res[propName] = obj[key];
        }
    }
    return JSON.stringify(res);
  }
  return flattenObj(JSON.parse(input));
''';
select json, 
  array(
    select as struct key, value 
    from unnest(extract_keys(leaves)) key with offset
    join unnest(extract_values(leaves)) value with offset
    using(offset)
  ) most_nested_elements
from your_table, unnest([struct(extract_all_leaves(json) as leaves)])    

if applied to sample data in your question

enter image description here

output is

enter image description here

CodePudding user response:

Not general solution like @Mikhail's answer but with some assumption you can also use a regular expression to extract most nested fields.

WITH sample AS (
  SELECT '{"field": {"properties": {"custom_name": {"type":"string", "label":"custom_name"}}}}' AS json
)
SELECT TRIM(SPLIT(kv, ':')[OFFSET(0)], '"') AS key,
       TRIM(SPLIT(kv, ':')[OFFSET(1)], '"') AS value
  FROM sample, UNNEST(REGEXP_EXTRACT_ALL(json, r'(\"\w \":\s*[^{][\w\"] )')) kv
;

enter image description here

  • Related