We are tying to format a json similar to this:

[
{"id": 1,
    "type": "A",
    "changes": [ 
        {"id": 12},
        {"id": 13}
    ],
    "wanted_key": "good",
    "unwanted_key": "aaa"
},
{"id": 2,
    "type": "A",
    "unwanted_key": "aaa"
},
{"id": 3,
    "type": "B",
    "changes": [
        {"id": 31},
        {"id": 32}
    ],
    "unwanted_key": "aaa",
    "unwanted_key2": "aaa"
},
{"id": 4,
    "type": "B",
    "unwanted_key3": "aaa"
},
null,
null,
{"id": 7}
]

into something like this:

[
  {
    "id": 1,
    "type": "A",
    "wanted_key": true  # every record must have this key/value
  },
  {
    "id": 12,  # note: this was in the "changes" property of record id 1
    "type": "A",  # type should be the same type than record id 1
    "wanted_key": true
  },
  {
    "id": 13,  # note: this was in the "changes" property of record id 1
    "type": "A",  # type should be the same type than record id 1
    "wanted_key": true
  },
  {
    "id": 2,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 3,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 31,  # note: this was in the "changes" property of record id 3
    "type": "B",  # type should be the same type than record id 3
    "wanted_key": true
  },
  {
    "id": 32,  # note: this was in the "changes" property of record id 3
    "type": "B",  # type should be the same type than record id 3
    "wanted_key": true
  },
  {
    "id": 4,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 7,
    "type": "UNKN",  # records without a type should have this type
    "wanted_key": true
  }
]

So far, I've been able to:

• remove null records
• obtain the keys we need with their default
• give records without a type a default type

What we are missing:

• from records having a changes key, create new records with the type of their parent record
• join all records in a single array

Unfortunately we are not entirely sure how to proceed... Any help would be appreciated.

So far our jq goes like this: del(..|nulls) | map({id, type: (.type // "UNKN"), wanted_key: (true)}) | del(..|nulls)

Here's our test code:

https://jqplay.org/s/eLAWwP1ha8P

CodePudding user response：

Here's one way:

map(
  select(values)
  | (.type // "UNKN") as $type
  | ., .changes[]?
  | {id, $type, wanted_key: true}
)

[
  {
    "id": 1,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 12,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 13,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 2,
    "type": "A",
    "wanted_key": true
  },
  {
    "id": 3,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 31,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 32,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 4,
    "type": "B",
    "wanted_key": true
  },
  {
    "id": 7,
    "type": "UNKN",
    "wanted_key": true
  }
]

Demo

CodePudding user response：

Something like below should work

map( 
  select(type == "object") | 
  ( {id}, {id : ( .changes[]? .id )} )   
  { type: (.type // "UNKN"), wanted_key: true }
)

jq play - demo

CodePudding user response：

The following should work:

map(select(values))
| map(., .type as $type | (.changes[]?   {$type}))
| map({id, type: (.type // "UNKN"), wanted_key: true})

1. Only select non-null values
2. Return the original items followed by their inner changes array ( outer type)
3. Extract 3 properties for output

Multiple map calls can usually be combined, so this becomes:

map(
    select(values)
    | ., (.type as $type | (.changes[]?   {$type}))
    | {id, type: (.type // "UNKN"), wanted_key: true}
)

Another option without variables:

map(
    select(values)
    | ., .changes[]?   {type}
    | {id, type: (.type // "UNKN"), wanted_key: true}
)
# or:
map(select(values))
| map(., .changes[]?   {type})
| map({id, type: (.type // "UNKN"), wanted_key: true})

or even with a separate normalization step for the unknown type:

map(select(values))
| map(.type //= "UNKN")
| map(., .changes[]?   {type})
| map({id, type, wanted_key: true})
# condensed to a single line:
map(select(values).type //= "UNKN" | ., .changes[]?   {type} | {id, type, wanted_key: true})