JQ Library: Overriding existing keys of an JSON object with another object keys-CodePudding

I am new to the JQ Library and I cannot figure out how to replace values in config_new.json with values from keys that exist in both config.json and config_new.json, recursively, without copying any other attributes from config.json.

Basically having:

// config_new.json
{
  "name": "testName",
  "age": "tooOld",
  "properties": {
      "title": "Mr",
      "fruits": ["apple", "banana"]
  },
}

// config.json
{
  "newName": "changedName",
  "age": "tooYoung",
  "properties": {
      "title": "Mr",
      "height": "tooTall",
      "fruits": ["banana"]
  },
  "certificate": "present"
}

// expected result
{
  "name": "testName",
  "age": "tooYoung",
  "properties": {
      "title": "Mr",
      "height": "tooTall",
      "fruits": ["banana"]
  },
}

So I am trying to override the values in config_new.json only with known values in config.json.

I have tried using

jq -s '.[0] * .[1]' config_new.json config.json

but this works only partially, because it also copies the key-value pairs that do not exist in config_new.json:

{
  "name": "testName",
  "newName": "changedName", // This should not be here
  "age": "tooYoung", // This was replaced from config.json
  "properties": {
      "title": "Mr",
      "height": "tooTall", // This should not be here
      "fruits": ["banana"]
  },
}

Could someone help me?

CodePudding user response：

I'd like to propose something, but I'm not sure it is something that works for your requirements: don't merge JSON documents, but write your "target" document as a jq program itself.

config_new.jq:

{
  "name": .name,
  "age": .age,
  "properties": {
      "title": .properties.title,
      "fruits": .properties.fruits
  }
}

This reads almost like "real" JSON.

Or if you want to reduce duplication:

{
  name,
  age,
  properties: .properties | {
      title,
      fruits
  }
}

and then migrate your old file to the new format:

jq -f config_new.jq config.json > config_new.json

A "copy values from the same keys from a different document" approach would be more complicated, but let's wait for other answers. I'm pretty sure there's a way, but I'm too dumb for it :) It probably involves reduce and path/getpath/setpath in some way.

CodePudding user response：

Here's a jq answer that recursively merges objects according to your criteria:

jq -s '
    def merge($source):
        . as $target
        | reduce ($source | keys | map(select(in($target))))[] as $key ($target;
            .[$key] = if (.[$key] | type) == "object"
                          then .[$key] | merge($source[$key])
                          else $source[$key]
                      end
        )
    ;

    . as [$new, $old]
    | $new | merge($old)
' config_new.json config.json

outputs

{
  "name": "testName",
  "age": "tooYoung",
  "properties": {
    "title": "Mr",
    "fruits": [
      "banana"
    ]
  }
}

CodePudding user response：

Alright, here is the solution I figured would be possible. Please leave a comment if there is room for improvement.

$ jq --slurpfile cfg config.json '. as $new
| reduce (paths(scalars,arrays) | select(any(numbers)|not)) as $path (
  {};
  setpath($path; ($cfg[0]|getpath($path))//($new|getpath($path))))' config_new.json
{
  "name": "testName",
  "age": "tooYoung",
  "properties": {
    "title": "Mr",
    "fruits": [
      "banana"
    ]
  }
}

It reduces by iterating over all paths that do not contain an array. Breakdown:

. as $new # store original "new" json
| reduce (paths(scalars,arrays) | select(any(numbers)|not)) as $path ( # reduce over all paths of input (original "new" json) that are leaf paths or arrays (but not array elements)!
  {}; # start with an empty result
  setpath( 
    $path; # set "$path" in the result
    ($cfg[0]|getpath($path)) # query value from existing config (read via slurpfile)
    // ($new|getpath($path)))) # if value does not exist or is null, use existing value (from original "new" json)

Note that a value of "null" or "false" in config.json will not overwrite the existing value in config_new.json. To handle null and false, you need to be smarter:

. as $new
| [$cfg[0] | paths] as $cfg_paths # store all paths from config.json
| reduce (paths(scalars,arrays) | select(any(numbers)|not)) as $path (
  {};
  setpath(
    $path;
    if $path|IN($cfg_paths[]) # check if $path exists in config
    then $cfg[0] else $new # use old or new config
    end | getpath($path) # get $path from old/new config respectively
  )
)

The final if can also be expressed with a select filter:

. as $new
| [$cfg[0] | paths] as $cfg_paths
| reduce (paths(scalars,arrays) | select(any(numbers)|not)) as $path (
  {};
  setpath(
    $path;
    $cfg[0] | select($path|IN($cfg_paths[])) // $new # $cfg if path exists, otherwise $new (equivalent to "if" above)
    | getpath($path)
  )
)