Best way to iterate over JSON array with variable keys with jq/bash-CodePudding

I have a JSON-file like the one below:

{
    "Results": [
        {
            "Severity": "LOW",
            "InstalledVersion": "1.21.1ubuntu2",
            "FixedVersion": "1.21.1ubuntu2.1"
        },
        {
            "Severity": "LOW",
            "InstalledVersion": "1.21.1ubuntu2"
        },
        {
            "Severity": "HIGH",
            "VulnerabilityID": "CVE-2016-2781"
        }
    ]
}

I would like to create the following JSON-file:

[
    {
        "Severity": "LOW",
        "InstalledVersion": "1.21.1ubuntu2",
        "FixedVersion": "1.21.1ubuntu2.1",
        "VulnerabilityID": ""
    },
    {
        "Severity": "LOW",
        "InstalledVersion": "1.21.1ubuntu2",
        "FixedVersion": "",
        "VulnerabilityID": ""
    },
    {
        "Severity": "HIGH",
        "InstalledVersion": "",
        "FixedVersion": "",
        "VulnerabilityID": "CVE-2016-2781"
    }
]

I have tried iterating over the first JSON-file, but when I do so in the following way, I get random parsing errors (with the real JSON-file I am working with). So I think there is something off about the piping that I am doing.

jq -c '.Results[]' my.json | while read i; do
    echo $i;
done

In the JSON-file I am working with jq -c '.Results[]' works perfectly, but when I do jq operations on the resulting $i while iterating, I suddently get several parsing errors.

Is there better way to iterate over key-value-pairs in a JSON-file? Ofcause I could just install Python/R and do the typical drill, but I would really like for this to work in a shell script.

CodePudding user response：

You can use to add two objects together by combining their fields; if both objects have the same one the value of the right hand one is used. So if you make your existing object the RHS:

 jq '[ .Results[] | {Severity:"", InstalledVersion:"", FixedVersion:"", VulnerabilityID:""}   . ]' input.json   
[
  {
    "Severity": "LOW",
    "InstalledVersion": "1.21.1ubuntu2",
    "FixedVersion": "1.21.1ubuntu2.1",
    "VulnerabilityID": ""
  },
  {
    "Severity": "LOW",
    "InstalledVersion": "1.21.1ubuntu2",
    "FixedVersion": "",
    "VulnerabilityID": ""
  },
  {
    "Severity": "HIGH",
    "InstalledVersion": "",
    "FixedVersion": "",
    "VulnerabilityID": "CVE-2016-2781"
  }
]

CodePudding user response：

One more variation

jq '[
    .Results[]
    | .FixedVersion //= ""
    | .VulnerabilityID //= ""
    | .InstalledVersion //= ""
]' file.json

CodePudding user response：

Another way of adding the fallbacks to a empty string:

[ .Results[] | .   { VulnerabilityID: (.VulnerabilityID // ""), FixedVersion: (.FixedVersion // ""), InstalledVersion: (.InstalledVersion // "") } ]

[
  {
    "Severity": "LOW",
    "InstalledVersion": "1.21.1ubuntu2",
    "FixedVersion": "1.21.1ubuntu2.1",
    "VulnerabilityID": ""
  },
  {
    "Severity": "LOW",
    "InstalledVersion": "1.21.1ubuntu2",
    "VulnerabilityID": "",
    "FixedVersion": ""
  },
  {
    "Severity": "HIGH",
    "VulnerabilityID": "CVE-2016-2781",
    "FixedVersion": "",
    "InstalledVersion": ""
  }
]

Online demo

CodePudding user response：

A more generic approach :

jq '.Results |
    (map(keys)|add|unique|map({key:.,value:""})|from_entries) as $default |
    map($default   .)
   ' input.json