JQ: Removing null values from a JSON array in jq-CodePudding

Still I can't figure out how to remove null values from a key of JSON array type. I need an empty key({}) if all the values are null or remove nulls from the JSON array. In the below example, I want to filter the nulls from Workloads JSON array. Please let me know any workaround . Thanks a lot. [ I am using jq version 1.5 ] For example, with this input:

{
  "EC2 Instance Metadata": [
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-1",
      "EC2 Private IPv4 address": "1.2.3.4",
      "Workloads": [
        null
      ],
      "Application Pods": [
        "pod-1"
      ],
      "Application Containers": [
        "container"
      ],
      "Container Images": [
        "image/1.0"
      ]
    },
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-2",
      "EC2 Private IPv4 address": "1.2.3.44",
      "Workloads": [
        null
      ],
      "Application Pods": [
        "pod-2"
      ],
      "Application Containers": [
        "container-2"
      ],
      "Container Images": [
        "image/2.0"
      ]
    },
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-3",
      "EC2 Private IPv4 address": "1.23.3.44",
      "Workloads": [
        "workload-1",
        "workload-2",
        "workload-3",
        null,
        null,
        null,
        "workload-4"
      ],
      "Application Pods": [
        "pod-1",
        "pod-2"
      ],
      "Application Containers": [
        "cnt-1",
        "cnt-2"
      ],
      "Container Images": [
        "image-1",
        "image-2"
      ]
    }
  ]
}

I want this output :-

{
  "EC2 Instance Metadata": [
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-1",
      "EC2 Private IPv4 address": "1.2.3.4",
      "Workloads": [
        {}
      ],
      "Application Pods": [
        "pod-1"
      ],
      "Application Containers": [
        "container"
      ],
      "Container Images": [
        "image/1.0"
      ]
    },
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-2",
      "EC2 Private IPv4 address": "1.2.3.44",
      "Workloads": [
        {}
      ],
      "Application Pods": [
        "pod-2"
      ],
      "Application Containers": [
        "container-2"
      ],
      "Container Images": [
        "image/2.0"
      ]
    },
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-3",
      "EC2 Private IPv4 address": "1.23.3.44",
      "Workloads": [
        "workload-1",
        "workload-2",
        "workload-3",
        "workload-4"
      ],
      "Application Pods": [
        "pod-1",
        "pod-2"
      ],
      "Application Containers": [
        "cnt-1",
        "cnt-2"
      ],
      "Container Images": [
        "image-1",
        "image-2"
      ]
    }
  ]
}

Tried almost all the hints mentioned Here, but no luck.

CodePudding user response：

This recursively (..) reduces arrays containing (also) values (anything but null) to only contain those values, otherwise (containing nothing but null) it replaces them with [{}]:

jq '(.. | arrays) |= if any(values) then map(values) else [{}] end'

{
  "EC2 Instance Metadata": [
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-1",
      "EC2 Private IPv4 address": "1.2.3.4",
      "Workloads": [
        {}
      ],
      "Application Pods": [
        "pod-1"
      ],
      "Application Containers": [
        "container"
      ],
      "Container Images": [
        "image/1.0"
      ]
    },
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-2",
      "EC2 Private IPv4 address": "1.2.3.44",
      "Workloads": [
        {}
      ],
      "Application Pods": [
        "pod-2"
      ],
      "Application Containers": [
        "container-2"
      ],
      "Container Images": [
        "image/2.0"
      ]
    },
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-3",
      "EC2 Private IPv4 address": "1.23.3.44",
      "Workloads": [
        "workload-1",
        "workload-2",
        "workload-3",
        "workload-4"
      ],
      "Application Pods": [
        "pod-1",
        "pod-2"
      ],
      "Application Containers": [
        "cnt-1",
        "cnt-2"
      ],
      "Container Images": [
        "image-1",
        "image-2"
      ]
    }
  ]
}

Demo

CodePudding user response：

If we loop over the Workload we can use the update assignment (|=) and an if to:

Use all(. == null) to check if all values are null, if so, replace with [ {} ]
Otherwise, use map(values) to only keep values that are not null

."EC2 Instance Metadata"[].Workloads |= 
    if all(. == null) 
        then [ {} ] 
        else map(values)
    end

Will give the following output, which you can try in this online demo

{
  "EC2 Instance Metadata": [
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-1",
      "EC2 Private IPv4 address": "1.2.3.4",
      "Workloads": [
        {}
      ],
      "Application Pods": [
        "pod-1"
      ],
      "Application Containers": [
        "container"
      ],
      "Container Images": [
        "image/1.0"
      ]
    },
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-2",
      "EC2 Private IPv4 address": "1.2.3.44",
      "Workloads": [
        {}
      ],
      "Application Pods": [
        "pod-2"
      ],
      "Application Containers": [
        "container-2"
      ],
      "Container Images": [
        "image/2.0"
      ]
    },
    {
      "EC2 Private IPv4 DNS Name": "compute.internal-3",
      "EC2 Private IPv4 address": "1.23.3.44",
      "Workloads": [
        "workload-1",
        "workload-2",
        "workload-3",
        "workload-4"
      ],
      "Application Pods": [
        "pod-1",
        "pod-2"
      ],
      "Application Containers": [
        "cnt-1",
        "cnt-2"
      ],
      "Container Images": [
        "image-1",
        "image-2"
      ]
    }
  ]
}