Bash: Ignore key value pairs from a JSON that failed to parse using jq-CodePudding

I'm writing a bash script to read a JSON file and export the key-value pairs as environment variables. Though I could extract the key-value pairs, I'm struggling to skip those entries that failed to parse by jq.

JSON (key3 should fail to parse)

{
 "KEY1":"ABC",
 "KEY2":"XYZ",
 "KEY3":"---ABC---\n
dskfjlksfj"

}

Here is what I tried

for pair in $(cat test.json  | jq -r -R  '. as $line | try fromjson catch $line | to_entries | map("\(.key)=\(.value)") | .[]' ); do
    echo $pair
    export $pair
done

And this is the error

jq: error (at <stdin>:1): string ("{") has no keys
jq: error (at <stdin>:2): string ("  \"key1...) has no keys

My code is based on these posts:

CodePudding user response：

You can use a combination of jq and shell command to skip those entries that failed to parse by jq. You can use the "try" statement in jq to attempt to parse the JSON data, and if it fails, it will catch the error and return the original input string. Then, you can check if the result of jq is a valid JSON object using the "jq type" command. If it's a valid JSON object, you can extract the key-value pairs, and if not, skip to the next iteration.

Here's an example:

for pair in $(cat test.json | jq -r -R '. as $line | try fromjson catch $line'); do
if [[ $(echo $pair | jq type) == "object" ]]; then
for keyvalue in $(echo $pair | jq -r 'to_entries | map("(.key)=(.value)") | .[]'); do
echo $keyvalue
export $keyvalue
done
fi
done

CodePudding user response：

[Note: this response was made to the original question, which has since been changed. The response essentially assumes the input consists of JSONLines interspersed with other lines.)

Since the goal seems to be to ignore lines that don't have valid key-value pairs, you can simply use catch empty:

while read -r line ; do 
    echo export "$line"
done < <(test.json jq -r -R  '
  . as $line
  | try fromjson catch empty
  | to_entries[]
  | "\(.key)=\"\(.value|@sh)\"" 
')

Note also the use of @sh and of the shell's read, and the fact that .value (in jq) and $line (in the shell) are both quoted. These are all important for robustness.