Why can't jq's slurp handle a combination of here-strings and other files?-CodePudding

I want to merge some json in a file with some json generated at runtime. jq seems to have no difficulty if all the files passed to it are here-strings, or files in the system. But if I try to mix the file types it seems the here-strings are ignored, see snippet below:

Two normal files:

bash-4.2# echo '{"key":0}' > zero
bash-4.2# echo '{"key":1}' > one
bash-4.2# jq --slurp add zero one
{
  "key": 1
}

Normal file and here-string (only normal file appears in result!):

bash-4.2# jq --slurp add zero <<< '{"key":1}'
{
  "key": 0
}

Here-string first, then normal file (only normal file appears in result!):

bash-4.2# jq --slurp add <<< '{"key":0,"anotherkey":2}' one
{
  "key": 1
}

Single here-string (works fine):

bash-4.2# jq --slurp add <<< '{"key":0}'
{
  "key": 0
}

~~Two here-strings (works fine):~~ EDIT: Output is misleading, something else is going on here.

bash-4.2# jq --slurp add <<< '{"key":0}' <<< '{"key":1}'
{
  "key": 1
}

My suspicion is that jq works just fine and I am ignorant of how bash resolves the here-strings. But, how would I debug this to improve my understanding?

Note: A very easy workaround would be to evaluate my runtime json and produce a file, then merge the two files as above. I really want to know why the bold examples above don't produce what I would expect.

CodePudding user response：

After reading through the comments this is my understanding:

<<< is evaluated by the shell first and redirects stdin. If jq receives no positional arguments after the filter, it reads from stdin. Therefore all these statements are equivalent:

echo "{}" | jq --slurp add
<<< {} jq --slurp add
jq <<< {} --slurp add
jq --slurp <<< {} add
jq --slurp add <<< {}

If jq does receive positional arguments after the filter, it interprets them as filenames. It adheres to the convention of treating - as stdin.

bash-4.2# echo '{"one":1,"two":1}' > first
bash-4.2# echo '{"three":3}' > third
bash-4.2# jq --slurp add first - third <<< '{"two":2}'
{
  "one": 1,
  "two": 2,
  "three": 3
}

CodePudding user response：

The here-string construct simply redirects standard input. You will separately need to tell jq to read standard input if you call it in a way where it receives file name arguments. The de facto standard way to do that is to specify - as the input (pseudo-) filename.

I believe one of your test cases didn't actually work, and just looked like it did because the input data was constructed so as to be a no-op.

CodePudding user response：

One idea would be to use process substitution which, in essence, provides jq with a (temp) file descriptor it can work with.

Using awk to demonstrate the file descriptor idea:

$ awk '{print FILENAME}' <(echo 'abc')
/dev/fd/63

Demonstrating with a few of your examples:

$ jq --slurp add zero <(echo '{"key":1}')
{
  "key": 1
}

$ jq --slurp add zero <(echo '{"keyx":1}')
{
  "key": 0,
  "keyx": 1
}

$ jq --slurp add <(echo '{"key":0,"anotherkey":2}') one
{
  "key": 1,
  "anotherkey": 2
}

$ jq --slurp add <(echo '{"key":0}') <(echo '{"key":1}')
{
  "key": 1
}

$ jq --slurp add <(echo '{"key":0}') <(echo '{"keyx":1}')
{
  "key": 0,
  "keyx": 1
}