jq how to add an element to multiple arrays-CodePudding

I have an issue with such json file:

{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16132421/",
  "id": "16132421",
  "gender": "W"
}
{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16104273/",
  "id": "16104273",
  "gender": "W"
}
{}
{
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}

I has 3 arrays. I would like to replicate the last array to each of the prevoius ones, so it will look like this:

{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16104259/",
  "id": "16104259",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}
{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16132421/",
  "id": "16132421",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}

I tried to just add or multiply the arrays, but it doesn't work.

When I add specific keys ( {name, url, id, gender} {brand, cats} ) it displays the key's in place, but it sets values as null.

Can you help me find the solution?

CodePudding user response：

Convert the stream into an array either by using -n with [inputs], or by using -s, then add the last item to all but the last .[:-1][]. To filter out the empty object, select by comparing it to {} or by checking its length.

jq -n '[inputs] | (.[:-1][] | select(. != {}))   last'

Demo

jq -s '(.[:-1][] | select(length > 0))   last'

Demo

Output:

{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16132421/",
  "id": "16132421",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}
{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16104273/",
  "id": "16104273",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}

CodePudding user response：

If the stream does not fit into your memory, you have to use the --stream option which breaks down the input into manageable pieces. With streamed processing, however, you cannot easily access different parts at the same time. Therefore, you'd need one pass to retrieve the last item, and another one to add it to the other ones. As during processing there is no indicator for an item to being the last one, you could instead filter out those that entirely match the one being added (which conveniently happens to also filter out previously empty items).

jq --argfile last <(jq 'reduce inputs as $in (.; $in)' big.json) --stream \
  -n 'fromstream(inputs)   $last | select(. != $last)' big.json

{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16132421/",
  "id": "16132421",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}
{
  "name": "The 24H Automatic Eye Pencil",
  "url": "/3ina/the-24h-automatic-pencil-waterproof-brow-pencil/p-16104273/",
  "id": "16104273",
  "gender": "W",
  "brand": "3INA",
  "cats": [
    "Makeup",
    "Eyebrows",
    "Eyebrow Makeup"
  ]
}