How would you collect the first few entries of a list from a large json file using jq?-CodePudding

I am trying to process a large json file for testing purposes that has a few thousand entries. The json contains a long list of data to is too large for me to process in one go. Using a jq, is there an easy way to get a valid snippet of the json that only contains the first few entries from the data list? For example is there a query that would look at the whole json file and return to me a valid json that only contains the first 4 entries from data? Thank you!

{
"info":{
"name":"some-name"
},
"data":[
{...},
{...},
{...},
{...}
}

CodePudding user response：

Based on your snippet, the relevant jq would be:

.data |= .[:4]

CodePudding user response：

Here's an example using the --stream option:

$ cat input.json
{
  "info": {"name": "some-name"},
  "data": [
    {"a":1},
    {"b":2},
    {"c":3},
    {"d":4},
    {"e":5},
    {"f":6},
    {"g":7}
  ]
}

jq --stream -n '
  reduce (
    inputs | select(has(1) and (.[0] | .[0] == "data" and .[1] < 4))
  ) as $in (
    {}; .[$in[0][-1]] = $in[1]
  )
' input.json

{
  "a": 1,
  "b": 2,
  "c": 3,
  "d": 4
}

Note: Using limit would have been more efficient in this case, but I tried to be more generic for the purpose of scalability.