Home > Software engineering >  JQ - Converting json to csv - "Multi-levels"
JQ - Converting json to csv - "Multi-levels"

Time:11-23

I'm trying to convert a json object into csv via jq. This is the json structure:

{
  "totalCount": 4440,
  "data": [
    {
      "company": {
        "name": "My_company_name",
        "countryCode": "US",
        "portfolioName": "My_portfolio"
      },
      "eventDate": "2021-11-22T00:00:00",
      "newValue": null,
      "oldValue": null,
      "ruleCode": 704,
      "ruleName": "New Accounts",
      "summary": "Explanations..."
    },
    {
      "company": {
        "name": "My_company_name 2",
        "countryCode": "UK",
        "portfolioName": "My_portfolio"
      },
      "eventDate": "2021-10-22T00:00:00",
      "newValue": null,
      "oldValue": null,
      "ruleCode": 701,
      "ruleName": "Hello",
      "summary": "otherExplanations..."    
      }
      ...
    ]
}

For data in "first level", I've no problems:

jq -r '.data | map({eventDate, ruleCode, ruleName, summary, oldValue, newValue}) | (first | keys_unsorted) as $keys | map([to_entries[] | .value]) as $rows | $keys,$rows[] | @csv' input.json > output.csv

But I'ld like to add the company name and country code for example, and I don't kown to do this, with this king of data in second "level".

I'ld to obtain something like that:

"eventDate","ruleCode","ruleName","summary","oldValue","newValue","companyName", "companyCountryCode"
"2021-11-22T00:00:00",704,"New Accounts","Explanations...",,,"My_company_name", "US"
"2021-11-22T00:00:00",701,"Hello","otherExplanations...",,,"My_company_name 2", "UK"

Could you help me ?

Thanks

CodePudding user response:

If you don't mind, I added a . to divide top-level from sub-level headers to make things easier (namely company.name and company.countryCode):

jq --raw-output '[
  "eventDate",
  "ruleCode",
  "ruleName",
  "summary",
  "oldValue",
  "newValue",
  "company.name",
  "company.countryCode"
] as $h
| $h, (.data[] | [getpath($h[] / ".")])
| @csv' 
"eventDate","ruleCode","ruleName","summary","oldValue","newValue","company.name","company.countryCode"
"2021-11-22T00:00:00",704,"New Accounts","Explanations...",,,"My_company_name","US"
"2021-10-22T00:00:00",701,"Hello","otherExplanations...",,,"My_company_name 2","UK"

Demo

  • Related