I have a pretty complicated JSON file that I encourage you to copy-paste here for better visualization.

{
  "data":[
    {
      "title":"Title1",
      "paragraphs":[
        {
          "context":"Context1",
          "qas":[
            {
             "answers":[
               {
                "answer_start":515,
                "text":"String text"
               }
              ],
             "question": "Why something something?",
             "id":"5733be284776f41900661182"
             },
             {
             "answers":[
               {
                "answer_start":505,
                "text":"String something text"
               }
              ],
             "question": "Why?",
             "id":"5733be284776f4190066345"
             }
            ]
           },
           {
          "context":"Context2",
          "qas":[
            {
             "answers":[
               {
                "answer_start":515,
                "text":"String text"
               }
              ],
             "question": "Why something something?",
             "id":"5733be284776f41900661182"
             },
             {
             "answers":[
               {
                "answer_start":505,
                "text":"String something text"
               }
              ],
             "question": "Why?",
             "id":"5733be284776f4190066345"
             }
            ]
           }
          ]
         },
         {
         "title":"Title2",
      "paragraphs":[
        {
          "context":"Context10",
          "qas":[
            {
             "answers":[
               {
                "answer_start":585,
                "text":"String text"
               }
              ],
             "question": "Why something something?",
             "id":"5733be284776f41900661682"
             },
             {
             "answers":[
               {
                "answer_start":545,
                "text":"String something text"
               }
              ],
             "question": "Why?",
             "id":"5733be284776f41900663"
             }
            ]
           },
           {
          "context":"Context7",
          "qas":[
            {
             "answers":[
               {
                "answer_start":525,
                "text":"String text"
               }
              ],
             "question": "Why something something?",
             "id":"5733be284776f41982"
             },
             {
             "answers":[
               {
                "answer_start":595,
                "text":"String something text"
               }
              ],
             "question": "Why?",
             "id":"5733be284776f419005"
             }
            ]
           }
          ]
         }
         ],
          "version":"1.1"
         }
          

I don't need all this information and I want my data format to look simpler, like this:

[
 {
   "id": "5733be284776f41900661182",
   "context": "Context1",
   "question": "Why something something?",
   "answers": {"text": ["String text"]}
 },
 {
    "id": "5733be284776f4190066345",
    "context": "Context1",
    "question": "Why?",
    "answers": {"text": ["String something text"]}
 },
 {
   "id": "5733be284776f41900661182",
   "context": "Context2",
   "question": "Why something something?",
   "answers": {"text": ["String text"]}
 },
 {
    "id": "5733be284776f4190066345",
    "context": "Context2",
    "question": "Why?",
    "answers": {"text": ["String something text"]}
 },
 {
   "id": "5733be284776f41900661682",
   "context": "Context10",
   "question": "Why something something?",
   "answers": {"text": ["String text"]}
 },
 {
    "id": "5733be284776f41900663",
    "context": "Context10",
    "question": "Why?",
    "answers": {"text": ["String something text"]}
 },
 {
   "id": "5733be284776f41982",
   "context": "Context7",
   "question": "Why something something?",
   "answers": {"text": ["String text"]}
 },
 {
    "id": "5733be284776f419005",
    "context": "Context7",
    "question": "Why?",
    "answers": {"text": ["String something text"]}
 }                                                                                                                                  
]

I don't care about the paragraphs, titles, answer start etc., and I would prefer it if the contexts were separated. Is there any way one can turn such a JSON into something much simpler? What I tried was to remove the unnecessary words in Python (replace them with " "), but it made a mess out of it. I also tried to open it as a dataframe and remove the columns, but there are many nested things that I want removed too.

CodePudding user response：

You can write a simple Python script to translate this format:

import json

with open('file.json', 'r') as fh:
    data = json.load(fh)

result = []

for article in data["data"]:
    for paragraph in article["paragraphs"]:
        for qa in paragraph["qas"]:
            answers = {"text": [answer["text"] for answer in qa["answers"]]}
            result.append({
                "id": qa["id"], 
                "context": paragraph["context"], 
                "question": qa["question"], 
                "answers": answers
            })

with open('output.json', 'w') as fh:
    json.dump(result, fh)