How to extract JSON from a nested JSON file?-CodePudding

I am calling an API and getting a response like the below.

{
    "status": 200,
    "errmsg": "OK",
    "data": {
        "total": 12,
        "items": [{
                "id": 11,
                "name": "BBC",
                "priority": 4,
                "levelStr": "All",
                "escalatingChainId": 3,
                "escalatingChain": {
                    "inAlerting": false,
                    "throttlingAlerts": 20,
                    "enableThrottling": true,
                    "name": "Example123",
                    "destination": [],
                    "description": "",
                    "ccdestination": [],
                    "id": 3,
                    "throttlingPeriod": 10
                }
            },
            {
                "id": 21,
                "name": "CNBC",
                "priority": 4,
                "levelStr": "All",
                "escalatingChainId": 3,
                "escalatingChain": {
                    "inAlerting": false,
                    "throttlingAlerts": 20,
                    "enableThrottling": true,
                    "name": "Example456",
                    "destination": [],
                    "description": "",
                    "ccdestination": [],
                    "id": 3,
                    "throttlingPeriod": 10
                }
            }
        ]
    }
}

I need to clean-up this JSON a bit and produce a simple JSON like below where escalatingChainName is the name in the escalatingChain list so that I can write this into a CSV file.

{
    "items": [{
            "id": 11,
            "name": "BBC",
            "priority": 4,
            "levelStr": "All",
            "escalatingChainId": 3,
            "escalatingChainName": "Example123"
        },
        {
            "id": 21,
            "name": "CNBC",
            "priority": 4,
            "levelStr": "All",
            "escalatingChainId": 3,
            "escalatingChainName": "Example456"
        }
    ]
}

Is there a JSON function that I can use to copy only the necessary key-value or nested key-values to a new JSON object?

With the below code, I am able to get the details list.

json_response = response.json()
items = json_response['data']
details = items['items']

I can print individual list items using

for x in details:
    print(x)

How do I take it from here to pull only the necessary fields like id, name, priority and the name from escalatingchain to create a new list or JSON?

CodePudding user response：

There is no existing function that will do what you want, so you'll need to write one. Fortunately that's not too hard in this case — basically you just create a list of new items by extracting the pieces of data you want from the existing ones.

import json

json_response = """\
    {
        "status": 200,
        "errmsg": "OK",
        "data": {
            "total": 12,
            "items": [{
                    "id": 11,
                    "name": "BBC",
                    "priority": 4,
                    "levelStr": "All",
                    "escalatingChainId": 3,
                    "escalatingChain": {
                        "inAlerting": false,
                        "throttlingAlerts": 20,
                        "enableThrottling": true,
                        "name": "Example123",
                        "destination": [],
                        "description": "",
                        "ccdestination": [],
                        "id": 3,
                        "throttlingPeriod": 10
                    }
                },
                {
                    "id": 21,
                    "name": "CNBC",
                    "priority": 4,
                    "levelStr": "All",
                    "escalatingChainId": 3,
                    "escalatingChain": {
                        "inAlerting": false,
                        "throttlingAlerts": 20,
                        "enableThrottling": true,
                        "name": "Example456",
                        "destination": [],
                        "description": "",
                        "ccdestination": [],
                        "id": 3,
                        "throttlingPeriod": 10
                    }
                }
            ]
        }
    }
"""


response = json.loads(json_response)

cleaned = []
for item in response['data']['items']:
    cleaned.append({'id': item['id'],
                    'name': item['name'],
                    'priority': item['priority'],
                    'levelStr': item['levelStr'],
                    'escalatingChainId': item['escalatingChainId'],
                    'escalatingChainName': item['escalatingChain']['name']})

print('cleaned:')
print(json.dumps(cleaned, indent=4))

CodePudding user response：

You can try:

data = {
    "status": 200,
    "errmsg": "OK",
    "data": {
        "total": 12,
        "items": [{
                "id": 11,
                "name": "BBC",
                "priority": 4,
                "levelStr": "All",
                "escalatingChainId": 3,
                "escalatingChain": {
                    "inAlerting": False,
                    "throttlingAlerts": 20,
                    "enableThrottling": True,
                    "name": "Example123",
                    "destination": [],
                    "description": "",
                    "ccdestination": [],
                    "id": 3,
                    "throttlingPeriod": 10
                }
            },
            {
                "id": 21,
                "name": "CNBC",
                "priority": 4,
                "levelStr": "All",
                "escalatingChainId": 3,
                "escalatingChain": {
                    "inAlerting": False,
                    "throttlingAlerts": 20,
                    "enableThrottling": True,
                    "name": "Example456",
                    "destination": [],
                    "description": "",
                    "ccdestination": [],
                    "id": 3,
                    "throttlingPeriod": 10
                }
            }
        ]
    }
}

for single_item in data["data"]["items"]:
    print(single_item["id"])
    print(single_item["name"])
    print(single_item["priority"])
    print(single_item["levelStr"])
    print(single_item["escalatingChain"]["inAlerting"])
    # and so on

CodePudding user response：

Two ways of approaching this depending on whether your dealing with a variable or .json file using python list and dictionary comprehension:

1. Where data variable of type dictionary (nested) already defined:

# keys you want
to_keep = ['id','name', 'priority', 'levelStr', 'escalatingChainId','escalatingChainName']
new_data = [{k:v for k,v in low_dict.items() if k in to_keep} 
            for low_dict in data['data']['items']]
# where item is dictionary at lowest level
escalations = [{v 'Name':k[v]['name']} for k in data['data']['items'] for v in k if type(k[v])==dict]
# merge both lists of python dictionaries to produce flattened list of dictionaries

new_data = [{**new,**escl} for new,escl in zip(new_data,escalations)]

2. or (and since your refer json package) if you have save the response to as a .json file:

import json 
with open('response.json', 'r') as handl:
    data = json.load(handl)
to_keep = ['id','name', 'priority', 'levelStr', 'escalatingChainId','escalatingChainName']
new_data = [{k:v for k,v in low_dict.items() if k in to_keep} 
            for low_dict in data['data']['items']]
escalations = [{v 'Name':k[v]['name']} for k in data['data']['items'] for v in k if type(k[v])==dict]
new_data = [{**new,**escl} for new,escl in zip(new_data,escalations)]

Both produce output:

[{'id': 11,
  'name': 'BBC',
  'priority': 4,
  'levelStr': 'All',
  'escalatingChainId': 3,
  'escalatingChainName': 'Example123'},
 {'id': 21,
  'name': 'CNBC',
  'priority': 4,
  'levelStr': 'All',
  'escalatingChainId': 3,
  'escalatingChainName': 'Example456'}]

hope this helps :-)