Parse complex JSON in Python-CodePudding

EDITED WITH LARGER JSON:

I have the following JSON and I need to get id element: 624ff9f71d847202039ec220

results": [
    {
      "id": "62503d2800c0d0004ee4636e",
      "name": "2214524",
      "settings": {
        "dataFetch": "static",
        "dataEntities": {
          "variables": [
            {
              "id": "624ffa191d84720202e2ed4a",
              "name": "temp1",
              "device": {
                "id": "624ff9f71d847202039ec220",
                "name": "282c0240ea4c",
                "label": "282c0240ea4c",
                "createdAt": "2022-04-08T09:01:43.547702Z"
              },
              "chartType": "line",
              "aggregationMethod": "last_value"
            },
            {
              "id": "62540816330443111016e38b",
              "device": {
                "id": "624ff9f71d847202039ec220",
                "name": "282c0240ea4c",
              },
              "chartType": "line",
            }
          ]
        }
    ...

Here is my code (EDITED)

    url = "API_URL"
    response = urllib.urlopen(url)
    data = json.loads(response.read().decode("utf-8"))
    print url
    all_ids = []
    for i in data['results']:  # i is a dictionary
       for variable in i['settings']['dataEntities']['variables']:
           print(variable['id'])
           all_ids.append(variable['id'])

But I have the following error:

        for variable in i['settings']['dataEntities']['variables']:
KeyError: 'dataEntities'

Could you please help? Thanks!!

CodePudding user response：

What is it printing when you print(fetc)? If you format the json, it will be easier to read, the current nesting is very hard to comprehend.

fetc is a string, not a dict. If you want the dict, you have to use the key.

Try:

url = "API_URL"
response = urllib.urlopen(url)
data = json.loads(response.read().decode("utf-8"))
print url
for i in data['results']:
    print(json.dumps(i['settings']))
    print(i['settings']['dataEntities']

EDIT: To get to the id field, you'll need to dive further.

i['settings']['dataEntities']['variables'][0]['id']

So if you want all the ids you'll have to loop over the variables (assuming the list is more than one)`, and if you want them for all the settings, you'll need to loop over that too.

Full solution for you to try (EDITED after you uploaded the full JSON):

url = "API_URL"
response = urllib.urlopen(url)
data = json.loads(response.read().decode("utf-8"))
print url

all_ids = []
for i in data['results']:  # i is a dictionary
    for variable in i['settings']['dataEntities']['variables']:
        print(variable['id'])
        all_ids.append(variable['id'])
        all_ids.append(variable['device']['id']

Let me know if that works.

CodePudding user response：

The shared JSON is not valid. A valid JSON similar to yours is:

{
  "results": [
    {
      "settings": {
        "dataFetch": "static",
        "dataEntities": {
          "variables": [
            {
              "id": "624ffa191d84720202e2ed4a",
              "name": "temp1",
              "span": "inherit",
              "color": "#2ccce4",
              "device": {
                "id": "624ff9f71d847202039ec220"
              }
            }
          ]
        }
      }
    }
  ]
}

In order to get a list of ids from your JSON you need a double for cycle. A Pythonic code to do that is:

all_ids = [y["device"]["id"] for x in my_json["results"] for y in x["settings"]["dataEntities"]["variables"]]

Where my_json is your initial JSON.