Parse JSON response with nested values (sometimes nonevalues)-CodePudding

I'm trying to parse multiple JSON responses from an API request using Python (one example below, but I have a list containing hundreds of these JSON responses). I'm looking to create a dictonary with the values from the "dealID", "companyName", and "dealSize": "amount" keys.

I'm able to successfully create a dictionary containing values from the first 3 keys above, but not the nested "amount". The issue I'm running into is that "amount" sometimes has a none value. To get around this, I created a function that replaces the none value with a default value. But now when I try to create a dictionary, I'm getting the error message "TypeError: string indices must be integers". Grateful for any advice. I've seemingly tried everything.

[{
    "dealId": "330843-24T",
    "dealNumber": 1,
    "companyId": "62224",
    "companyName": "Company A",
    "dealDate": "2014-04-16",
    "dealSize": {
        "amount": 1500000.0,
        "currency": "USD",
        "nativeAmount": 1500000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "dealSizeStatus": "Actual",
    "dealStatus": {
        "code": "COMP",
        "description": "Completed"
    },
    "percentAcquired": "None",
    "raisedToDate": {
        "amount": 1500000.0,
        "currency": "USD",
        "nativeAmount": 1500000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "vcRound": "1st Round",
    "vcRoundUpDownFlat": "None",
    "totalInvestedCapital": {
        "amount": 1500000.0,
        "currency": "USD",
        "nativeAmount": 1500000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "investorOwnership": "None",
    "stockSplit": "None",
    "dealType1": {
        "code": "Seed",
        "description": "Seed Round"
    },
    "dealType2": "None",
    "dealType3": "None",
    "dealClass": {
        "code": "VC",
        "description": "Venture Capital"
    },
    "dealSynopsis": "The company raised $1.5 million of seed funding",
    "totalInvestedEquity": {
        "amount": 300000.0,
        "currency": "USD",
        "nativeAmount": 300000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "debtType1": "None",
    "debtType2": "None",
    "debtType3": "None",
    "debtAmount1": {
        "amount": 1200000.0,
        "currency": "USD",
        "nativeAmount": 1200000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "debtAmount2": "None",
    "debtAmount3": "None",
    "debtRaisedInRound": {
        "amount": 1200000.0,
        "currency": "USD",
        "nativeAmount": 1200000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "contingentPayout": "None",
    "valuationAvailable": true,
    "capTableAvailable": false,
    "trancheInfoAvailable": false,
    "debtLenderInfoAvailable": true
}, {
    "dealId": "4053346-57T",
    "dealNumber": 3,
    "companyId": "624245-02",
    "companyName": "Company A",
    "dealDate": "2017-02-27",
    "dealSize": {
        "amount": 2800000.0,
        "currency": "USD",
        "nativeAmount": 2800000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "dealSizeStatus": "Actual",
    "dealStatus": {
        "code": "COMP",
        "description": "Completed"
    },
    "percentAcquired": "None",
    "raisedToDate": {
        "amount": 14298988.0,
        "currency": "USD",
        "nativeAmount": 14298988.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "vcRound": "3rd Round",
    "vcRoundUpDownFlat": "None",
    "totalInvestedCapital": {
        "amount": 2800000.0,
        "currency": "USD",
        "nativeAmount": 2800000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "investorOwnership": "None",
    "stockSplit": "None",
    "dealType1": {
        "code": "EVC",
        "description": "Early Stage VC"
    },
    "dealType2": {
        "code": "A1",
        "description": "Series A1"
    },
    "dealType3": "None",
    "dealClass": {
        "code": "VC",
        "description": "Venture Capital"
    },
    "dealSynopsis": "The company raised $2.8 million",
    "totalInvestedEquity": {
        "amount": 2800000.0,
        "currency": "USD",
        "nativeAmount": 2800000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "debtType1": "None",
    "debtType2": "None",
    "debtType3": "None",
    "debtAmount1": "None",
    "debtAmount2": "None",
    "debtAmount3": "None",
    "debtRaisedInRound": "None",
    "contingentPayout": "None",
    "valuationAvailable": false,
    "capTableAvailable": false,
    "trancheInfoAvailable": false,
    "debtLenderInfoAvailable": false
},```

CodePudding user response：

There is "[", what means this json inside the list. You need modify every element:

for my_json in some_json:
    print(my_json['argument'])

example:

response = r.get(url)
response_text = json.loads(response.text)
for responce_json in responce_text:
    lat = responce_json['lat']
    lon = responce_json['lon']

CodePudding user response：

I am not sure exactly the output you want.

if the output like this:

{'dealID': ['330843-24T', '4053346-57T'], 'companyName': ['Company A', 'Company A'], 'dealSize': [1500000.0, 2800000.0]}

so the code like this:

keys_needed = ['dealID', 'companyName', 'dealSize']
new_dict = {'dealID': [],
            'companyName': [],
            'dealSize': []}
# data is your dictionary from your json format
for d in data:
    
    new_dict['dealID'].append(d.get('dealId'))
    new_dict['companyName'].append(d.get('companyName'))
    new_dict['dealSize'].append(d.get('dealSize') and d.get('dealSize').get('amount'))

print (new_dict)

CodePudding user response：

I have created a minimal example of your input-data like this:

input_data = [
    {
        "dealId": "330843-24T",
        "companyName": "Company A",
        "dealSize": {
            "amount": 1500000.0,
        }},
    {
        "dealId": "4053346-57T",
        "companyName": "Company A",
        "dealSize": {
            "amount": None,
        }
    }]

How do you want to replace your None values? In this solution dealSize is set to 0, if the amount is None. Feel free to ask any question or correct me if my assumptions were wrong :)

output = [{"dealId": deal.get("dealId"), 
           "companyName": deal.get("companyName"), 
           "dealSize": deal.get("dealSize", {}).get("amount") or 0} for deal in input_data]

output:

[{'dealId': '330843-24T', 'companyName': 'Company A', 'dealSize': 1500000.0}, 
 {'dealId': '4053346-57T', 'companyName': 'Company A', 'dealSize': 0}]

EDIT: copy/paste

input_data = [
    {
        "dealId": "330843-24T",
        "companyName": "Company A",
        "dealSize": {
            "amount": 1500000.0,
        }},
    {
        "dealId": "4053346-57T",
        "companyName": "Company A",
        "dealSize": {
            "amount": None,
        }
    }]

output = [{"dealId": deal.get("dealId"),
           "companyName": deal.get("companyName"),
           "dealSize": deal.get("dealSize", {}).get("amount") or 0} for deal in input_data]