how to get seperate multiple dict pairs from a list to new line in pandas-CodePudding

I have a dictionary output in below format in JSON File:

[{"Status": "NEW", "ID": "1", "type": "type1"},{"Status": "NEW", "ID": "2", "type": "type1"}]

I would like to have this in seperate line in JSON file:

{"Status": "NEW", "ID": "1", "type": "type1"}
{"Status": "NEW", "ID": "2", "type": "type1"}

Code I have is:

# I have to convert dataframe to dictionary first and then remove all null objects
json_dict=df.to_dict('records')
def remove_empty_elements(d):
    """recursively remove empty lists, empty dicts, or None elements from a dictionary"""

    def empty(x):
        return x is None or x == {} or x == []

    if not isinstance(d, (dict, list)):
        return d
    elif isinstance(d, list):
        return [v for v in (remove_empty_elements(v) for v in d) if not empty(v)]
    else:
        return {k: v for k, v in ((k, remove_empty_elements(v)) for k, v in d.items()) if not empty(v)}

data=remove_empty_elements(json_dict)
with open("nonnullfile.json", 'w') as fout:
    json_dumps_str = json.dumps(data)
    print(json_dumps_str, file=fout)

This code gives me output with a list of dictionaries but I would like to see in separate lines. I can do this using iterrows() and write in new line but that is very very slow.

Can anyone help me if there is any other solution?

CodePudding user response：

Try using the indent argument in json.dumps.

# ... your preceding code ... #

with open("nonnullfile.json", 'w') as fout:
    json_dumps_str = json.dumps(data, indent=4)

# ... everything else ... #

CodePudding user response：

When you call remove_empty_elements and the return value data is a list, you can do:

with open("nonnullfile.json", 'w') as fout:
    for element in data:
        element_json_dumps_str = json.dumps(element)
        print(element_json_dumps_str, file=fout)

This way you'll be able to write each dictionary of your list in a new line of your json file.