How to convert nested list into a dataframe-CodePudding

I have a nested list looked liek this:

[[{'aaa': '42'}, {'bbb': '60'}, {'ccc': '25'}, {'ddd': '14'}, {'eee': '15'}, {'eee': '84'}],
 [{'aaa': '4'}, {'bbb': '0'}, {'ccc': '25'}, {'ddd': '1'}, {'eee': '1'}, {'eee': '8'}]]

And I want to convert it into a Dataframe. I looked at many solutions on our portal but nothing works for me :( Please help me as I'm a fresher in Python and Pandas module. Thank you!

Here is my desired Output:

I'm doing something like this but it's not working for me: DataFrame(data_dicts)

CodePudding user response：

Assuming L your input list, you can use a list/dictionary comprehension:

df = pd.DataFrame([{k:v for d in l for d in l for k,v in d.items()} for l in L])

NB. note that you have duplicated keys in your dictionaries, the last ones take precedence

output:

  aaa bbb ccc ddd eee
0  42  60  25  14  84
1   4   0  25   1   8

input:

L = [[{'aaa': '42'}, {'bbb': '60'}, {'ccc': '25'}, {'ddd': '14'}, {'eee': '15'}, {'eee': '84'}],
     [{'aaa': '4'}, {'bbb': '0'}, {'ccc': '25'}, {'ddd': '1'}, {'eee': '1'}, {'eee': '8'}]]

CodePudding user response：

You can use a collections.defaultdict to collect all values of a key together and then use pd.concat to construct a dataframe -

from collections import defaultdict

lld = [[{'aaa': '42'}, {'bbb': '60'}, {'ccc': '25'}, {'ddd': '14'}, {'eee': '15'}, {'eee': '84'}],
 [{'aaa': '4'}, {'bbb': '0'}, {'ccc': '25'}, {'ddd': '1'}, {'eee': '1'}, {'eee': '8'}]]

d = defaultdict(list)
for sublist in lld:
    for subdict in sublist:
        for k, v in subdict.items():
            d[k].append(v)
df = pd.concat([pd.DataFrame(value, columns=[key]) for key, value in d.items()], axis=1)

Output

   aaa  bbb  ccc  ddd eee
0   42   60   25   14  15
1    4    0   25    1  84
2  NaN  NaN  NaN  NaN   1
3  NaN  NaN  NaN  NaN   8