Export data in CSV but with different columns set. Can it be done via a library?-CodePudding

Data input:

[
    {'a': 1, 'b': 2, 'c': 3},
    {'b': 2, 'd': 4, 'e': 5, 'a': 1},
    {'b': 2, 'd': 4, 'a': 1}
]

CVS output (columns order does not matter):

a, b, c, d, e
1, 2, 3
1, 2, ,  4, 5
1, 2, ,  4

Standard library csv module cannot cover such kind of input.

Is there some package or library for a single-method export? Or a good solution to deal with column discrepancies?

CodePudding user response：

Straightforward with pandas:

import pandas as pd

lst = [
{'a': 1, 'b': 2, 'c': 3},
{'b': 2, 'd': 4, 'e': 5, 'a': 1},
{'b': 2, 'd': 4, 'a': 1}
]

df = pd.DataFrame(lst)
print(df.to_csv(index=None))

Output:

a,b,c,d,e
1,2,3.0,,
1,2,,4.0,5.0
1,2,,4.0,

CodePudding user response：

It can be done fairly easily using the included csv module with a little preliminary processing.

import csv

data = [
    {'a': 1, 'b': 2, 'c': 3},
    {'b': 2, 'd': 4, 'e': 5, 'a': 1},
    {'b': 2, 'd': 4, 'a': 1}
]

fields = sorted(set.union(*(set(tuple(d.keys())) for d in data)))  # Determine columns.

with open('output.csv', 'w', newline='') as file:
    writer = csv.DictWriter(file, fieldnames=fields)
    writer.writeheader()
    writer.writerows(data)

print('-fini-')

Contents of file produced:

a,b,c,d,e
1,2,3,,
1,2,,4,5
1,2,,4,

CodePudding user response：

you have to pass a restval argument to Dictwriter which is the default argument for missing keys in dictionaries

writer = Dictwriter(file, list('abcde'), restval='')