df =pd.DataFrame(eval(res))
print(df.head())
df.to_csv('data.csv',index=False)

This is my code to convert the data i need to a csv file but because the data is a dictionary, of a list of rougly 70 other dictionaries i can't get the format the way i want. here i want timestamp to be my first column and then each key after that to be another column with the prices/volume at each unix timestep. How can i modify my code to achieve this? I dont need the data column.

CodePudding user response：

I have assumed that all 70 rows in the image are JSON.

This code does the job

df = pd.read_csv(<csv file path>)
new_data = []

for index in df.index:
  row = eval(df.iloc[index, :].values[0])
  new_data.append(row.values())

new_df = pd.DataFrame(new_data, columns = row.keys())
new_df = pd.concat([new_df, df["itemId"]], axis = 1)

Here I convert convert the JSON object in each row to a Python dictionary and then append it into a list. Once I have the whole list, I convert it into a pandas dataframe.

A shorter way to do this,

rows = list(map(eval, df["data"].values.ravel()))
vals = map(dict.values, rows)
new_df = pd.DataFrame(vals, columns = list(rows)[0].keys())
new_df = pd.concat([new_df, df["itemId"]], axis = 1)

The table I get -