Python: Writing list of integers to .csv file and reading back to same format-CodePudding

I'm having a typical copy and pasters syntax confusion, probably because I don't really understand what I'm doing!

I'm trying to save a series of facemark_landmark coordinates gleaned from the mediapipe library. I iterate through frames of a movie and each frame gives me 468 integer pairs, screen coordinates of each landmark.

Each frame I am adding the list of coordinates to a list called stored_data

stored_data.append(landmark_coords)

When the movie is finished I write the data to file:

with open("out.csv", "w") as f:
        wr = csv.writer(f)
        wr.writerows(stored_data)

This seems to work fine and my csv file looks like this: (181, 95) , (170, 87) , ..... 468 pairs on each line, one line for each frame of the movie.

But I am getting tangled up trying to convert the csv file back to the correct format.

My current effort is:

stored_data = []

with open("out.csv", "r") as f:
        rr = csv.reader(f)   
        for row in rr:
            frland=[]
            for l in row:
                coords = [map(float,l.split(',')) for i in l]
                frland.append(coords)
            stored_data.append(frland)

I am getting the error: ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (468,) inhomogeneous part.

Can anyone give me a hand?

CodePudding user response：

CSV, in my experience, is for sharing data between different people and processes. CSV doesn't have native types per se, everything gets serialized as a string, but yeah, now you're having trouble decoding some values when you try to read it back.

And because you're having to decode, that's a good indicator that something else might be better than CSV. If you just need to store the mediapipe results between processes, I echo what Ari said in the comments, use pickle: it's there to store complex types so you can avoid having to think about encoding and decoding and make it as simple as "save, then open":

import pickle

orig_stored_data = [[(181, 95), (170, 87)], [(20, 40), (30, 50)]]

f_pickle = open("stored_data.pkl", "wb")
pickle.dump(orig_stored_data, f_pickle)

f_pickle = open("stored_data.pkl", "rb")
new_stored_data = pickle.load(f_pickle)

assert new_stored_data == orig_stored_data

If you have to use CSV, you'll need to properly interpret the tuples in each column. I think this is incomplete:

coords = [map(float,l.split(',')) for i in l]

as you're not removing the parentheses before splitting on the comma.

A more straight-forward approach is to use the literal_eval() function in the ast module:

import ast, csv, io, pprint

f_in = io.StringIO('"(181, 95)","(170, 87)"\n"(  20,   40)","(30,50)"')

stored_data = []

reader = csv.reader(f_in)
for row in reader:
    frland = []
    for col in row:
        coords = ast.literal_eval(col)
        frland.append(coords)
    stored_data.append(frland)

and stored_data looks like:

[[(181, 95), (170, 87)],
 [(20, 40), (30, 50)]]