I'm quite new to Python and I'm wondering what would be the best way to save a dataframe in a way so that it can be easily reimported. I'm running a code that checks for newly added files and imports them. In case it is stopped for any reason, I would like to reimport the last data that was saved before it stopped.

So, the data I'm interested in is in avg_data. To save it to txt, I use:

df_file_name = folder_path   '/'   file_name   '_lastData.txt'
        
        with open(df_file_name, "w") as new_file:
             avg_data.to_string(new_file, index=False)

But with this I struggle to properly reimport.. I use:

previous_data_str = folder_path   '/'   file_name   '_lastData.txt'
if os.path.isfile(previous_data_str) :
    old_data = pd.read_csv(previous_data_str)

to reimport, but this doesn't work... I don't get an error message but the data is now all in one column... using sep="\t" leads to the same, using sep=" " leads to too many empty columns... I uploaded a sample .txt file here

Not sure what I'm doing wrong here... and if it is even the best to save it in this way. I would greatly appreciate some tips on how to handle this best, thanks!

CodePudding user response：

Simply save it to csv with pandas.DataFrame.to_csv and then you can easily import it with pandas.read_csv.