I have build a webscraper that extracts data into lists. The lists are converted into pd.Series() and after that the Series are converted into a DataFrame. Currently the data is stored in the following way:

dataframe_for_excel_file_structure = {'id': pd.Series(ids) ,'date': date_for_each_sheet, 'type_of_property': pd.Series(type_of_property), 'area': pd.Series(sqm_area), 'location': pd.Series(
    locations), 'price_per_m2': pd.Series(price_per_m2), 'total_price': pd.Series(prices), 'published_by': pd.Series(publisher), 'link': pd.Series(link_for_offer)}
dataframe_for_excel = pd.DataFrame(dataframe_for_excel_file_structure)

filename_for_sqm = time.strftime("%Y%m%d")
dataframe_for_excel.to_excel(filename_for_sqm   '.xlsx')

This creates a file everyday. I want to be able to store the data in one either as currently Excel or a CSV file.

I have tried the following:

with open('document.csv','a') as fd:
fd.write(myCsvRow)

This was taken from another post is Stackoverflow.

This doesn't work for me as it accepts only strings (one), with the error:

TypeError: write() argument must be str, not list

Therefore I am looking for another solution to my problem.

The end goal is to have one file containing all for everyday and not one file for each day.

Any suggestions will be highly appreciated!

CodePudding user response：

If you want to store data to same file every day

old_df = pd.read_csv('daily.csv')
old_df = old_df.append(dataframe_for_excel, ignore_index=True)
old_df.to_csv('daily.csv')