Apply python code to all csv files in a folder-CodePudding

I have 3000 csv files for machine learning and I need to treat each of these files separately, but the code I will apply is the same. File sizes and number of lines vary between 16 kb and 25 mb, and 60 lines and 330 thousand lines, respectively. Also, each csv file has 77 columns. I wrote only code inside the loop with the help of the previous post, but after applying the code, I cannot update within the same files. I just applied the code from previous post and get error "No such file or directory: '101510EF'" (101510EF is my first csv file on my folder)

Loooking forward to your help. Thank you!

CodePudding user response：

You don't need the line: file_name=os.path.splitext(...)

Just this:

path = "absolute/path/to/your/folder"
os.chdir(path)
all_files = glob.glob('*.csv')

for file in all_files:
   df = pd.read_csv(file)
   df["new_column"] = df["seq"]   df["log_id"]
   df.to_csv(file)

CodePudding user response：

You need to provide absolute path WITH extension to pd.read_csv and df.to_csv methods. e.g. c:/Users/kaanarik/Desktop/tez_deneme/ornel/101510EF.csv