Save a list of spark data frames to multiple csv files in pyspark-CodePudding

I was trying to write a list of spark dfs to local csv by doing this: for i in view: i.repartition(1).write.csv('D:\cc_file.csv',header=True), and in the 2nd iteration because I've already have the file created in the first, it won't write a file with the same file name. I wonder if there is a way to create a new file name in each iteration.

Thanks!

CodePudding user response：

Try writing with f-string -

for i in view: i.repartition(1).write.csv(f'D:\cc_file_{i}.csv',header=True)

And the name of the file should be different.

Or -

    for i in view: i.repartition(1).write.csv(f"D:\cc_file_{i}.csv",header=True)