I have a dynamic frame created with the required data. I'm using pyspark in aws glue job. I'm creating a file as below

df_final_data.coalesce.write.format("text").option("mode", "append").save(var_bucket)

But this is creating a file with LF line endings.

How can I create a file with CRLF line endings or is there a way to replace these LF to CRLF after the file is created?

CodePudding user response：

Try using the lineSep parameter:

df_final_data.coalesce.write.format("text").option("lineSep", "\r\n").option("mode", "append").save(var_bucket)