How wirte many rows of Dataset<string> in only single file txt with Spark-CodePudding

I have a Dataset<String> and I need to write it in a txt file.

In this Dataset<String> I have three rows.

When I try, Spark writes three different files to the directory, I need to merge these lines into just a single string and then write.

my code :enter image description here

input.write().mode(SaveMode.Overwrite).text("src/main/resources/avro/BVA_RET_20210618")

when use .show() enter image description here

on debug modenter image description here

spark create this way:enter image description here

I need the file with three rows together like this:enter image description here

I need to merge three rows in a single file.

CodePudding user response：

Just do a reparation before writing, obviously do this only when the data is very small.

df
   .coalesce(1)                              //or .repartition(1)
   .mode(SaveMode.Overwrite)
   .text("src/main/resources/avro/BVA_RET_20210618")

CodePudding user response：

Thank you @Ronak Jain this work to me.