Home > other >  Sort the Shuffle didn't generate data, the index file
Sort the Shuffle didn't generate data, the index file

Time:09-25

In Spark1.6, the default is to use the Sort Shuffle, such as using reduceByKey will generate Shuffle files,

Such as execute wordcount statement
Sc. TextFile (" README. Md ".) flatMap (l=& gt; L.s plit (" ")). The map (w=& gt; 1), (w) reduceByKey (+ _ _). Collect

He didn't generate the file, under the TMP only generates some folders, folders are empty, why?

I also refer to the web site, http://jingpin.jikexueyuan.com/article/13115.html, he has generated,
  • Related