Is .nonEmpty
method an action or transformation in spark scala?
For ex:
val don = Seq("Hi", "Hello")
if (don.nonEmpty) {
println("Not empty")
}
CodePudding user response:
Here is an example where .nonEmpty
is used to filter a sequence, so it is a transformation, while the action is collect:
val rdd: RDD[Seq[String]] = sparkContext.parallelize(
Seq(Seq("Hi", "Hello"), Seq()))
val NotEmptyRDD = rdd.filter(x => x.nonEmpty)
val notEmptyArray = NotEmptyRDD.collect()
for(element <- notEmptyArray)
System.out.println(element)