Home > Back-end >  apply a function to list in Scala
apply a function to list in Scala

Time:10-28

I'm trying to learn Scala. I ask for help to understand loop foreach I have a function, it reads only last csv from path. But it works when I point only one way:

val path = "src/main/resources/historical_novel/"
def getLastFile(path: String, spark: SparkSession): String = {

  val hdfs = ...}


but how can I apply this function to the list such as

val paths: List[String] = List(
  "src/main/resources/historical_novel/",
  "src/main/resources/detective/",
  "src/main/resources/adventure/",
  "src/main/resources/horror/")

I want to get such result:

src/main/resources/historical_novel/20221027.csv
src/main/resources/detective/20221026.csv
src/main/resources/adventure/20221026.csv
src/main/resources/horror/20221027.csv

I create df with column (path), then apply function through WithColumn and it is work, but I want to do it with foreach, understand it.

CodePudding user response:

let's say your function is like this

def f(s: String): Unit = {}

you can simply do this

paths.foreach(p => f(p))

CodePudding user response:

foreach applies a function you define or provide on each element in a Collection.

The simplest example is to print each path to the console

paths.foreach(path => println(path))

To apply a series of functions as you describe you can use {} in the foreach body and call multiple functions.

paths.foreach(path => {
    val file = loadFile(path)
    writeToDataBase(file)
  })
  • Related