Unsupported literal type class scala.collection.immutable.Nil$ List()-CodePudding

My scala code goes like this.

def getFileName(fileName: String, postfixList: List[String]): String = {
  if (...) {
    return fileName   postfixList(0)
  }
    return fileName   postfixList(1)
}

val getFileNameUdf = udf(getFileName(_: String, _: List[String]): String)

def main(args: Array[String]): Unit = {
  val postfixList = List("a", "b")
  val rawFsRecordDF = sparkSession.read.option("delimiter", "\t").schema(fsImageSchema)
    .withColumn("fileName", getFileNameUdf(col("name"), lit(postfixList)))

}

For each of column values, I want to append a certain postfix.

Running this code, I get

diagnostics: User class threw exception: java.lang.RuntimeException: Unsupported literal type class scala.collection.immutable.Nil$ List()

How should I pass list to a UDF function?

Any comment or link would be appreciated!

CodePudding user response：

lit is good for basic types. For parameterized types, you should use typedLit.

val rawFsRecordDF = sparkSession.read.option("delimiter", "\t").schema(fsImageSchema)
    .withColumn("fileName", getFileNameUdf(col("name"), typedLit(postfixList)))

should work.