MAPING FUNCTIONS IN SPARK-SCALA-CodePudding

I'm starting with Spark with Scala and I'm wondering how can I do this:

I have a Dataframe with a column with those distinct values (R1,R2,M1,M2,I1,I2) and I want to map those values and create a new column which values depend on the values mapped in the other column. For example I want to map the first column and get something like the second column

   R1    it starts with R
   R1    it starts with R
   R2    it starts with R
   M1    it starts with M
   M2    it starts with M
   I1    it starts with I

Thanks

CodePudding user response：

import org.apache.spark.sql.functions._
import spark.implicits._
val substring = udf((str: String) => "Please, first use search ".concat(str.substring(0,1)))
val source = Seq("R1", "R2", "M1", "M2", "I1", "I2")
  .toDF("col1")
  .withColumn("col2",  substring(col("col1")))

source.show(false)
//     ---- -------------------------- 
//    |col1|col2                      |
//     ---- -------------------------- 
//    |R1  |Please, first use search R|
//    |R2  |Please, first use search R|
//    |M1  |Please, first use search M|
//    |M2  |Please, first use search M|
//    |I1  |Please, first use search I|
//    |I2  |Please, first use search I|
//     ---- --------------------------