How to trim zeros after bigDecimal value using Scala Spark-CodePudding

I'm beginner in spark scala. I have tried regex but it is used only for string Datatypes. I want Col_2 to be decimal without changing the precision and scale. Can someOne please guide me to solve this.

Col_1 : string

Col_2 : decimal(38,18) -->BigDecimal

    Col_1   Col_2
    ABC     2.000000000
    DEF     2.501000000
    XMN     0.00
    RST     1.28

wanted result

Col_1 : string Col_2 : string

    Col_1   Col_2
    ABC     2
    DEF     2.501
    XMN     0
    RST     1.28

CodePudding user response：

You can achieve this result using the code below, but it's better to keep numeric data using one of the numeric formats.

import spark.implicits._
val inputData = List(("ABC" -> 2.000000000),("DEF" -> 2.501000000),("XMN" -> 0.00),("RST" -> 1.28))
val resultDF = inputData.toDF
  .selectExpr("_1 as Col_1", "regexp_replace(cast(_2 as String), '\\.?0 $', '') as Col_2")
resultDF.show

Output

 ----- ----- 
|Col_1|Col_2|
 ----- ----- 
|  ABC|    2|
|  DEF|2.501|
|  XMN|    0|
|  RST| 1.28|
 ----- -----