I am trying to code the following :

df.select(nvl(col("id"),0))

When I execute this, I get an error value nvl not found.

Please help me to solve this issue.

CodePudding user response：

In Spark its called coalesce, you can check this article for more details

# create new column with non Null values
tmp = testDF.withColumn('newColumn', coalesce(testDF['id'], testDF['number']))

# Check the content of new df
tmp.show()

 ---- ------ --------- 
|  id|number|newColumn|
 ---- ------ --------- 
|   1|     1|        1|
|   2|     2|        2|
|null|     3|        3|
|   4|  null|        4|
 ---- ------ --------- 

In your case it may look like this:

df.select(coalesce(col("id"),lit(0)))