I have a dataframe with double column ( 20180310), i want to convert it to Date format (2018-03-10). I used this function df.withColumn( "new_col", F.to_date(F.col("old_col").cast("string"), "yyyyMMdd") ) but it returning error message or Null. Do you have any idea please?

CodePudding user response：

Because when you cast from double to string, the column will have this form: 2.018031E7

cast to integer before casting to string:

df.withColumn( "new_col", F.to_date(F.col("old_col").cast("integer").cast("string"), "yyyyMMdd"))