I am trying to change date format from 20211018 to 202110180000. of a column of a spark data frame.

I have the following code to create new dt column out of existing data_dt

func =  udf (lambda x: datetime.datetime.strptime(x, '%Y%m%d'), DateType())
    
result_df = result_df.withColumn('dt', func(col('data_dt')))
result_df = result_df.select('data_dt', 
                   from_unixtime(unix_timestamp('data_dt', '%Y%m