I get null for the timestamp 27-04-2021 14:11 with this code. What mistake am I doing? Why is the timestamp format string DD-MM-yyyy HH:mm not correct here?

df = spark.createDataFrame([('27-04-2021 14:11',)], ['t'])
df = df.select(to_timestamp(df.t, 'DD-MM-yyyy HH:mm').alias('dt'))
display(df)

CodePudding user response：

D is for day of the year, and d is for day of the month.

Try this:

df = df.select(F.to_timestamp(df.t, "dd-MM-yyyy HH:mm").alias("dt"))