Apache Spark How to convert a datetime from Australia/Melbourne time to UTC?-CodePudding

How do I convert a datetime string like 21/10/2021 15:15:28 in Australia/Melbourne time to UTC in Apache Spark in Scala?

CodePudding user response：

Try this (of course you can ignore the creation of the data, I added it for you to understand the flow: columns' names and types):

Seq("21/10/2021 15:15:28").toDF("timeStr")
      .withColumn("australiaMelbourneTime", to_timestamp(col("timeStr"), "dd/MM/yyyy HH:mm:ss"))
      .withColumn("utcTime", to_utc_timestamp(col("australiaMelbourneTime"), "Australia/Melbourne"))

Output (tested):

 ------------------- ---------------------- ------------------- 
|            timeStr|australiaMelbourneTime|            utcTime|
 ------------------- ---------------------- ------------------- 
|21/10/2021 15:15:28|   2021-10-21 15:15:28|2021-10-21 04:15:28|
 ------------------- ---------------------- -------------------