I would like to use Spark 3.3.0 version features like Trigger.availableNow in AWS Glue 3.0 with scala, but the AWS Glue 3.0 usage Apache spark version 3.1.1, Is there any way to use apache spark 3.3.0 in AWS Glue 3.0 with scala.

CodePudding user response：

No, that is unfortunately not possible. You can only use the versions that Glue provides. If you want to use a different Spark version you would have to use AWS EMR, they support Spark 3.2.0.