Azure Databricks EventHub connection error-CodePudding

I'm trying to get data from Azure EventHub into Databricks. In order to establish a connection, I'm executing following code:

conf = {}
connectionString = "my connection string key"
conf["eventhubs.connectionString"] = sc._jvm.org.apache.spark.eventhubs.EventHubsUtils.en

The error that I'm getting is Py4JError: org.apache.spark.eventhubs.EventHubsUtils.en does not exist in the JVM

I read that this error has to do with the Maven Library, I am using 9.1 LTS (includes Apache Spark 3.1.2, Scala 2.12) and have installed following libraries:

com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.17
com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.18
com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.12
com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.21

Any ideas?

CodePudding user response：

It's a typo - instead of sc._jvm.org.apache.spark.eventhubs.EventHubsUtils.en you need to use sc._jvm.org.apache.spark.eventhubs.EventHubsUtils.encrypt.

Also, you need to leave only one library - you can't mix libraries for 2.12 & 2.11. It's better to use com.microsoft.azure:azure-eventhubs-spark_2.12:2.3.21