Is there a way to find out what the driver IP is on a databricks cluster? The Ganglia UI shows all the nodes on the main page and there doesn't seem to be a way to filter only for the driver.

CodePudding user response：

You can go into the Spark cluster UI - Master tab within the cluster. The url listed contains IP for the driver and the workers' IPs are listed at the bottom.

Depending on your use case, it may be helpful to know that in an init script you can get the DB_DRIVER_IP from an environment variable. https://docs.databricks.com/clusters/init-scripts.html#environment-variables

There are other environment variables set at runtime that can be accessed in a scala notebook:

System.getenv.get("MASTER")         // spark://10.255.128.6:7077
System.getenv.get("SPARK_LOCAL_IP") // 10.255.128.6