Mount ADLS Gen2 Storage - File must be dbfs or s3n: /-CodePudding

I want to mount ADLS Gen2 Storage in Databricks with this code:

%python

configs = {
  "fs.azure.account.key.<storage-account-name>.dfs.core.windows.net": <storage-account-key>,
  "spark.hadoop.fs.azure.account.key.<storage-account-name>.dfs.core.windows.net": <storage-account-key>
}

dbutils.fs.mount(
  source = "abfss://<container-name>@<storage-account-name>>.dfs.core.windows.net/",
  mount_point = "/mnt/aaa",
  extra_configs = configs)

But I get:

Any idea what can be root cause?

CodePudding user response：

Mount ADLS Gen2 Storage in Databrick

There are two scenarios you can Mount and implement ADLS Gen2 Storage in Databrick.

Scenario 1:

Directly take the Access key from ADLS Gen2 Blob Storage past in <storage-account-Access key> of extra_configs as shown in the create mount.

Syntax

spark.conf.set("fs.azure.account.key.<storage-account-name>.dfs.core.windows.net", dbutils.secrets.get(scope="<Scope-Name>",key="Key_Value"))

Create Mount

dbutils.fs.mount(
    source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/",
    mount_point = "/mnt/io243",
    extra_configs = {"fs.azure.account.key.<storage-account-name>.blob.core.windows.net":"<storage-account-Access key>"})

Scenario 2:

Go to the Access Keys and copy the key -> Create Secret for Access Key in Azure Key Vault .

Create an Azure Key Vault and

dbutils.fs.mount(
            source = "wasbs://<container-name>@<storage-account-name>.blob.core.windows.net/",
            mount_point = "/mnt/io243",
            extra_configs = {"fs.azure.account.key.<storage-account-name>.blob.core.windows.net":"dbutils.secrets.get(scope = "databricks-secret-scope", key = "blob-container-key")}

Reference:

https://bigdataprogrammers.com/create-mount-point-in-azure-databricks/

https://docs.microsoft.com/en-us/azure/databricks/data/databricks-file-system

https://www.youtube.com/watch?v=yeNgrBxHmCc