Home > Net >  Where are the Azure Databricks DBFS datasets stored?
Where are the Azure Databricks DBFS datasets stored?

Time:04-16

I upload data to the Azure Databricks and I can see there are datasets in the DBFS list, but I cannot find any datasets in the resource group that Databricks belongs to.

Where are Azure Databricks DBFS datasets stored?

Also, if I create Delta Lake, how can I call the delta lake from Azure Data factory? I cannot see where the delta lake is stored.

CodePudding user response:

Where are Azure Databricks DBFS datasets stored?

The Default storage location in DBFS is known as the DBFS root. You can find any datasets in /databricks-datasets: See Ref1

Databricks File System (DBFS) is a distributed file system mounted into an Azure Databricks workspace and available on Azure Databricks clusters. DBFS is on top of scalable object storage ADLS gen2.

how can I call the delta lake from Azure Data factory?

You can refer to this video and also Follow below References :

https://www.youtube.com/watch?v=evqQRwsF_Ps

Reference:

https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-databricks-delta-lake?tabs=data-factory

https://docs.microsoft.com/en-us/azure/databricks/data/databricks-file-system#--important-information-about-dbfs-permissions

https://docs.microsoft.com/en-us/azure/databricks/data/databricks-datasets

  • Related