I upload data to the Azure Databricks and I can see there are datasets in the DBFS list, but I cannot find any datasets in the resource group that Databricks belongs to.
Where are Azure Databricks DBFS datasets stored?
Also, if I create Delta Lake, how can I call the delta lake from Azure Data factory? I cannot see where the delta lake is stored.
CodePudding user response:
Where are Azure Databricks DBFS datasets stored?
The Default storage location in DBFS is known as the DBFS root.
You can find any datasets in /databricks-datasets
: See
Databricks File System (DBFS) is a distributed file system mounted into an Azure Databricks workspace and available on Azure Databricks clusters. DBFS is on top of scalable object storage ADLS gen2.
how can I call the delta lake from Azure Data factory?
You can refer to this video and also Follow below References :
https://www.youtube.com/watch?v=evqQRwsF_Ps
Reference:
https://docs.microsoft.com/en-us/azure/databricks/data/databricks-datasets