I am new to the Databricks Delta Live table. I have some small doubts and need your help to understand the concept behind it. I am unable to proceed without this.

I have a file in the Azure data lake container, and I know that I need to give the path under "cloud_files" so that delta live table can read files from this folder and show them. But my doubt is, if I give only the path, how do I mention the storage account name and container name? Also, do I need to provide an access key in order to read the data securely ?

I think I am missing something, I have gone through various articles and Youtube demo videos, and everywhere they just mention the path but do not tell me how to configure the path.

Please help me to understand this concept.

Thank You.

This is my code for the Delta Live table:

CREATE LIVE TABLE customers_raw
COMMENTS "This is raw table"
AS
SELECT *
FROM cloud_files("/raw_data/customers.csv", "csv")

CodePudding user response：

You need to specify full URL for this folder, like, abfss://<container>@<storage>.dfs.core.windows.net/raw_data/customers.csv. Otherwise if you specify it /raw_data/customers.csv it will consider it as a folder on DBFS, and will fail. Please note that in this case you will need to setup corresponding Spark properties so DLT can access data - you can find it in the following answer.