I have to ingest data from ADLS gen2 to tables in ADX. I first tried to use ADF, but considering the run time and everything, this was an really inefficient way to do it. I think it would be so much easier if there is a way to ingest data into ADX via databricks. I only can access ADX through Service Pricipals so I can't directly connect to the storage account from ADX.

Any help will be much appreciated!

CodePudding user response：

If it is for continuous ingestion consider creating an Event Grid data connection

If it is for one-time ingestion consider using the "lightIngest" tool, and provide the applicable connection string for your use case.

CodePudding user response：

I would also recommend the Event Grid Data connection. Please note that you can set up this data connection using managed identities.