I have a json file stored in Azure Blob Storage and I have loaded it into Azure SQL DB using Data Factory. Now I would like to find a way in order to load only new records from the file to my database (as the file is being updated every week or so). Is there a way to do it?

Thanks!

CodePudding user response：

You can use the upsert ( slowly changing dimension type 1 ) that is already implemented in Azure Data Factory. It will add new record and update old record that changed.

Here a quick tutorial :

In Alter row you can use Upsert if condition.

Here mention condition as 1 == 1