Home > Back-end >  ADF request from REST API within a time frame
ADF request from REST API within a time frame

Time:10-29

Assume that data needs to fetched by ADF using REST API and data is in JSON format.

Now for my call to REST API I have a time limit of say 45 seconds.

If source JSON file has say 1 million records then using parameters I can fetch only subset of this data.

But if my query via ADF exceeds 45 seconds then I should be able to receive a pre-defined message with an error code that I am requesting too much data and should refine my query criteria. ADF should be able to receive this message with the code while processing the request.

How can this be achieved in ADF?

Thx

CodePudding user response:

So the objective is to always return a message, either the succesful response by the REST API or a default message in case of an API error.

You can create dependencies between activities, check Activity dependencies, source: https://datasavvy.me/2021/02/18/azure-data-factory-activity-failures-and-pipeline-outcomes/

The orange dependency will only be executed when the Web Activity returns an error. You can set here the default message.

CodePudding user response:

What I understand from your question and comment is, You want to send error message as email notification.

There is no Send Email Activity in Azure Data Factory. To get error message as email notification, You have to Create Logic app and then use it in ADF.

Follow below 2 articles by Koen Verbeeck

First link gives step by step procedure to create Logic app and Second link gives steps to send error message as email notification.

Azure Data Factory Pipeline Email Notification

Send Notifications from an Azure Data Factory Pipeline

  • Related