I have a requirement to dynamically run a data factory pipeline based on the the master pipeline parameter value. The parameter value is in the pipeline name. For example, My main pipeline name is : MasterLoadData and my child pipelines are: LoadDataCAN, LoadDataEUR, LoadDataNYK etc The location names CAN, EUR, NYK etc are the parameters.

Each of these pipelines performs unique functionality. I have multiple pipelines (approx 30) in this way. Instead of hardcoding the pipelines in the switch activity, I would like to use the web activity and dynamically construct the pipeline name and then run it.

I want to be able to verify if the child pipeline exists before I trigger the web activity. Is there a way to do this?

Any help is appreciated.

CodePudding user response：

You get the list of pipeline names in your data factory by calling this HTTP request.

1. Get the list of pipelines using web activity.

URL: https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelines?api-version=2018-06-01

Method: GET

2. Pass the output of web activity to a ForEach activity to loop each pipeline.

3. Inside ForEach activity, you can validate the current item (pipeline name: item().name) and perform the required activities.

You can refer to this link for detailed explanation.