I am using Azure Data Factory in which a data flow is used, I want to split my file in to two based on a condition. I am attaching an image with 2 lines, the first one is working but I want to use more programatic approach to achieve the same output:

I have a column named indicator inside my dataset, I want to use contains functionality to split the data, basically having 1 file where a string value inside indicator column has substring Weekly or does not.

Similar to what I would use in pandas:

df1 = df[df.indicator.str.contains('Weekly')]
df2 = df[~df.indicator.str.contains('Weekly')]

CodePudding user response：

You can try the below expression as well in the Conditional split.

contains() expects an array. So first split the column content to create the array and give this to contains function.

contains(split(indicator, ' '),#item=='weekly')

This is my sample data.

Conditional split:

Weekly data in the output:

Remaining data:

CodePudding user response：

If you are looking for the existing of a value inside of a string scalar column, use instr().

https://learn.microsoft.com/en-us/azure/data-factory/data-flow-expressions-usage#instr