Home > Net >  How to Flatten a semicolon Array properly in Azure Data Factory?
How to Flatten a semicolon Array properly in Azure Data Factory?

Time:10-14

Context: I've a data flow that extracts data from SQL DB, when data comes is just one column with a string separated by tab, in order to manipulate the data properly, I've tried to separate every single column with its corresponding data:

enter image description here

  • Firstly, to 'rebuild' the table properly I used a 'Derived Column' activity replacing tab with semicolons instead (1) dropLeft(regexReplace(regexReplace(regexReplace(descripcion,[\t],';'),[\n],';'),[\r],';'),1)
  • So, after that use 'split()' function to get an array and build the columns (2) split(descripcion, ';')

enter image description here

Problem: When I try to use 'Flatten' activity (as here enter image description here

Expected output:

column2 column1 column3
2000017 ENVASE CORONA CLARA 24/355 ML GRAB PC13
2004297 ENVASE V FAM GRAB 12/940 ML USADO PC15

Could you say me what i'm doing wrong, guys? thanks by the way.

CodePudding user response:

You can use the derived column activity itself, try as below.

After the first derived column, what you have is a string array which can just be split again using derived schema modifier.

Where firstc represent the source column equivalent to your column descripcion

Column1: split(firstc, ';')[1]

Column2: split(firstc, ';')[2]

Column3: split(firstc, ';')[3]

enter image description here

Optionally you can select the columns you need to write to SQL sink

enter image description here

enter image description here

  • Related