Home > Back-end >  Azure Data Factory Counting number of Files in Folder
Azure Data Factory Counting number of Files in Folder

Time:11-11

I am attempting to determine if a folder is empty.

My current method involves using a GetMeta shape and running the following to set a Boolean.

@greater(length(activity('Is Staging Folder Empty').output.childItems), 0)

This works great when files are present.

When the folder is empty (a state I want to test for) I get

"The required Blob is missing".

Can I trap this condition?

What alternatives are there to determine if a folder is empty?

CodePudding user response:

I have reproduced the above and got same error.

enter image description here

This error occurs when the folder is empty, and the source is a Blob storage. You can see it is working fine for me when the Source is ADLS.

enter image description here

for sample I have used set variable.

enter image description here

inside false of if.

enter image description here

if folder is empty:

enter image description here

Can I trap this condition?

What alternatives are there to determine if a folder is empty?

One alternative can be to use ADLS instead of Blob storage as source.

(or)

You can do like below, if you want to avoid this error with Blob storage as source. Give an if activity for the failure of Get Meta and check the error in the expression.

@startswith(string(activity('Get Metadata1').error.message), 'The required Blob is missing')

enter image description here

In True activities (required error and folder is empty) I have used set variable for demo.

enter image description here

In False activities (If any other error apart from the above occurs) use Fail activity to fail the pipeline.

Fail Message: @string(activity('Get Metadata1').error.message)

enter image description here

For success of Get Meta activity, there is no need to check the count of child Items because Get Meta data fails if the folder is empty. So, on success Go with your activities flow.

CodePudding user response:

An alternative would be

Blob: enter image description here

Dataset : enter image description here

where test is the container and test is the folder inside the container which I am trying to scan (Which ideally doesnt exist as seen above)

Use get meta data activity to check if the folder exists : enter image description here

enter image description here

If false, exit else count for the files

  • Related