I'm using Copy Data activity to send data to Azure Data Lake Storage Gen2. To do so, I must create a new folder every day that has the current timestamp at the end of it. So today my pipeline launches, in the proper path in the Storage Explorer I should see:
Folder_name_2021-11-02
I don't care about the format, I just need the date. In 'path' ('Sink' section of Copy data activity) I wrote:
@parent_folder/folder_name_utcNow()
But I get the following error: @parent_folder/folder_name_utcNow() is not valid: the string character '/' at position '13' is not expected.
CodePudding user response:
You need to create the string in the right format and put it in the right place. Here is an example where I am using concat
to join multiple strings with formatDateTime
to convert the output of utcNow
into a string format I can use in a filename. I am using this dynamic content in the File path property of the Sink output dataset.
NB I am using the forward slashes in my format string ensure I get that data lake hierarchy. If you just want a single file with the date in it, replace them with underscores or remove them altogether:
@concat('raw/ball/', formatDateTime(utcNow(), 'yyyy/MM/dd/HH/mm/'))
The output dataset:
You probably want something like:
@concat('raw/parent_folder/folder_name_/', formatDateTime(utcNow(), 'yyyyMMdd'))
Although it is more common to use the date folders as per my original example.