Home > Enterprise >  Use a JSON body query on a REST Web service in Azure Data Factory v2
Use a JSON body query on a REST Web service in Azure Data Factory v2

Time:09-28

My goal is to authenticate on a JSON web service :

POST /api/authenticate/basic HTTP/1.1
Content-Type: application/json
Content-Length: 123
Host: test.biz

{
  "login" : "user",
  "password" : "pass",
  "site" : "foo-site"
}

I need to use a JSON array in the request body. Sadly on Microsoft documentation they say that they don't support JSON request.

working example on adf

{
    "name": "pipeline1",
    "properties": {
        "activities": [
            {
                "name": "Web1",
                "type": "WebActivity",
                "dependsOn": [],
                "policy": {
                    "timeout": "7.00:00:00",
                    "retry": 0,
                    "retryIntervalInSeconds": 30,
                    "secureOutput": false,
                    "secureInput": false
                },
                "userProperties": [],
                "typeProperties": {
                    "url": "https://d1.a.biz/api",
                    "method": "POST",
                    "body": {
                        "login": "",
                        "password": "",
                        "site": ""
                    }
                }
            },
            {
                "name": "Set variable1",
                "type": "SetVariable",
                "dependsOn": [
                    {
                        "activity": "Web1",
                        "dependencyConditions": [
                            "Succeeded"
                        ]
                    }
                ],
                "userProperties": [],
                "typeProperties": {
                    "variableName": "token",
                    "value": {
                        "value": "@activity('Web1').output.Response",
                        "type": "Expression"
                    }
                }
            }
        ],
        "variables": {
            "token": {
                "type": "String"
            }
        },
        "annotations": []
    } }

CodePudding user response:

Firstly, you are allowed to put a JSON body in a Rest API POST but you can't set this in the Linked service or dataset, you need to set it in the Copy activity itself See the below pic and ADF code. enter image description here

{
    "name": "Copy data1",
    "type": "Copy",
    "dependsOn": [],
    "policy": {
        "timeout": "7.00:00:00",
        "retry": 0,
        "retryIntervalInSeconds": 30,
        "secureOutput": false,
        "secureInput": false
    },
    "userProperties": [],
    "typeProperties": {
        "source": {
            "type": "RestSource",
            "httpRequestTimeout": "00:01:40",
            "requestInterval": "00.00:00:00.010",
            "requestMethod": "POST",
            "requestBody": "{\n    \"method\": \"GetFeed\",\n    \"params\": {\n        \"credentials\": {\n            \"database\": \"aci_logistics\",\n            \"sessionId\": \"xxxxxxxxxxxxxxxx\",\n            \"userName\": \"[email protected]\"\n        },\n        \"typeName\": \"LogRecord\",\n        \"search\": {\n            \"fromDate\": \"2021-09-28T00:00:00.000Z\",\n            \"toDate\": \"2021-08-01T01:00:00.000Z\"\n        }\n    }\n}"
        },
        "sink": {
            "type": "JsonSink",
            "storeSettings": {
                "type": "AzureDataLakeStoreWriteSettings"
            },
            "formatSettings": {
                "type": "JsonWriteSettings"
            }
        },
        "enableStaging": false
    },
    "inputs": [
        {
            "referenceName": "RestResource4",
            "type": "DatasetReference"
        }
    ],
    "outputs": [
        {
            "referenceName": "Json3",
            "type": "DatasetReference"
        }
    ]
}

Secondly, yes, you can extract a certain field from the result of your POST. Assuming you have already created a variable. You will need a set variable activity (like you already have). The value would be the output of your web activity but depends on which field you need and the structure/format of the result of your POST. In my example, I got a JSON result as below.

{
    "result": {
        "credentials": {
            "database": "xxxxx",
            "sessionId": "xxxxxxx",
            "userName": "xxxxxxxxxxxxxx"
        }
    }
}

Let's say I wanted to extract the sessionId, the value in my Set variable activity would look something like

@activity('Web1').output.result.credentials.sessionId
  • Related