I have an Azure Data Factory (V2) and a pipeline that connects to multiple external machines over 1 SFTP linked service.

I have fully parameterized the linked service. There are 2 parameters which relate to authentication in the API that are not currently supported for Azure Key Vault; SSH Host Fingerprint and Username. These are both required for the linked service to operate.

The arguments for these parameters will change depending on the connection that is required and therefore cannot be hardcoded.

One possibility is to capture these values in a database and read them in at runtime. A possible problem is that the responsible persons for security would not accept this. Which might lead to a conversation about encrypting the values, which would not work because the values would need to be decrypted prior to usage by Azure Data Factory.

I also thought about having a database owned by the security team with these values, and it not be accessible to engineering. We would of course enable secure input and secure output in the activity that reads this database. However this only prevents logging of the values, ultimately we can still access these values.

I need this to be possible to meet requirements, but the solution needs to be secure and not in a trivial way. I really don't think it is acceptable to have these values un-secured.

CodePudding user response：

You can store the sensitive data in a key vault and get the value from key vault via web activity. The below GIT location has the JSON code for the same: https://github.com/NandanHegde15/Azure-DataFactory-Generic-Pipelines/blob/main/Get Secret From KeyVault/Pipeline/GetSecretFromKeyVault.json