How do I reference a different Variable group per stage - stage per environment in environments

When I run the pipeline below I get an error within jobs.yml - it commplains about an "unexpected value" for the environment parameter... and within stages.yml "unexpected value: parameters" - what am I doing wrong here? The idea is to get the environments and the associated variableGroup - loop thru the environments array in Stages.yml and create a stage per environment... insert correct variableGroup per stage... use variablesGroup values to perform jobs in jobs.yml - each variable group contains the same vars.. with different values.

This is main.yml

#main.yml
trigger: none

extends:
  template: /Build/Stages.yml
  parameters:
    environments:
      - environment: Dev
        variableGroup: Project-Dev-VarGrp
      - environment: QA
        variableGroup: Project-QA-VarGrp
        dependsOn: Dev
      - environment: UAT
        variableGroup: Project-UAT-VarGrp
        dependsOn: QA
      - environment: UAT2
        variableGroup: Project-UAT2-VarGrp
        dependsOn: UAT

Then here is the next bit... Stages.yml

parameters:
  - name: dataFolder
    displayName: 'Data folder to process'
    type: string
    default: '/DataMigrations/Master_Data/'
  - name: dataFiles
    displayName: List of Data Files or Folder names
    type: string
    default: 'Dev_Data.zip'
  - name: environments
    type: object
    default: []
    
stages:
  - ${{ each environment in parameters.environments }}: 
    - stage: '${{ parameters.environment }}'
      jobs:
        - template: ./jobs.yml
      variables:
      - group: ${{ parameters.variableGroup }}
        parameters:
           environment: '${{ parameters.environment }}'
           crmURL: $(crmURL)
           oauthAppId: $(ClientID)
           ClientPass: $(ClientPass)
           dataFolder: '${{ parameters.dataFolder }}'
           env: '${{ parameters.environment }}'

and here is jobs.yml

    jobs:
  - deployment: DeployData
    displayName: 'Deploying Data to ${{ parameters.environment }}'
    environment: ${{ parameters.environment }}
    pool:
      vmImage: 'windows-latest'
    strategy:
      runOnce:
        deploy:
          steps:
          - checkout: self
            clean: false
          - powershell: | 
              Write-Host "##vso[task.setvariable variable=crmConnectionString]'AuthType=ClientSecret;Url=$(crmURL);ClientId=$(ClientID);ClientSecret=$(ClientPass)'"
            displayName: 'PreDeploy configuration'
          - task: PowerShell@2
            displayName: 'Powershell: Run update-data.ps1'
            inputs:
              targetType: 'filePath'
              filePath: $(System.DefaultWorkingDirectory)/DataMigrations/Scripts/update-data.ps1
              arguments: -folderName '${{ parameters.dataFolder }}' -environment '${{ parameters.env }}'
              workingDirectory: $(System.DefaultWorkingDirectory)/DataMigrations
          - task: PowerShell@2
            displayName: 'Powershell: Run zip-import-data.ps1'
            inputs:
              targetType: 'filePath'
              filePath: $(System.DefaultWorkingDirectory)/DataMigrations/Scripts/zip-import-data.ps1
              arguments: -folderName '${{ parameters.dataFolder }}' -dataMigrationFilenames '${{ parameters.dataFiles }}' -connectionString $(crmConnectionString)
              workingDirectory: $(System.DefaultWorkingDirectory)/DataMigrations

CodePudding user response：

Few things here:

When you're using ${{ each environment in parameters.environments }} then the nested environment and variableGroup are available using this syntax ${{ environment.environment }} and ${{ environment.variableGroup}}

In your Stages.yml file, you're trying to invoke the ./jobs.yml template but the associated parameters are defined after the - group: ${{ parameters.variableGroup }}. A valid syntax should looks like this:

stages:
- ${{ each environment in parameters.environments }}: 
  - stage: '${{ environment.environment }}'
    variables:
    - group: ${{ environment.variableGroup }}
    jobs:
    - template: ./jobs.yml    
      parameters:
        environment: '${{ environment.environment }}'
        crmURL: $(crmURL)
        oauthAppId: $(ClientID)
        ClientPass: $(ClientPass)
        dataFolder: '${{ parameters.dataFolder }}'
        env: '${{ environment.environment }}'

You also have few space typos. I know it's annoying but you need to have the exact yaml syntax otherwise the files can't be parsed.