How Can I pass secret manager secret using cloudbuild to app engine environment variable in app.yaml-CodePudding

Below is my app.yaml

runtime: python39
entrypoint: gunicorn -b :$PORT main:app
runtime_config:
  python_version: 3

env_variables:
     SEC: %sec%

manual_scaling:
  instances: 1
resources:
  cpu: 1
  memory_gb: 0.5
  disk_size_gb: 10

This is my cloudbuild.yaml for app engine trying to pass a secrete vale to app.yaml as env veriable

   steps:
    - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
      entrypoint: 'bash'
      args: ['-c', "export _VAL=$(echo $$SEC) 
              && echo $$SEC;echo $_VAL  && sed -i 's/%sec%/'$$SEC'/g' app.yaml
              && gcloud config set app/cloud_build_timeout 1600 && gcloud app deploy"
      ]
      secretEnv: ["SEC"]
    availableSecrets:
      secretManager:
      - versionName: projects/$PROJECT_ID/secrets/db_secret/versions/3
        env: 'SEC'
    
    timeout: '1600s'

CodePudding user response：

You are in the right way, you have made a quote mistake because you try to run a inline command.

Prefer the | structure, more readable and therefore easy to debug and to write. Each line is executed sequentially, similar to the && in your inline command

steps:
    - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
      entrypoint: 'bash'
      args: 
        - '-c'
        - |
           export _VAL=$(echo $$SEC)
           echo $$SEC
           echo $_VAL
           sed -i "s/%sec%/$$SEC/g" app.yaml
           gcloud config set app/cloud_build_timeout 1600 
           gcloud app deploy
      secretEnv: ["SEC"]
    availableSecrets:
      secretManager:
      - versionName: projects/$PROJECT_ID/secrets/db_secret/versions/3
        env: 'SEC'
    
    timeout: '1600s'

EDIT 1:

If your password contain special characters that break the SED expression, you will have an error. You can use an alternative solution to replace the whole line, for example

steps:
    - name: 'gcr.io/google.com/cloudsdktool/cloud-sdk'
      entrypoint: 'bash'
      args: 
        - '-c'
        - |
           export _VAL=$(echo $$SEC)
           echo $$SEC
           echo $_VAL
           cat app.yaml | awk "{ if (NR == $(grep -n '%sec%' app.yaml | cut -d : -f 1)) print \"    SEC: $$SEC\"; else print }" > app.yaml
           gcloud config set app/cloud_build_timeout 1600 
           gcloud app deploy
      secretEnv: ["SEC"]
    availableSecrets:
      secretManager:
      - versionName: projects/$PROJECT_ID/secrets/db_secret/versions/3
        env: 'SEC'
    
    timeout: '1600s'

Have a try on it

CodePudding user response：

I've done this in the past by just using ">>" to append the env vars to the bottom of my app.yaml. With this method, the env_variables section of your app.yaml needs to be last.

I don't use this method anymore though, the secrets show up in your cloudbuild log. I just import the secret manager package to grab my secrets inside the application these days.

cloudbuild.yaml

steps:
- name: "gcr.io/cloud-builders/gcloud"
  secretEnv: ['SECRET_ONE','SECRET_TWO']
  entrypoint: 'bash'
  args: 
  - -c
  - |
    echo $'\n  SECRET_ONE: '$$SECRET_ONE >> app.yaml
    echo $'\n  SECRET_TWO: '$$SECRET_TWO >> app.yaml
    gcloud -q app deploy
availableSecrets:
  secretManager:
  - versionName: projects/012345678901/secrets/SECRET_ONE
    env: 'SECRET_ONE'
  - versionName: projects/012345678901/secrets/SECRET_TWO
    env: 'SECRET_TWO'

app.yaml

runtime: go116
main: cmd
service: serviceone
env_variables:
  PROJECT_ID: project-a0a00
  PORT: 8080