How to pass Github secret (json file) to Dockerfile-CodePudding

i want to deploy my DBT/Bigquery project in a Docker container using CICD through Github actions. I am struggling to get the GCP credentials into the container. I put the credentials in a Github secret, as I obviously cannot put the credential file on Github. How can I pass the Github secret as an argument to keyfile.json so that it is copied into the container?

My Dockerfile:

FROM fishtownanalytics/dbt:0.21.0

ARG RUN_TARGET=foo

RUN groupadd --gid 50000 docker && \
    useradd --home-dir /home/docker --create-home --uid 50000 --gid 50000 --skel /dev/null docker

USER docker

RUN mkdir /home/docker/.dbt
# Ordering is least to most frequently touched folder/file
COPY profiles.yml /home/docker/.dbt/profiles.yml
COPY keyfile.json /home/docker/keyfile.json
COPY macros /home/docker/macros
COPY dbt_project.yml /home/docker/dbt_project.yml
COPY models /home/docker/models

WORKDIR /home/docker/

# Run dbt on container startup.
CMD ["run"]

My github/workflows/main.yml file looks as follows:

name: Build and Deploy to dbt project

on: push


jobs:  
  build-and-push:

    runs-on: ubuntu-latest
    
    steps:

      - name: Checkout repository
        uses: actions/checkout@v2

      - name: dotenv-load
        id: dotenv
        uses: falti/[email protected]

      - name: Set up Python 3.9
        uses: actions/setup-python@v2
        with:
          python-version: 3.9

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

      - name: Configure Docker
        run: gcloud auth configure-docker -q  

      - name: Build and push Docker  
        uses: mr-smithers-excellent/docker-build-push@v5
        with:
          image: repo/image
          tags: v1, latest
          registry: eu.gcr.io
          username: _json_key
          password: ${{ secrets.GCP_SA_KEY }}

This gives the following error when building:

COPY failed: file not found in build context or excluded by .dockerignore: stat keyfile.json: file does not exist

I have tried passing the github secret as a build-args, but to no success.

Or is it really bad practice to put the credentials in the container and should I approach it in a different way? (edited)

CodePudding user response：

Subsequent gcloud commands work for me after the below step. Try adding it immediately after your checkout step.

- name: Set up gcloud
    uses: google-github-actions/setup-gcloud@master
    with:
      service_account_key: ${{ secrets.GCP_SERVICE_ACCOUNT_KEY }}
      project_id: ${{ secrets.GCP_PROJECT_ID }}

CodePudding user response：

I ended up using the oath method for authentication:

jaffle_shop:
  target: dev
  outputs:
    dev:
      type: bigquery
      method: oauth
      project: project_name
      dataset: dataset_name
      threads: 1
      timeout_seconds: 300
      location: europe-west4 # Optional, one of US or EU
      priority: interactive
      retries: 1

name: Build and Deploy to dbt project

on: push


jobs:  
  build-and-push:

    runs-on: ubuntu-latest
    
    steps:

      - name: Checkout repository
        uses: actions/checkout@v2

      - name: dotenv-load
        id: dotenv
        uses: falti/[email protected]
      
      - name: get sha
        id: vars
        run: |
          echo ::set-output name=sha_short::$(git rev-parse --short=8 ${{ github.sha }})
      - name: Set up Python 3.9
        uses: actions/setup-python@v2
        with:
          python-version: 3.9

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

      - name: Login
        uses: google-github-actions/setup-gcloud@master
        with:
          project_id: ${{ steps.dotenv.outputs.GCP_PROJECT }}
          service_account_key: ${{ secrets.GCP_SA_KEY }}
          export_default_credentials: true

      - name: Configure Docker
        run: gcloud auth configure-docker -q  

      - name: Build and push Docker  
        uses: mr-smithers-excellent/docker-build-push@v5
        with:
          image: repo/image
          tags: v1, latest
          registry: eu.gcr.io
          username: _json_key
          password: ${{ secrets.GCP_SA_KEY }}