Pass variables from .env file to dockerfile through docker-compose-CodePudding

project
└───app
│   │   ...
│   │   Dockerfile
│   │
└───prod.env
└───docker-compose.yml

My docker-compose looks like this:

services:
   app:
      build:
         context: .\app
         args:
            ARG1: val1
            ARG2: val2
      env_file:
         - prod.env

But I've tried this too:

services:
   app:
      build:
         context: .\app
         args:
            ARG1: ${ARG1}
            ARG2: ${ARG2}
      env_file:
         - prod.env

My prod.env file looks like this:

ARG1 = 'val1'
ARG2 = 'val2'

But I've tried this too:

ARG1=val1
ARG2=val2

I would like for either the values of args or the values from the prod.env file to be passed to the dockerfile.

This is what I've tried to get this:

ARG ARG1
ARG ARG2

RUN echo ${ARG1}
RUN echo ${ARG2}

ENV ARG1 ${ARG1}
ENV ARG2 ${ARG2}

RUN echo ${ARG1}
RUN echo ${ARG2}

ENV ARG1 "new val2"
ENV ARG2 "new val2"

RUN echo ${ARG1}
RUN echo ${ARG2}

It always end with blank values.

Any help would be greatly appreciated. I feel like no answers from other posts have worked when I tried them.

To build I use docker-compose --env-file prod.env build

Thanks

Update Sergio Santiago asked if I could run docker-compose config and show the results.

Here are the final files I used for this test.

docker-compose:

services:
   app:
      build:
         context: .\app
         args:
            ARG1: val1
            ARG2: val2
      env_file:
         - prod.env

prod.env:

ARG3 = 'val3'
ARG4 = 'val4'

And here is the output of docker-compose --env-file prod.env config

networks:
  demo-net: {}
services:
  app:
    build:
      args:
        ARG1: val1
        ARG2: val2
      context: C:\project\app
    environment:
      ENV: prod.env
      ARG3: val3
      ARG4: val4

I would like to add that clearly from here getting the variable from the .env file to the docker-compose file is not the issue. I also have a flask app running on the container and through os.environ it is able to use the variables in the .env file. I just can't figure out how to give the same access to the Dockerfile.

CodePudding user response：

I'm posting a new answer to highlight the various assumptions related to the OP's question, in particular, the fact that there's a subtle difference between the ".env" unique filename and *.env files (arguments for env_file:).

But apart from this subtlety, the process to pass arguments from docker-compose.yml to docker build -f Dockerfile . and/or docker run -e … is easy, as shown by the comprehensive example below.

Minimal working example

Let's consider the following files in a given directory, say ./docker.

File docker-compose.yml:

services:
  demo-1:
    image: demo-${ENV_NUM}
    build:
      context: .
      args:
        ARG1: "demo-1/${ARG1}"
        ARG3: "demo-1/${ARG3}"
  demo-2:
    image: demo-2${ENV_FILE_NUM}
    build:
      context: .
      args:
        ARG1: "demo-2/${ARG1}"
        ARG3: "demo-2/${ARG3}"
    env_file:
      - var.env

Remark: even if we use a build: field, it appears to be a good idea to also add an image: field to automatically tag the built image; but note that these image names must be pairwise different.

File .env:

KEY="some value"
ENV_NUM=1
ARG1=.env/ARG1
ARG2=.env/ARG2
ARG3=.env/ARG3

File var.env:

ENV_FILE_NUM="some number"
ARG1=var.env/ARG1
ARG2=var.env/ARG2
ARG3=var.env/ARG3
ARG4=var.env/ARG4

File Dockerfile:

FROM debian:10

# Read build arguments (default value if omitted at CLI)
ARG ARG1="default 1"
ARG ARG2="default 2"
ARG ARG3="default 3"

# the build args are exported at build time
RUN echo "ARG1=${ARG1}" | tee /root/arg1.txt
RUN echo "ARG2=${ARG2}" | tee /root/arg2.txt
RUN echo "ARG3=${ARG3}" | tee /root/arg3.txt

# Export part of these args at runtime also
ENV ARG1="${ARG1}"
ENV ARG2="${ARG2}"

# exec-form is mandatory for ENTRYPOINT/CMD
CMD ["/bin/bash", "-c", "echo ARG1=\"${ARG1}\" ARG2=\"${ARG2}\" ARG3=\"${ARG3}\"; echo while at build time:; cat /root/arg{1,2,3}.txt"]

Experiment session 1

First, as suggested by @SergioSantiago in the comments, a very handy command to preview the effective docker-compose.yml file after interpolation is docker-compose config:

$ docker-compose config

WARN[0000] The "ENV_FILE_NUM" variable is not set. Defaulting to a blank string. 
name: docker
services:
  demo-1:
    build:
      context: /home/debian/docker
      dockerfile: Dockerfile
      args:
        ARG1: demo-1/.env/ARG1
        ARG3: demo-1/.env/ARG3
    image: demo-1
    networks:
      default: null
  demo-2:
    build:
      context: /home/debian/docker
      dockerfile: Dockerfile
      args:
        ARG1: demo-2/.env/ARG1
        ARG3: demo-2/.env/ARG3
    environment:
      ARG1: var.env/ARG1
      ARG2: var.env/ARG2
      ARG3: var.env/ARG3
      ARG4: var.env/ARG4
      ENV_FILE_NUM: some number
    image: demo-2
    networks:
      default: null
networks:
  default:
    name: docker_default

Here, as indicated by the warning, we see there's an issue for interpolating ENV_FILE_NUM despite the fact this variable is mentioned by var.env. The reason is that env_files lines just add new environment variables for the underlying docker run -e … command, but don't interpolate anything in the docker-compose.yml.

Contrarily, one can notice that the value ARG1=.env/ARG1 taken from ".env" is interpolated within the args: field of docker-compose.yml, cf. the output line:

args:
  ARG1: demo-1/.env/ARG1
  …

This very distinct semantics of ".env" vs. env_files is described in this page of the official documentation.

Experiment session 2

Next, let us run:

$ docker-compose up --build
                                                                                         
WARN[0000] The "ENV_FILE_NUM" variable is not set. Defaulting to a blank string.                                                                     
[ ] Building 10.4s (13/13) FINISHED
 => [demo-1 internal] load build definition from Dockerfile
 => => transferring dockerfile: 609B
 => [demo-2 internal] load build definition from Dockerfile
 => => transferring dockerfile: 609B
 => [demo-1 internal] load .dockerignore
 => => transferring context: 2B
 => [demo-2 internal] load .dockerignore
 => => transferring context: 2B
 => [demo-2 internal] load metadata for docker.io/library/debian:10
 => [demo-2 1/4] FROM docker.io/library/debian:10@sha256:ebe4b9831fb22dfa778de4ffcb8ea0ad69b5d782d4e86cab14cc1fded5d8e761
 => => resolve docker.io/library/debian:10@sha256:ebe4b9831fb22dfa778de4ffcb8ea0ad69b5d782d4e86cab14cc1fded5d8e761
 => => sha256:85bed84afb9a834cf090b55d2e584abd55b4792d93b750db896f486680638344 50.44MB / 50.44MB
 => => sha256:ebe4b9831fb22dfa778de4ffcb8ea0ad69b5d782d4e86cab14cc1fded5d8e761 1.85kB / 1.85kB
 => => sha256:40dd1c1b1c36eac161ab63b6ce3a57d56ad79a667a37717a31721bac3f30aaf9 529B / 529B
 => => sha256:26a2b081e03207d26a105340161109ba0f00e857cbb0ff85aaeeeadd46b709c5 1.46kB / 1.46kB
 => => extracting sha256:85bed84afb9a834cf090b55d2e584abd55b4792d93b750db896f486680638344
 => [demo-2 2/4] RUN echo "ARG1=demo-2/.env/ARG1" | tee /root/arg1.txt
 => [demo-1 2/4] RUN echo "ARG1=demo-1/.env/ARG1" | tee /root/arg1.txt
 => [demo-1 3/4] RUN echo "ARG2=default 2" | tee /root/arg2.txt
 => [demo-2 3/4] RUN echo "ARG2=default 2" | tee /root/arg2.txt
 => [demo-2 4/4] RUN echo "ARG3=demo-2/.env/ARG3" | tee /root/arg3.txt
 => [demo-1 4/4] RUN echo "ARG3=demo-1/.env/ARG3" | tee /root/arg3.txt
 => [demo-2] exporting to image
 => => exporting layers
 => => writing image sha256:553f294a410ceeb3c0ac9d252d443710c804d3f7437ad7fffa586967517f5e7a
 => => naming to docker.io/library/demo-1
 => => writing image sha256:84bb2bd0ffae67ffed0e74efbf9253b6d634a6f37c6f99bc4eedea81846a9352
 => => naming to docker.io/library/demo-2
                                     
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
[ ] Running 3/3              
 ⠿ Network docker_default     Created
 ⠿ Container docker-demo-1-1  Created
 ⠿ Container docker-demo-2-1  Created

Attaching to docker-demo-1-1, docker-demo-2-1

docker-demo-1-1  | ARG1=demo-1/.env/ARG1 ARG2=default 2 ARG3=
docker-demo-1-1  | while at build time:
docker-demo-1-1  | ARG1=demo-1/.env/ARG1
docker-demo-1-1  | ARG2=default 2
docker-demo-1-1  | ARG3=demo-1/.env/ARG3

docker-demo-2-1  | ARG1=var.env/ARG1 ARG2=var.env/ARG2 ARG3=var.env/ARG3
docker-demo-2-1  | while at build time:
docker-demo-2-1  | ARG1=demo-2/.env/ARG1
docker-demo-2-1  | ARG2=default 2
docker-demo-2-1  | ARG3=demo-2/.env/ARG3

docker-demo-1-1 exited with code 0
docker-demo-2-1 exited with code 0

Here, we can see again that the ".env" values and those of file_env: [ *.env ] play different roles that don't overlap.

Furthermore:

Given the absence of a Dockerfile command line ENV ARG3="${ARG3}", the value of build-arg ARG3 is not propagated at runtime (see the ARG3= line in the output above).
But the value can be exported at runtime anyway if it is defined/overidden in the environment: or env_file: sections in the docker-compose.yml file (see the ARG3=var.env/ARG3 line in the output above).

For more details, see the documentation of the ARG directive.

CodePudding user response：

You can use the .env file from docker compose so you use the same time that you defined in your service definition:

services:
    app:
         build:
            context: .\app
            args:
               ARG1: ${ARG3}
               ARG2: ${ARG4}
         env_file:
            - .env

In this way both can share the same env file, but you still have the drawback of redefining the variables as a placeholder.

This is a suggestion, choose the one that fits better to you