Home > Software design >  Pass Docker argument environment to deploy in k8s using helm
Pass Docker argument environment to deploy in k8s using helm

Time:03-11

I need a information , i am trying all day with no help , any help is much appreciated .

this is my Dockerfile


FROM amazonlinux:2.0.20181114
RUN yum install -y java-1.8.0-openjdk-headless 

# Add jar file to container. JAR_FILE also provided as argument
ARG JAR_FILE='**/*.jar'
ADD ${JAR_FILE} document_service.jar
RUN echo -e ' \n export DATABASENAME=`aws secretsmanager get-secret-value --secret-id myathlon/$env_name/nldwh/databasename --query SecretString --output text` \n echo $DATABASENAME'  >> /opt/entrypoint.sh
RUN echo -e ' \n export DATABASEUSER=`aws secretsmanager get-secret-value --secret-id myathlon/$env_name/nldwh/username --query SecretString --output text` \n echo $DATABASEUSER'  >> /opt/entrypoint.sh
RUN echo -e ' \n export  AWSBUCKETNAME=`aws secretsmanager get-secret-value --secret-id myathlon/$env_name/awss3/bucketname --query SecretString --output text` \n echo $AWSBUCKETNAME'  >> /opt/entrypoint.sh
RUN echo -e ' \n export  AWSACCESSKEY=`aws secretsmanager get-secret-value --secret-id myathlon/$env_name/awss3/accesskey --query SecretString --output text` \n echo $AWSACCESSKEY'  >> /opt/entrypoint.sh
RUN echo -e ' \n export  AWSSECRETKEY=`aws secretsmanager get-secret-value --secret-id myathlon/$env_name/awss3/secretkey --query SecretString --output text` \n echo $AWSSECRETKEY'  >> /opt/entrypoint.sh
RUN echo -e ' \n export DATABASEPASS=`aws secretsmanager get-secret-value --secret-id myathlon/$env_name/nldwh/password --query SecretString --output text` \n echo $DATABASEPASS \n cd \n java -jar /document_service.jar' >> /opt/entrypoint.sh

ARG env_name

# Run the generated shell script.
ENTRYPOINT ["/opt/entrypoint.sh"]

This is my values.yml file

replicaCount: 1

#pass repository and targetPort values during runtime
image:
  repository: 
  tag: "latest"
  pullPolicy: Always
service:
  type: ClusterIP
  port: 80
  targetPort: 
  
profile: "aws" 
cmd:
   ArgA: dev

This is my deployment.yml file

spec:
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          #args: [
          #"--ArgA={{ .Values.cmd.ArgA }}" ]```

my helm install command is

helm upgrade --install $(servicename) -f values_dev.yaml

My docker build file

- download: 'current' 
    
- task: Bash@3
  inputs:
    targetType: 'inline'
    script: |
      cd $(Build.SourcesDirectory)/../drop
      ls -lrt 
      ls -lrt target
      pwd
      echo $MYATHLONTRUST
      docker build --no-cache --pull -t ${{ parameters.servicename }}:latest .
      docker image ls | head -2
  displayName: 'Building Docker Image'

error message and output:

/usr/bin/bash /var/build/Ubuntu0205/_work/_temp/ebf1fa80-e5c1-4b35-9368-75166c468b69.sh
total 48
drwxr-xr-x 3 srv-vstsagent srv-vstsagent 4096 Mar 10 11:19 $HOME
drwxr-xr-x 4 srv-vstsagent srv-vstsagent 4096 Mar 10 11:19 src
drwxr-xr-x 7 srv-vstsagent srv-vstsagent 4096 Mar 10 11:19 target
-rw-r--r-- 1 srv-vstsagent srv-vstsagent 5864 Mar 10 11:55 azure-pipelines.yml
-rw-r--r-- 1 srv-vstsagent srv-vstsagent  248 Mar 10 11:55 Dockerfile.old
-rw-r--r-- 1 srv-vstsagent srv-vstsagent  440 Mar 10 11:55 Dockerfile
-rw-r--r-- 1 srv-vstsagent srv-vstsagent 1072 Mar 10 11:55 entrypoint.sh
-rw-r--r-- 1 srv-vstsagent srv-vstsagent 9547 Mar 10 11:55 pom.xml
-rw-r--r-- 1 srv-vstsagent srv-vstsagent  915 Mar 10 11:55 README.md
total 149932
drwxr-xr-x 2 srv-vstsagent srv-vstsagent      4096 Mar 10 11:19 maven-archiver
drwxr-xr-x 3 srv-vstsagent srv-vstsagent      4096 Mar 10 11:19 sonar
drwxr-xr-x 3 srv-vstsagent srv-vstsagent      4096 Mar 10 11:19 maven-status
drwxr-xr-x 3 srv-vstsagent srv-vstsagent      4096 Mar 10 11:19 classes
drwxr-xr-x 3 srv-vstsagent srv-vstsagent      4096 Mar 10 11:19 generated-sources
-rw-r--r-- 1 srv-vstsagent srv-vstsagent     90036 Mar 10 11:55 myathlon-restapi-document-service-0.0.1-SNAPSHOT.jar.original
-rw-r--r-- 1 srv-vstsagent srv-vstsagent 153417052 Mar 10 11:56 myathlon-restapi-document-service-0.0.1-SNAPSHOT.jar
/var/build/Ubuntu0205/_work/34/drop

Dependency Updated:
  elfutils-libelf.x86_64 0:0.176-2.amzn2  libblkid.x86_64 0:2.30.2-2.amzn2.0.5 
  libmount.x86_64 0:2.30.2-2.amzn2.0.5    libuuid.x86_64 0:2.30.2-2.amzn2.0.5  

Complete!
Error removing intermediate container 46b80034bc4a: No such container: 46b80034bc4ae08bc76c6c75bc081f051c1a0a0494d63c4ec2e1c9cff6ba39cb
 ---> 68085751f7cd
Step 3/8 : WORKDIR /app                           # avoid / container root directory
 ---> Running in 8b75df62cfcf
Error removing intermediate container 46b80034bc4a: No such container: 46b80034bc4ae08bc76c6c75bc081f051c1a0a0494d63c4ec2e1c9cff6ba39cb
 ---> 536ee4f4ebf6
Step 4/8 : ARG JAR_FILE='**/*.jar'
 ---> Running in de5054ea9f5b
Error removing intermediate container 46b80034bc4a: No such container: 46b80034bc4ae08bc76c6c75bc081f051c1a0a0494d63c4ec2e1c9cff6ba39cb
 ---> 8e206092f7c9
Step 5/8 : COPY ${JAR_FILE} document_service.jar  # prefer COPY to ADD
COPY failed: file not found in build context or excluded by .dockerignore: stat document_service.jar: file does not exist
REPOSITORY                                                                        TAG                                                                       IMAGE ID       CREATED             SIZE
<none>                                                                            <none>                                                                    8e206092f7c9   2 minutes ago       660MB
Finishing: Building Docker Image

what i am trying to achieve is during the deployment , the docker file variable env_name should be replaced with the environment dev or test or acc .(this should happen during the docker deployment stage not in the build stage because i am planning to use the same image for dev and test).

so for test if i deploy the same image which i build for dev then when deploying , it should fetch the aws secrets for test and not dev

I tried all i can but cant figure out , the $1 is not getting replaced with dev value .

Any help is really really appreciated .

Thanks

CodePudding user response:

There are two ways to set environment variables in a Dockerfile. ARG values are only visible in RUN instructions, and can't be changed after the image has been built. ENV values are visible while the container is running to (in the ENTRYPOINT/CMD) and can be changed when the container is run, but can't be directly set at build time.

Since this is a value you're trying to set when the container is running, you want ENV and not ARG. In your Kubernetes manifest, you can change this using a Pod spec's env: setting. (Also Compose environment: or the docker run -e option.)

More specifically in a Helm context, I would recommend making "environment name" a configurable value, rather than pass in an undifferentiated list of command-line arguments or environment values.

# values.yaml

# environmentName specifies the environment name used to look up
# secrets in AWS Secrets Manager.
environmentName: dev

Then in the Pod spec embedded inside the Deployment spec, add this in an env: block.

# charts/myathlon/templates/deployment.yaml
spec:
  template:
    spec:
      containers:
        - name: {{ .Chart.Name }}
          env:
            - name: env_name
              value: {{ .Values.environmentName }}

As a cleanup I'd also recommend breaking out the entrypoint script into its own script file, rather than trying to build it a line at a time in the Dockerfile. This will be easier to read and maintain.

#!/bin/sh
# entrypoint.sh

# Check: $env_name must be set
if [ -z "$env_name" ]; then
  echo '$env_name is not set; stopping' >&2
  exit 1
fi

# Retrieve secrets from Secrets Manager
export DATABASENAME=`aws secretsmanager get-secret-value --secret-id myathlon/$env_name/nldwh/databasename --query SecretString --output text`
...
export DATABASEPASS=`aws secretsmanager get-secret-value --secret-id myathlon/$env_name/nldwh/password --query SecretString --output text`

# Run the main container CMD
exec "$@"

Since you don't have a default value for $env_name, you don't particularly need to declare anything in the Dockerfile. I included a check at the top of the extracted entrypoint wrapper script that will exit if its unset at container startup time. That reduces the Dockerfile to:

FROM amazonlinux:2.0.20181114
RUN yum install -y java-1.8.0-openjdk-headless 
WORKDIR /app                           # avoid / container root directory
ARG JAR_FILE='**/*.jar'
COPY ${JAR_FILE} document_service.jar  # prefer COPY to ADD
COPY entrypoint.sh .                   # also COPY in entrypoint script
ENTRYPOINT ["/app/entrypoint.sh"]      # split ENTRYPOINT wrapper from
CMD ["java", "-jar", "/app/document_service.jar"] # main CMD
  • Related