I'm deploying an application with a Dockerfile and docker-compose. It loads a model from an AWS bucket to run the application. When the containers get restarted (not intentionally but because of the cloud provider), it loads again the model from AWS. What I would like to achive is storing the model on a persistent volume. In case of a restart, I would like to check whether the volume exists and is not empty and if so run a different docker-compose file which has a different bash command, not loading the model from AWS again.

This is part of my docker-compose.yml:

  rasa-server:
    image: rasa-bot:latest
    working_dir: /app
    build:
      context: ./
      dockerfile: Dockerfile
    volumes:
     - ./models /app/models

    command: bash -c "rasa run --model model.tar.gz --remote-storage aws --endpoints endpoints.yml --credentials credentials.yml --enable-api --cors \"*\" --debug --port 5006" 

In case of a restart the command would look like this:

    command: bash -c "rasa run --model model.tar.gz --endpoints endpoints.yml --credentials credentials.yml --enable-api --cors \"*\" --debug --port 5006" 

Note that this

--remote-storage aws

was removed.

This is my Dockerfile:

FROM python:3.7.7-stretch AS BASE

RUN apt-get update \
    && apt-get --assume-yes --no-install-recommends install \
        build-essential \
        curl \
        git \
        jq \
        libgomp1 \
        vim

WORKDIR /app

RUN pip install --no-cache-dir --upgrade pip

RUN pip install rasa==3.1
RUN pip3 install boto3

ADD . .

I know that I can use this:

docker volume ls

to list volumes. But I do not know how to wrap this in a if condition to check whether

     - ./models /app/models

exists and is not empty and if it is not empty run a second docker-compose.yml which contains the second modified bash command.

CodePudding user response：

You could have an if statement in your bash command to use aws or not depending on the result you get from docker volume ls using -f name= you can filter based on the volume name and then you can check if its not null and run a different command.
Note that this command is just an example and I have no idea if it works or not as I don't use bash day -to-day

command: bash -c "
    VOLUME = docker volume ls -f name=FOO
    if [ -z "$VOLUME" ]; 
    then
        rasa run --model model.tar.gz --remote-storage aws --endpoints endpoints.yml --credentials credentials.yml --enable-api --cors \"*\" --debug --port 5006
    else 
        rasa run --model model.tar.gz --endpoints endpoints.yml --credentials credentials.yml --enable-api --cors \"*\" --debug --port 5006
    fi
    " 

CodePudding user response：

I would accomplish this by making the main container command actually be a script that looks to see if the file exists and optionally fills in the command line argument.

#!/bin/sh

MODEL_EXISTS=$(test -f /app/models/model.tar.gz && echo yes)

exec rasa run \
  --model model.tar.gz \
  ${MODEL_EXISTS:---remote-storage aws} \
  --endpoints endpoints.yml \
  ...

The first line uses the test(1) shell command to see if the file already exists, and sets the variable MODEL_EXISTS to yes if it exists and empty if it does not. Then in the command, there is a shell parameter expansion: if the variable MODEL_EXISTS is unset or empty :- then expand and split the text --remote-storage aws. (This approach inspired by BashFAQ/050.)

In your Dockerfile, COPY this script into your image and make it be the default CMD. It needs to be executable like any other script (run chmod x on your host and commit that change to source control); since it is executable and begins with a "shebang" line, you do not need to explicitly name the shell when you run it.

...
COPY rasa-run ./
CMD ["./rasa-run"]

In your Compose file, you do not need to override the command:, change the working_dir: from what the Dockerfile sets, or change from a couple of Compose-provided defaults. You should be able to reduce this to

version: '3.8'
services:
  rasa-server:
    build: .
    volumes:
      - ./models:/app/models

More generally for this class of question, I might suggest:

1. Prefer setting a Dockerfile default CMD to a Compose command: override; and
2. Write out non-trivial logic in a script and run that script as the main container command; don't write complicated conditionals in an inline CMD or command:.