Running Flask celery and gunicorn from a single docker container-CodePudding

I am trying to run both flask app with celery and gunicorn in same docker container. I am usinf supervisord for this. Gunicorn is executed from docker-compose.yml as

services:
  web:
    container_name: "flask"
    build: ./
    volumes:
      - ./app:/app
    ports:
      - "8000:8000"
    environment:
      - DEPLOYMENT_TYPE=production
      - FLASK_APP=app/main.py
      - FLASK_DEBUG=1
      - MONGODB_DATABASE=testdb
      - MONGODB_USERNAME=testuser
      - MONGODB_PASSWORD=testuser
      - MONGODB_HOSTNAME=mongo
    command: gunicorn app.main:app --workers 1 --name main --reload -b 0.0.0.0:8000 --preload
    depends_on:
      - redis
      - mongo
    links:
      - mongo

In the supervisord.conf, celery is setup

[supervisord]
nodaemon=true

[program:celeryworker]
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
command=/opt/venv/bin/celery -A app.routes.celery_tasks.celery worker --loglevel=info -B -s app/celerybeat-schedule

The supervisor is executed from dockerfile

FROM ubuntu:20.04
LABEL maintainer="nebu"
ENV GROUP_ID=1000 \
    USER_ID=1000
RUN apt-get update && apt-get -y upgrade
RUN apt-get install -y apt-transport-https ca-certificates supervisor procps cron vim python3.8-venv python3-gdbm 
WORKDIR /app
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1
ENV VIRTUAL_ENV=/opt/venv
RUN python3 -m venv $VIRTUAL_ENV
ENV PATH="$VIRTUAL_ENV/bin:$PATH"
RUN ["python", "-m", "pip", "install", "--upgrade", "pip", "wheel"]
RUN apt-get install -y  python3-wheel
COPY ./requirements.txt /app/requirements.txt
RUN ["python", "-m", "pip", "install", "--no-cache-dir", "--upgrade", "-r", "/app/requirements.txt"]
COPY ./app /app
RUN ["mkdir", "-p","/var/log/supervisor"]
COPY ./app/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
CMD ["/usr/bin/supervisord"]

But when i bring all the containers up, celery and beat are not coming up. So I entered the shell of flask app container and executed /usr/bin/supervisord, then the celery worker is brought up without any error. How is it possible to bring both gunicorn and celery worker-beat together from the same docker container.

CodePudding user response：

CMD ["/usr/bin/supervisord"] in dockerfile and command: gunicorn app.main:app --workers 1 --name main --reload -b 0.0.0.0:8000 --preload in docker-compose.yml have a conflict.

In fact, command in docker-compose.yml will override CMD in dockerfile. That is why only gunicorn is up.

My suggestion: add gunicorn related stuff to supervisord.conf and remove command in docker-compose.yml.

In this way, supervisor will bring both gunicorn and celery up.