Using Docker with Python Poetry?-CodePudding

I've been using Docker with pipenv for my data science deployment setup and now I want to change to Poetry. My Dockerfile is:

FROM python:3.8-alpine3.13

ENV POETRY_VIRTUALENVS_CREATE=false \
    POETRY_VERSION=1.1.11

RUN apk add --no-cache python3-dev gcc libc-dev musl-dev openblas gfortran build-base postgresql-libs postgresql-dev libffi-dev \
  && pip install poetry
COPY pyproject.toml poetry.lock ./
RUN poetry export -f requirements.txt --output requirements.txt && sed -i 's/^-e //' requirements.txt

USER root
RUN apt-get update && apt-get install -y --no-install-recommends python3-dev gcc libc-dev musl-dev openssh-client git libpq-dev \
  && apt-get clean -y


# install dependencies from requirements.txt
RUN pip install --no-cache-dir --user -r requirements.txt

COPY . .
CMD ["jupyter", "notebook", "--port=8888", "--no-browser", "--ip=0.0.0.0", "--allow-root"]

My pyproject.toml:

[tool.poetry]
name = ""
version = "1.0.0"
description = ""
authors = [""]

[tool.poetry.dependencies]
python = "^3.8"

... # lots of libraries, omitted here

[tool.poetry.dev-dependencies]
black = "*"
ipykernel = "6.*"
ipython = "7.*"
isort = "5.*"
jupyter = "*"
pytest = "6.*"
pre-commit = "2.*"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

This is based on other Dockerfiles that I found on StackOverflow. I run into the following problem:

Step 7/10 : RUN apt-get update && apt-get install -y --no-install-recommends python3-dev gcc libc-dev musl-dev openssh-client git libpq-dev   && apt-get clean -y
 ---> Running in 447ffb83d555
/bin/sh: apt-get: not found
The command '/bin/sh -c apt-get update && apt-get install -y --no-install-recommends python3-dev gcc libc-dev musl-dev openssh-client git libpq-dev   && apt-get clean -y' returned a non-zero code: 127
Running containerdocker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "jupyter": executable file not found in $PATH: unknown.
ERRO[0000] error waiting for container: context canceled 
make: *** [Makefile:6: jupyter_notebook] Error 127

So this looks like the Poetry is not used, where Jupyter is installed, and therefore it cannot be found. How can I fix this?

CodePudding user response：

It is not a problem with poetry, but because you're trying to install packages using apt-get in an alpine image (which doesn't include apt-get). You're also installing some of the packages multiple times, since you're using first using apk and then using apt-get. Also you don't need to install python, since you're using the python image as base.

I would also suggest just using poetry to install instead of dumping to requirements.txt and installing poetry using the official installer. You will also have to bump python since python 3.8 is not supported.

Take a look at this answer for combining docker and Poetry: Integrating Python Poetry with Docker

Your Dockerfile could look like this instead:

FROM python:3.9-alpine3.13

ENV POETRY_VIRTUALENVS_CREATE=false \
    POETRY_VERSION=1.1.11 \
    YOUR_ENV=development

RUN apk add --no-cache python3-dev gcc libc-dev musl-dev openblas gfortran build-base postgresql-libs postgresql-dev libffi-dev curl

RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python -
ENV PATH "/root/.local/bin:$PATH"


# install dependencies
COPY pyproject.toml poetry.lock ./
RUN poetry install $(test "$YOUR_ENV" == production && echo "--no-dev") --no-interaction --no-ansi

COPY . .
CMD ["jupyter", "notebook", "--port=8888", "--no-browser", "--ip=0.0.0.0", "--allow-root"]

CodePudding user response：

Based on Asgeer's answer I managed to fix the Dockerfile. However, using Alpine turned out to be pretty impossible and the final solution is:

FROM jupyter/scipy-notebook:latest

ENV POETRY_VIRTUALENVS_CREATE=false \
    POETRY_VERSION=1.1.11

USER root
RUN apt-get update \
    && apt-get --yes install apt-utils \
    && apt-get --yes install curl

RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python -
ENV PATH "/home/jovyan/.local/bin:$PATH"

# install dependencies
COPY pyproject.toml poetry.lock ./
RUN poetry install --no-interaction --no-ansi

COPY . .
CMD ["jupyter", "notebook", "--port=8888", "--no-browser", "--ip=0.0.0.0", "--allow-root"]