I have multiple services that needs to use the same locally built image.
So I created a helper service which would build the image and all other services would depend on the builder service to ensure that image is built first. Here's docker-compose.yml with only 2 services but in actual, I have around 20 services.
version: "3.5"
services:
api:
image: test-image # using the image built below
networks:
- mylab
depends_on:
- tests-container-builder
volumes:
- ./tests:/tests
command: pytest -v -p no:cacheprovider /tests/api.py
license-chrome:
image: test-image
networks:
- mylab
depends_on:
- tests-container-builder
volumes:
- ./tests:/tests
command: sh -c "/tests/health-check-script.sh && pytest -v -p no:cacheprovider -n=1 /tests/license.py"
tests-container-builder:
build:
context: .
dockerfile: ./tests/Dockerfile
target: test-env
image: test-image # this service builds the image
But when I build and run the container using
docker-compose up --build api; \
docker-compose up --scale license-chrome
I'm getting error like
ERROR: for mylab_visual_ner-chrome_1 no such image: sha256:ceff9945d8722c89331696aa33d88f84703e322fbb64c1ebaa8f83c6e19a4cfa: No such image: sha256:ceff9945d8722c89331696aa33d88f84703e322fbb64c1ebaa8f83c6e19a4cfa
ERROR: for visual_ner-chrome no such image: sha256:ceff9945d8722c89331696aa33d88f84703e322fbb64c1ebaa8f83c6e19a4cfa: No such image: sha256:ceff9945d8722c89331696aa33d88f84703e322fbb64c1ebaa8f83c6e19a4cfa
ERROR: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.
Continue with the new image? [yN]
It works fine if I remove the test-container-builder service and use build command inside all services.
Is it considered an anti-pattern to use a different service to build image? What is the proper way to do this ?
CodePudding user response：
I'd consider an absolutely-best-practice Compose setup to only contain long-running containers, and not containers you don't expect to actually run. In your case the "builder" container doesn't actually do anything and I wouldn't include it in the Compose setup. Conversely, build:ing a literally identical image from the same source code is all but free; Docker needs to scan the build context to determine that the two images are identical, but in the end you will just get two different names for the same physical image.
Taking @TheFool's suggestion to use YAML anchors to avoid repeating the build setup, I might write:
version: '3.8'
services:
api:
build: &build-definition
context: .
dockerfile: ./tests/Dockerfile
target: test-env # (can you avoid this?)
command: pytest -v -p no:cacheprovider /tests/api.py
license-chrome:
build: *build-definition
# (does the health check belong in an ENTRYPOINT wrapper in the image?)
command: sh -c "/tests/health-check-script.sh && pytest -v -p no:cacheprovider -n=1 /tests/license.py"
I've omitted unnecessary networks:, since Compose creates a perfectly usable network named default for you, and volumes:, since your code should generally be built into the image.
If you want to explicitly build the image and include it in the Compose setup (maybe you want to push it to a registry) then you need to manually docker-compose build the builder image, not any of the containers you're going to run. The Compose setup can look like
version: '3.8'
services:
api:
image: registry.example.com/my-name/test-image:${TAG:-latest}
command: pytest -v -p no:cacheprovider /tests/api.py
license-chrome:
image: registry.example.com/my-name/test-image:${TAG:-latest}
command: sh -c "/tests/health-check-script.sh && pytest -v -p no:cacheprovider -n=1 /tests/license.py"
tests-container-builder:
image: registry.example.com/my-name/test-image:${TAG:-latest}
build:
context: .
dockerfile: ./tests/Dockerfile
target: test-env
restart: "no"
command: exit 0
Compared to my previous Compose setup and your question, I make it explicit that the container should just exit immediately and not restart, and I use a registry-qualified name. depends_on: only affects the order in which containers start starting and doesn't include any dependencies on image builds.
If you're using this setup, you need to docker-compose build the builder "container" before you run the rest:
export TAG=20211124 # often useful to avoid ...:latest
docker-compose build test-container-builder # not "api"
# docker-compose push test-container-builder # if required
docker-compose up -d # could just launch specific containers
CodePudding user response：
This is maybe not a real answer, but I have recreated your scenario, and it surprisingly works. I did expect it to fail if you only want to build the container that hasn't a build key in the config.
docker system prune -af # ensure we dont get confused with pre build images or chaching
docker-compose up --build two # try to run only service two
Compose file:
version: "3.9"
volumes:
data: null
services:
one:
image: foo
build:
context: ./
dockerfile: Dockerfile
target: stage1
command: sh -c 'sleep 5 && echo "one done"'
two:
image: foo
command: sh -c 'sleep 5 && echo "two done"'
volumes:
- data:/stuff
- ./test:/test
depends_on:
- one
Dockerfile
FROM busybox as stage1
RUN echo "Hello World"
FROM busybox
CMD sleep infinity
The output I am getting is the following. It may make sense because when reading the definition of the --build flag, Build images before starting containers., it sounds a bit like it doesn't care about what service you want to start. It just builds all images.
docker-compose up --build two
Creating network "test_default" with the default driver
Building one
[ ] Building 3.3s (6/6) FINISHED
=> [internal] load build definition from Dockerfile 0.1s
=> => transferring dockerfile: 121B 0.0s
=> [internal] load .dockerignore 0.1s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/busybox:latest 2.3s
=> [stage1 1/2] FROM docker.io/library/busybox@sha256:b5cfd4befc119a590ca1a81d6bb0fa1fb19f1fbeb 0.4s
=> => resolve docker.io/library/busybox@sha256:b5cfd4befc119a590ca1a81d6bb0fa1fb19f1fbebd0397f2 0.0s
=> => sha256:b5cfd4befc119a590ca1a81d6bb0fa1fb19f1fbebd0397f25fae164abe1e8a6a 2.29kB / 2.29kB 0.0s
=> => sha256:50e44504ea4f19f141118a8a8868e6c5bb9856efa33f2183f5ccea7ac62aacc9 527B / 527B 0.0s
=> => sha256:ffe9d497c32414b1c5cdad8178a85602ee72453082da2463f1dede592ac7d5af 1.46kB / 1.46kB 0.0s
=> => sha256:3cb635b06aa273034d7080e0242e4b6628c59347d6ddefff019bfd82f45aa7 772.78kB / 772.78kB 0.3s
=> => extracting sha256:3cb635b06aa273034d7080e0242e4b6628c59347d6ddefff019bfd82f45aa7d5 0.1s
=> [stage1 2/2] RUN echo "Hello World" 0.4s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:81893d72f6be148599854469098e9f8c80a73e28acd5991a1757033440d34e5c 0.0s
=> => naming to docker.io/library/foo 0.0s
Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them
Creating test_one_1 ... done
Creating test_two_1 ... done
Attaching to test_two_1
two_1 | two done
test_two_1 exited with code 0