Yesterday i pushed the base image layer for my app that contained the environment needed to run my_app.
That push was massive but it is done and up in my repo.
This is currently the image situation in my local machine:
➜ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
dr_prof_patrick/my_app my_app_v0 7e4cb75b4735 22 minutes ago 5.36GB
dr_prof_patrick/my_app my_app_base_image b1cccd87e4f7 37 hours ago 5.35GB
python 3.8 67ec76d9f73b 8 days ago 909MB
python 3 f48ea80eae5a 8 days ago 917MB
I introduced some minor changes to the image as you can see here:
➜ docker history dr_prof_patrick/my_app:my_app_v0
IMAGE CREATED CREATED BY SIZE COMMENT
7e4cb75b4735 22 minutes ago /bin/sh -c #(nop) CMD ["python3" "main.py"] 0B
55fe27affa9a 22 minutes ago /bin/sh -c pip install -r requirements.txt 16.1kB
3eba19411e42 22 minutes ago /bin/sh -c #(nop) WORKDIR /my_app 0B
54249235bcba 22 minutes ago /bin/sh -c #(nop) COPY dir:7f9da91b4e3f9ed60… 6.03MB
b1cccd87e4f7 37 hours ago /bin/sh -c pip install -r requirements.txt 4.04GB
<missing> 37 hours ago /bin/sh -c #(nop) COPY file:8e879a6a889ff22f… 305B
<missing> 37 hours ago /bin/sh -c apt-get install ffmpeg libsm6 lib… 385MB
<missing> 37 hours ago /bin/sh -c apt-get update 17.7MB
<missing> 8 days ago /bin/sh -c #(nop) CMD ["python3"] 0B
<missing> 8 days ago /bin/sh -c set -ex; wget -O get-pip.py "$P… 8.31MB
<missing> 8 days ago /bin/sh -c #(nop) ENV PYTHON_GET_PIP_SHA256… 0B
It tries to push the base layer again which takes like 2 hours.
➜ docker push dr_prof_patrick/my_app:my_app_v0
The push refers to repository [docker.io/dr_prof_patrick/my_app]
548f7bc62c43: Layer already exists
fa5433c54740: Layer already exists
f09b381b1d57: Pushing 1.102MB/4.041GB
a0f39ee33d3d: Layer already exists
a1e799af3370: Pushing 8.842MB/384.5MB
c93f6a714096: Pushing 1.77MB/17.7MB
e9df9d3bdd45: Layer already exists
1271cc224a6b: Layer already exists
740ef99eafe1: Pushing 5.42MB/48.56MB
b7b662b31e70: Pushing 6.628MB/18.47MB
6f5234c0aacd: Waiting
8a5844586fdb: Waiting
a4aba4e59b40: Waiting
5499f2905579: Waiting
a36ba9e322f7: Waiting
I dont know how my workflow prevented docker from uploading only the latest changes - any help would be appreciated.
EDIT
Thought the docker file that was used to create the my_app_v0 could help:
FROM dr_prof_patrick/my_app:my_app_base_image
COPY . /my_app
WORKDIR /my_app
RUN pip install -r requirements.txt
CMD ["python3", "main.py"]
CodePudding user response：
Docker invalidates COPY layers once the context changes -- regardless of what the next steps actually depend on. Copy files at the last possible moment -- in your case, copy requirements.txt first, and the rest later. Like this:
FROM dr_prof_patrick/my_app:my_app_base_image
WORKDIR /my_app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
CMD ["python3", "main.py"]
Also take a look at your .dockerignore and don't copy useless files. The best strategy I see used is to use .dockerignore as a whitelist, not a blacklist, by ignoring everything first and then un-ignoring the files you need:
*
!requirements.txt
CodePudding user response：
docker push pushes all layers (5 at the time by default) of the image that are not equal to the image in the repository (aka the layers that did not change), not a single layer, in the end resulting in a new image in your repository.
You can see it as if Docker made a diff between the local and the remote image and pushed only the differences between those two, which will end up being a new image - equal to the one you have in your machine but with "less work" to reach the desired result since it doesn't need to push literally all the layers.
In your case it's taking a lot of time since the 4 Gb layer changed (since the content of what you are copying is different now), making Docker push a big part of the size of your image.
Link for the docker push documentation, if needed: https://docs.docker.com/engine/reference/commandline/push/