What can be the cause of different permissions inside a fresh Docker container?-CodePudding

I have the situation that a fresh Docker container leads to different results when executed on different machines. Specifically the file system permissions are different:

docker run --rm my-private-image /bin/sh -c "ls -l /"

...
drwxr-xr-t    2 root     root          4096 Dec 18 23:33 tmp
vs.
drwxrwxrwt    2 root     root          4096 May 29  2020 tmp

The problem exists only recently, when I moved the /var/lib/docker directory to another partition. So most likely I screwed it up myself.
I already deleted the image in question to force Docker to fetch it freshly to correct the mistake, but no luck here (it corrected different owners/groups, but not the permissions).

The image in question (I called it my-private-image) is based upon trafex/alpine-nginx-php7, which is responsible for /tmp.

My question now is:

How/Why does Docker keep the different layers even after I deleted the image?
And what can I do the rectify the situation?

(I could of course just delete the whole /var/lib/docker and reinstall Docker to solve that, but I want to understand Docker's internals better)

CodePudding user response：

Doing a quick check of the image history, nothing jumps out as modifying /tmp, so I'm pretty sure it will be the base layer:

$ docker history trafex/alpine-nginx-php7                                                                                                                                                                         
IMAGE          CREATED        CREATED BY                                      SIZE      COMMENT
d03c5e607375   5 months ago   /bin/sh -c #(nop)  HEALTHCHECK &{["CMD-SHELL…   0B        
<missing>      5 months ago   /bin/sh -c #(nop)  CMD ["/usr/bin/supervisor…   0B        
<missing>      5 months ago   /bin/sh -c #(nop)  EXPOSE 8080                  0B        
<missing>      5 months ago   /bin/sh -c #(nop) COPY --chown=nobodydir:922…   58B       
<missing>      5 months ago   /bin/sh -c #(nop) WORKDIR /var/www/html         0B        
<missing>      5 months ago   /bin/sh -c #(nop)  USER nobody                  0B        
<missing>      5 months ago   /bin/sh -c chown -R nobody.nobody /var/www/h…   1.15kB    
<missing>      5 months ago   /bin/sh -c mkdir -p /var/www/html               0B        
<missing>      5 months ago   /bin/sh -c #(nop) COPY file:12908bc96c18db8f…   459B      
<missing>      5 months ago   /bin/sh -c #(nop) COPY file:ba2b24ac43720041…   27B       
<missing>      5 months ago   /bin/sh -c #(nop) COPY file:791d5f77ccca3899…   2.16kB    
<missing>      5 months ago   /bin/sh -c #(nop) COPY file:e7e19bb0340c77dd…   2.91kB    
<missing>      5 months ago   /bin/sh -c ln -s /usr/bin/php8 /usr/bin/php     13B       
<missing>      5 months ago   /bin/sh -c apk --no-cache add   curl   nginx…   121MB     
<missing>      5 months ago   /bin/sh -c #(nop)  LABEL Description=Lightwe…   0B        
<missing>      5 months ago   /bin/sh -c #(nop)  LABEL Maintainer=Tim de P…   0B        
<missing>      6 months ago   /bin/sh -c #(nop)  CMD ["/bin/sh"]              0B        
<missing>      6 months ago   /bin/sh -c #(nop) ADD file:f278386b0cef68136…   5.6MB

Inspecting the image, that sha looks like the 72e8... layer:

        "RootFS": {
            "Type": "layers",
            "Layers": [
                "sha256:72e830a4dff5f0d5225cdc0a320e85ab1ce06ea5673acfe8d83a7645cbd0e9cf",
                "sha256:8e1de91f5d76729f122777c082e5a0a04bf157438b40223add6e5b0d74974b4a",
                "sha256:02bef6095ce13955a5feda9394ceb41e3ff06b1547c179550495f0ea51fa7d81",
                "sha256:3dc8fd1f904d7ed46a9ef40a6083f7da2dd76c5d33acbfe5b49a57054a7e691a",
                "sha256:5e1da4b6ee09de869cc02ebd7b0ef093da7b1b3cf5e1a5258398960679252d1d",
                "sha256:3c93b50985ab6d6ff15f51828432c04d0163232e46d510131ce6fafb44832da9",
                "sha256:4da6c12a08b014c74acc2348d4e5fec101a10f27caf2ac2cd80596c1775cd6ad",
                "sha256:f44b520b457aa8b531c86e63f2c9c391d73f5eca898e698713c679b684637890",
                "sha256:a7a1e7d3913b5bd1b0d13653840349ce407c81beec236055edab400c5a1a0096",
                "sha256:e68f2db5551b5b4c15ad89dae5ee54ea4a3b864c71475146cedfda1ee6d75ec7"
            ]
        },

In docker, those are currently stored within /var/lib/docker/image/overlay2/layerdb and that has a pointer to the overlay2 folder:

# cat image/overlay2/layerdb/sha256/72e830a4dff5f0d5225cdc0a320e85ab1ce06ea5673acfe8d83a7645cbd0e9cf/cache-id 
a2cd81767ad8c4e1fc556585df7f9904089e4d3884304f1c3a343c234b9a8f08

# ls -al overlay2/a2cd81767ad8c4e1fc556585df7f9904089e4d3884304f1c3a343c234b9a8f08/diff/tmp
total 8
drwxrwxrwt  2 root root 4096 Jun 15  2021 .
drwxr-xr-x 19 root root 4096 Jul  7 15:51 ..

If your filesystem has been corrupted at this level in the move, I'd personally delete the copy (the entire /var/lib/docker directory, not just pieces from it) and start over. Docker will not pull layers that it already has locally, and you are probably seeing just the first issue with the /tmp folder permissions.