Docker-compose volumes: persistant, but where?-CodePudding

I've been struggling with this for weeks on-and-off now, and I can't figure out what I'm doing wrong. It must be something incredibly simple, because I followed many tutorials to the letter and tried just about every command available.

I try to do something very basic: create one (or more) containers with bind volume(s) so I can edit my code locally. I've stripped down a simple test setup to this:

services:
  nginx:
    container_name: nginx
    image: nginx:stable-alpine
    ports: 
      - "80:80"
    volumes:
      - "./data:/var/www/html"

I've tried different images, built the above through a Dockerfile and configuring nginx to act as you would want it to (serving php/html from /var/www/html), but that made no difference (and does not seem to be directly related to my problem).

Now, from everything I've seen and read, this should create a directory './data' in the folder where the docker-compose.yml file is. But it doesn't. I know an index.php in ./data won't work in this scenario because nginx isn't configured yet, but still I'd expect the contents of ./data to be available when accessing the container in a terminal.

When I use the terminal in the container (through Docker Desktop) I can browse the files in the container, but there is no link to my local ./data folder. Configuring nginx to serve files from the /var/www/html directory works as expected: the browser window shows whatever is punt in the document root.

Now, the really weird part: the contents of the container's /var/www/html directory seem to be persisent. If I put an index.php in it, use docker-compose down, build, prune, up --build or even removing all containers, images, volumes manually keeps the file in /var/www/html after I build and up the container again. Only a system reboot clears this mysterious phenomenon.

From what I understand, bringing down a container should whipe all changes made in the container's filesystem. Re-upping or rebuilding from the image should give a fresh start, but it doesn't.

As no volume is created where I expect it to be (./data on my local dev machine), but any files I put in /var/www/html of the container seem persistent throughout the whole process, I can only imagine some caching is happening -or- a volume is created somewhere in a place I don't expect (that is cleared after a system reboot).

Both docker cli and Docker Desktop app show no volumes whatsoever:

$ docker volume ls
DRIVER    VOLUME NAME

Inspecting the running container in Docker Desktop says (/var/www/test/data is supposed to be my local directory):

Mounts
/VAR/WWW/HTML     /var/www/test/data

My questions:

why is no volume directory ./data created (or used, if I create it manually)?
why are the contents of /var/www/html in the nginx container kind of persistent (even after downing/clearing/pruning/rebuilding everything)?

I have looked everywhere, but I am looking in the wrong places or using the wrong search terms... I cannot find this problem popping up elsewhere. Could it be my system, my specific setup? Could it be write permissions on a file system level?

Typical workflow:

create docker-compose.yml file
run docker-compose build (tried skipping this and running docker-compose up --build)
run docker-compose up

No errors are shown in the process, and nginx logs tell me the server is up and running all ok.

System/version info:

$ cat /etc/os-release
NAME="Linux Mint"
VERSION="21 (Vanessa)

$ docker --version
Docker version 20.10.22, build 3a2c30b

$ docker-compose --version
Docker Compose version v2.13.0

$ docker version
Client: Docker Engine - Community
 Cloud integration: v1.0.29
 Version:           20.10.22
 API version:       1.41
 Go version:        go1.18.9
 Git commit:        3a2c30b
 Built:             Thu Dec 15 22:28:04 2022
 OS/Arch:           linux/amd64
 Context:           desktop-linux
 Experimental:      true

Server: Docker Desktop 4.15.0 (93002)
 Engine:
  Version:          20.10.21
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.18.7
  Git commit:       3056208
  Built:            Tue Oct 25 18:00:19 2022
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.10
  GitCommit:        770bd0108c32f3fb5c73ae1264f7e503fe7b2661
 runc:
  Version:          1.1.4
  GitCommit:        v1.1.4-0-g5fd4c4d
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Tried just about everything, including system reboots and creating the scenario over and over. The problem remains. I'v checked a loooot of questions and answers here, but the closest I come to this problem is people trying to mount volumes in the image... That's not what I'm doing, I believe... I work with docker-compose.yml solely (until I get this working).

CodePudding user response：

/var/www/test/data is supposed to be my local directory
Server: Docker Desktop 4.15.0 (93002)

Docker Desktop runs Docker in a hidden Linux virtual machine. This is true on all platforms, even native Linux. This means containers can't necessarily access the entire host filesystem.

On Linux, using Docker Desktop, only the home directory is shared with the Docker VM by default. This is one of a couple of settings that will apparently work but have unexpected side effects: from your other descriptions it looks like you are bind-mounting something (the directory contents are being persisted) but it's most likely a directory in the hidden Linux VM.

I'd suggest three main options here:

Move your project into your home directory; this is what you've already done in your other answer.
Add the host's /var/www directory under the "File Sharing" tab in the Docker Desktop "Settings" page.
Don't use Docker Desktop on native Linux. Uninstall it and use the standalone Docker Engine instead. (If you already have it installed you may need to run docker context use default.)

CodePudding user response：

@Garuno you are on to something about the directory! I've copied docker-compose.yml to a new directory in my ~/Projects folder and it now works as expected... the local directory is persisted, and gets mounted after rebuild/re-up... yay!

Works:

:~/Projects/test3$ ls -al
total 16
drwxrwxr-x 3 jeroen jeroen 4096 Dec 20 14:29 .
drwxr-xr-x 3 jeroen jeroen 4096 Dec 20 14:28 ..
drwxr-xr-x 2 jeroen jeroen 4096 Dec 20 14:30 data
-rw-rw-r-- 1 jeroen jeroen  111 Dec 20 14:28 docker-compose.yml

Does not work:

:/var/www/test2$ ls -al
total 12
drwxr-xr-x 2 jeroen jeroen 4096 Dec 20 14:27 .
drwxr-xr-x 9 jeroen jeroen 4096 Dec 20 11:22 ..
-rw-r--r-- 1 jeroen jeroen  110 Dec 20 14:15 docker-compose.yml

The obvious difference is in some group rights on the current directory and the docker-compose.yml file, but even when I chmod /var/www/test2 exactly the same as the Project directory, the bind volume doesn't work.

It does, however, in /home/jeroen/Projects/test3 ... the data directory gets created, and files touched locally or via the terminal in the container command line show up as they should.

I'm baffled how this would work, or what goes wrong, but it seems my /var/www directory just won't work. For now, I'm just happy I got it to work somehow.

If you have any info that sheds extra light on this, please post it as an answer, I'd gladly mark it as the solution for others that might face the same mystery.