How to set chown in kubernetes deployment persistently?-CodePudding

I'm running a mongodb instance as a kubernetes pod in a single node cluster (bare metal ubuntu machine). The volume is configured ReadWriteOnce as the mongodb pod is accessed only by pods in one node.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongo
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongo
  strategy:
    type: Recreate
  template:
    metadata:
      labels:
        app: mongo
    spec:
      hostname: mongo
      volumes:
        - name: data
          persistentVolumeClaim:
            claimName: data
        - name: restore
          persistentVolumeClaim:
            claimName: restore
      containers:
        - name: mongo
          image: mongo:4.4.14
          args: ["--auth"]
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 27017
          volumeMounts:
            - mountPath: /data/db
              name: data
            - mountPath: /restore
              name: restore

But from time to time I cannot run commands like inserting documents to a non existing collection or running mongodump. Then I do get the error MongoServerError: 1: Operation not permitted. This is caused by a chown problem: ls -ld /data/db/ is returning

drwxr-sr-x 4 nobody 4294967294 16384 Jun 28 18:19 /data/db/

I can fix the problem by running

chown mongodb:mongodb /data/db

But after some time it changes again, so the same problem happens again and I have to rerun the chown mongodb:mongodb /data/db

I tried to set

securityContext:
    runAsUser: 1000
    fsGroup: 2000

But then the mongodb pod is failing:

{"t":{"$date":"2022-07-03T10:09:24.379 00:00"},"s":"I",  "c":"CONTROL",  "id":23285,   "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2022-07-03T10:09:24.383 00:00"},"s":"W",  "c":"ASIO",     "id":22601,   "ctx":"main","msg":"No TransportLayer configured during NetworkInterface startup"}
{"t":{"$date":"2022-07-03T10:09:24.384 00:00"},"s":"I",  "c":"NETWORK",  "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2022-07-03T10:09:24.384 00:00"},"s":"I",  "c":"STORAGE",  "id":4615611, "ctx":"initandlisten","msg":"MongoDB starting","attr":{"pid":1,"port":27017,"dbPath":"/data/db","architecture":"64-bit","host":"mongodb"}}
{"t":{"$date":"2022-07-03T10:09:24.384 00:00"},"s":"I",  "c":"CONTROL",  "id":23403,   "ctx":"initandlisten","msg":"Build Info","attr":{"buildInfo":{"version":"4.4.14","gitVersion":"0b0843af97c3ec9d2c0995152d96d2aad725aab7","openSSLVersion":"OpenSSL 1.1.1f  31 Mar 2020","modules":[],"allocator":"tcmalloc","environment":{"distmod":"ubuntu2004","distarch":"x86_64","target_arch":"x86_64"}}}}
{"t":{"$date":"2022-07-03T10:09:24.384 00:00"},"s":"I",  "c":"CONTROL",  "id":51765,   "ctx":"initandlisten","msg":"Operating System","attr":{"os":{"name":"Ubuntu","version":"20.04"}}}
{"t":{"$date":"2022-07-03T10:09:24.384 00:00"},"s":"I",  "c":"CONTROL",  "id":21951,   "ctx":"initandlisten","msg":"Options set by command line","attr":{"options":{"net":{"bindIp":"*"},"security":{"authorization":"enabled"}}}}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"E",  "c":"STORAGE",  "id":20557,   "ctx":"initandlisten","msg":"DBException in initAndListen, terminating","attr":{"error":"IllegalOperation: Attempted to create a lock file on a read-only directory: /data/db"}}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"REPL",     "id":4784900, "ctx":"initandlisten","msg":"Stepping down the ReplicationCoordinator for shutdown","attr":{"waitTimeMillis":10000}}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"COMMAND",  "id":4784901, "ctx":"initandlisten","msg":"Shutting down the MirrorMaestro"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"SHARDING", "id":4784902, "ctx":"initandlisten","msg":"Shutting down the WaitForMajorityService"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"NETWORK",  "id":20562,   "ctx":"initandlisten","msg":"Shutdown: going to close listening sockets"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"NETWORK",  "id":4784905, "ctx":"initandlisten","msg":"Shutting down the global connection pool"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"STORAGE",  "id":4784906, "ctx":"initandlisten","msg":"Shutting down the FlowControlTicketholder"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"-",        "id":20520,   "ctx":"initandlisten","msg":"Stopping further Flow Control ticket acquisitions."}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"NETWORK",  "id":4784918, "ctx":"initandlisten","msg":"Shutting down the ReplicaSetMonitor"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"SHARDING", "id":4784921, "ctx":"initandlisten","msg":"Shutting down the MigrationUtilExecutor"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"CONTROL",  "id":4784925, "ctx":"initandlisten","msg":"Shutting down free monitoring"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"STORAGE",  "id":4784927, "ctx":"initandlisten","msg":"Shutting down the HealthLog"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"STORAGE",  "id":4784929, "ctx":"initandlisten","msg":"Acquiring the global lock for shutdown"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"-",        "id":4784931, "ctx":"initandlisten","msg":"Dropping the scope cache for shutdown"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"FTDC",     "id":4784926, "ctx":"initandlisten","msg":"Shutting down full-time data capture"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"CONTROL",  "id":20565,   "ctx":"initandlisten","msg":"Now exiting"}
{"t":{"$date":"2022-07-03T10:09:24.386 00:00"},"s":"I",  "c":"CONTROL",  "id":23138,   "ctx":"initandlisten","msg":"Shutting down","attr":{"exitCode":100}}

I also tried

  initContainers:
    - name: mongodb-init
      image: mongo:4.4.14
      command: ["sh", "-c", "chown 999:999 /data/db"]
      volumeMounts:
        - name: data
          mountPath: /data/db

But after some time I again get:

drwxr-sr-x 4 nobody 4294967294 16384 Jun 28 18:19 /data/db/

I do not understand what is causing this behaviour.

CodePudding user response：

It might be useful to run ls -n /data/db to see what uid has ownership of the directory when it's failing. nobody is not useful information.

You set securityContext to 1000, but in the initContainer you set ownership to 999. That won't work.

What user is the container running as? run id.

What user is the mongo process using? There's no assigned user in the Deployment.spec, so I imagine it's a mongoDB config setting. (docs say uid is 999)

Figure out which user id needs to own that directoryissue. Then a possible solution is to mount the path /data with the storage provisioner, then create the path /data/db with an initcontainer for the mongoDB deployment using the correct uid. That way the directories will have the correct ownership.

CodePudding user response：

If you set securityContext then according to the official documentation:

the runAsUser field specifies that for any Containers in the Pod, all processes run with user ID 1000.

there is no fear of mismatching between different processes of the container.

Perhaps you could give (although not recommended) root user access to the pod by setting runAsUser to 0? Just to see if security context can fix the issue. It should.