Jenkins is not mounting the AWS EFS file system and using the default volume instead-CodePudding

I'm trying to use jenkins with an EFS persistence volume on EKS. however all my attempts to make it use the provided EFS file system did not succeed. What makes me wonder is that when i tested with a busybox image the EFS was successfully mounted and could see data written to the shared storage.

EFS definition

resource "aws_efs_file_system" "jenkins_shared_file_system" {
  creation_token   = "Jenkins shared file system"
  performance_mode = "generalPurpose"
  throughput_mode  = "bursting"
  encrypted        = true
  tags             = {
    Name = "Jenkins shared file system"
  }
}

resource "aws_efs_mount_target" "jenkins_efs_private_subnet_1_mount_target" {
  file_system_id  = aws_efs_file_system.jenkins_shared_file_system.id
  subnet_id       = aws_subnet.ci_cd_private_subnet_1.id
  security_groups = [aws_security_group.jenkins_efs_sg.id]
}

resource "aws_efs_mount_target" "jenkins_efs_private_subnet_2_mount_target" {
  file_system_id  = aws_efs_file_system.jenkins_shared_file_system.id
  subnet_id       = aws_subnet.ci_cd_private_subnet_2.id
  security_groups = [aws_security_group.jenkins_efs_sg.id]
}

resource "aws_efs_access_point" "jenkins_efs_access_point" {
  file_system_id = aws_efs_file_system.jenkins_shared_file_system.id
  tags = {
    Name = "Jenkins EFS access point"
  }
  posix_user {
    gid = 1000
    uid = 1000
  }
  root_directory  {
    path          = "/jenkins"
    creation_info  {
      owner_uid   = 1000
      owner_gid   = 1000
      permissions = 777
    }
  }
}

The CSI driver is installed following instructions from https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html

here is the persistence configurations

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com

---

apiVersion: v1
kind: PersistentVolume
metadata:
  name: efs-pv
  namespace: jenkins
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: efs-sc
  csi:
    driver: efs.csi.aws.com
    volumeHandle: fs-12345::fsap-12345

---

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-pvc
  namespace: jenkins
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-sc
  resources:
    requests:
      storage: 5Gi

and the jenkins values config

controller:
  componentName: jenkins-controller
  image: "jenkins/jenkins"
  tag: lts-jdk11
  imagePullPolicy: IfNotPresent
  installPlugins: false
  disableRememberMe: false
  resources:
    requests:
      cpu: 2
      memory: 2Gi
    limits:
      cpu: 6
      memory: 4Gi
  runAsUser: 1000
  fsGroup: 1000
  serviceType: ClusterIP
  persistence:
    enabled: true
    existingClaim: efs-pvc
    storageClassName: efs-sc

  ingress:
    enabled: true
    apiVersion: "networking.k8s.io/v1"
    ingressClassName: nginx
    kubernetes.io/ingress.class: nginx
    rules:
    - host: foo.jenkins.com
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: jenkins
              port:
                number: 80
    tls:
     - secretName: jenkins-tls
       hosts:
         - foo.jenkins.com

the outout before deploying jenkins with helm

kubernetes git:(jenkins) ✗ kc get sc,pv,pvc -n jenkins
NAME                                        PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/efs-sc          efs.csi.aws.com         Delete          Immediate              false                  11m
storageclass.storage.k8s.io/gp2 (default)   kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  69m

NAME                      CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
persistentvolume/efs-pv   5Gi        RWX            Retain           Bound    jenkins/efs-pvc   efs-sc                  11m

NAME                            STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/efs-pvc   Bound    efs-pv   5Gi        RWX            efs-sc         11m

and after deploying

NAME                                        PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
storageclass.storage.k8s.io/efs-sc          efs.csi.aws.com         Delete          Immediate              false                  15m
storageclass.storage.k8s.io/gp2 (default)   kubernetes.io/aws-ebs   Delete          WaitForFirstConsumer   false                  73m

NAME                                                        CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM             STORAGECLASS   REASON   AGE
persistentvolume/efs-pv                                     5Gi        RWX            Retain           Bound    jenkins/efs-pvc   efs-sc                  15m
persistentvolume/pvc-94adfdfb-a1db-4f16-8189-84ac20474607   8Gi        RWO            Delete           Bound    jenkins/jenkins   gp2                     12s

NAME                            STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
persistentvolumeclaim/efs-pvc   Bound    efs-pv                                     5Gi        RWX            efs-sc         15m
persistentvolumeclaim/jenkins   Bound    pvc-94adfdfb-a1db-4f16-8189-84ac20474607   8Gi        RWO            gp2            17s

The output of mount when i exec inside the pod shows no NFS mounted volume. which is really weird

Any help is really appreciated. thank you !

CodePudding user response：

A good rest and a clear mind helped me to fix this issue after an entire day of hitting my head against the wall. The problem is that persistence block should be independent and not under the controller block.

persistence:
    enabled: true
    existingClaim: efs-pvc
    storageClassName: efs-sc

controller:
  componentName: jenkins-controller
  image: "jenkins/jenkins"
  tag: lts-jdk11
  imagePullPolicy: IfNotPresent
  installPlugins: false
  disableRememberMe: false
  resources:
    requests:
      cpu: 2
      memory: 2Gi
    limits:
      cpu: 6
      memory: 4Gi
  runAsUser: 1000
  fsGroup: 1000
  serviceType: ClusterIP
  

  ingress:
    enabled: true
    apiVersion: "networking.k8s.io/v1"
    ingressClassName: nginx
    kubernetes.io/ingress.class: nginx
    rules:
    - host: foo.jenkins.com
      http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: jenkins
              port:
                number: 80
    tls:
     - secretName: jenkins-tls
       hosts:
         - foo.jenkins.com