Cannot create private Kubernetes registry-CodePudding

I want to create a private Kubernetes registry from this tutorial: https://www.linuxtechi.com/setup-private-docker-registry-kubernetes/

I implemented this:

Generate Self-Signed Certificate
cd /opt 
sudo mkdir certs
cd certs
sudo touch registry.key
cd /opt
sudo openssl req -newkey rsa:4096 -nodes -sha256 -keyout \
 ./certs/registry.key -x509 -days 365 -out ./certs/registry.crt
ls -l certs/

Create registry folder
cd /opt
mkdir registry

Copy-paste private-registry.yaml into /opt/registry

apiVersion: apps/v1
kind: Deployment
metadata:
  name: private-repository-k8s
  labels:
    app: private-repository-k8s
spec:
  replicas: 1
  selector:
    matchLabels:
      app: private-repository-k8s
  template:
    metadata:
      labels:
        app: private-repository-k8s
    spec:
      volumes:
      - name: certs-vol
        hostPath:
          path: /opt/certs
          type: Directory
      - name: registry-vol
        hostPath:
          path: /opt/registry
          type: Directory

      containers:
        - image: registry:2
          name: private-repository-k8s
          imagePullPolicy: IfNotPresent
          env:
          - name: REGISTRY_HTTP_TLS_CERTIFICATE
            value: "/certs/registry.crt"
          - name: REGISTRY_HTTP_TLS_KEY
            value: "/certs/registry.key"
          ports:
            - containerPort: 5000
          volumeMounts:
          - name: certs-vol
            mountPath: /certs
          - name: registry-vol
            mountPath: /var/lib/registry


kubernetes@kubernetes1:/opt/registry$ kubectl create -f private-registry.yaml
deployment.apps/private-repository-k8s created
kubernetes@kubernetes1:/opt/registry$  kubectl get deployments private-repositor                                                               y-k8s
NAME                     READY   UP-TO-DATE   AVAILABLE   AGE
private-repository-k8s   0/1     1            0           12s
kubernetes@kubernetes1:/opt/registry$

I have the following questions:

I have a control plane and 2 work nodes. Is it possible to have a folder located only on the control plane under /opt/registry and deploy images on all work nodes without using shared folders?
As alternative more resilient solution I want to have a control plane and 2 work nodes. Is it possible to have a folder located on all work nodes and on the control plane under /opt/registry and deploy images on all work nodes without using manually created shared folders? I want Kubernetes to manage repository replication on all nodes. i.e data into /opt/registry to be synchronized automatically by Kubernetes.
Do you know how I can debug this configuration? As you can see pod is not starting.

EDIT: Log file:

kubernetes@kubernetes1:/opt/registry$ kubectl logs private-repository-k8s-6ddbcd9c45-s6dfq
Error from server (BadRequest): container "private-repository-k8s" in pod "private-repository-k8s-6ddbcd9c45-s6dfq" is waiting to start: ContainerCreating
kubernetes@kubernetes1:/opt/registry$

Attempt 2:

I tried this configuration deployed from control plane:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv1
spec:
  capacity:
    storage: 256Mi # specify your own size
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  local:
    path: /opt/registry # can be any path
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions: # specify the node label which maps to your control-plane node.
        - key: kubernetes1
          operator: In
          values:
          - controlplane-1
  accessModes:
    - ReadWriteOnce # only 1 node will read/write on the path.
    # - ReadWriteMany # multiple nodes will read/write on the path

Note! control plane hostname is kubernetes1 so I changed the value into above configuration. I get this:

kubernetes@kubernetes1:~$ cd /opt/registry
kubernetes@kubernetes1:/opt/registry$ kubectl create -f private-registry1.yaml
persistentvolume/pv1 created
kubernetes@kubernetes1:/opt/registry$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                       READY   STATUS              RESTARTS       AGE
default       private-repository-k8s-6ddbcd9c45-s6dfq    0/1     ContainerCreating   0              2d1h
kube-system   calico-kube-controllers-58dbc876ff-dgs77   1/1     Running             4 (125m ago)   2d13h
kube-system   calico-node-czmzc                          1/1     Running             4 (125m ago)   2d13h
kube-system   calico-node-q4lxz                          1/1     Running             4 (125m ago)   2d13h
kube-system   coredns-565d847f94-k94z2                   1/1     Running             4 (125m ago)   2d13h
kube-system   coredns-565d847f94-nt27m                   1/1     Running             4 (125m ago)   2d13h
kube-system   etcd-kubernetes1                           1/1     Running             5 (125m ago)   2d13h
kube-system   kube-apiserver-kubernetes1                 1/1     Running             5 (125m ago)   2d13h
kube-system   kube-controller-manager-kubernetes1        1/1     Running             5 (125m ago)   2d13h
kube-system   kube-proxy-97djs                           1/1     Running             5 (125m ago)   2d13h
kube-system   kube-proxy-d8bzs                           1/1     Running             4 (125m ago)   2d13h
kube-system   kube-scheduler-kubernetes1                 1/1     Running             5 (125m ago)   2d13h

kubernetes@kubernetes1:/opt/registry$ kubectl logs private-repository-k8s-6ddbcd9c45-s6dfq
Error from server (BadRequest): container "private-repository-k8s" in pod "private-repository-k8s-6ddbcd9c45-s6dfq" is waiting to start: ContainerCreating

Unfortunately again the image is not created.

CodePudding user response：

For 1st question, you can try creating PersistentVolume with node affinity set to specific controlplane node and tie it with the deployment via PersistentVolumeClaim.Here's an example:

apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv1
spec:
  capacity:
    storage: 256Mi # specify your own size
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  local:
    path: /opt/registry # can be any path
  nodeAffinity:
    required:
      nodeSelectorTerms:
      - matchExpressions: # specify the node label which maps to your control-plane node.
        - key: kubernetes.io/hostname
          operator: In
          values:
          - controlplane-1
  accessModes:
    - ReadWriteOnce # only 1 node will read/write on the path.
    # - ReadWriteMany # multiple nodes will read/write on the path

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pv1-claim
spec: # should match specs added in the PersistenVolume
  accessModes:
    - ReadWriteOnce
  volumeMode: Filesystem
  resources:
    requests:
      storage: 256Mi

apiVersion: apps/v1
kind: Deployment
metadata:
  name: private-repository-k8s
  labels:
    app: private-repository-k8s
spec:
  replicas: 1
  selector:
    matchLabels:
      app: private-repository-k8s
  template:
    metadata:
      labels:
        app: private-repository-k8s
    spec:
      volumes:
       - name: task-pv-storage
         persistentVolumeClaim:
           claimName: pv1-claim # specify the PVC that you've created. PVC and Deployment must be in same namespace.

      containers:
        - image: registry:2
          name: private-repository-k8s
          imagePullPolicy: IfNotPresent
          env:
          - name: REGISTRY_HTTP_TLS_CERTIFICATE
            value: "/certs/registry.crt"
          - name: REGISTRY_HTTP_TLS_KEY
            value: "/certs/registry.key"
          ports:
            - containerPort: 5000
          volumeMounts:
          - name: task-pv-storage
            mountPath: /opt/registry

For question # 2, can you share the logs of your pod?

CodePudding user response：

You can try with following file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: private-repository-k8s
  labels:
    app: private-repository-k8s
spec:
  replicas: 1
  selector:
    matchLabels:
      app: private-repository-k8s
  template:
    metadata:
      labels:
        app: private-repository-k8s
    spec:
      volumes:
       - name: task-pv-storage
         persistentVolumeClaim:
           claimName: pv1-claim # specify the PVC that you've created. PVC and Deployment must be in same namespace.
      containers:
        - image: registry:2
          name: private-repository-k8s
          imagePullPolicy: IfNotPresent
          env:
          - name: REGISTRY_HTTP_TLS_CERTIFICATE
            value: "/certs/registry.crt"
          - name: REGISTRY_HTTP_TLS_KEY
            value: "/certs/registry.key"
          ports:
            - containerPort: 5000
          volumeMounts:
          - name: task-pv-storage
            mountPath: /opt/registry
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: pv1-claim
spec: # should match specs added in the PersistenVolume
  accessModes:
    - ReadWriteMany
  volumeMode: Filesystem
  resources:
    requests:
      storage: 256Mi
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv1
spec:
  capacity:
    storage: 256Mi # specify your own size
  volumeMode: Filesystem
  persistentVolumeReclaimPolicy: Retain
  local:
    path: /opt/registry # can be any path
  accessModes:
    - ReadWriteMany