K8s Containers dont start ImagePullBackOff, ErrImagePull-CodePudding

I am trying to start up a couple of containers locally using k8s but container creation is stopped cause of ImagePullBackOff, ErrImagePull. The yaml is fine, tested it on another workstation. And i can pull images using regular docker. But it fails in k8s/minikube environment

Error container logs is

Error from server (BadRequest): container "mongo-express" in pod "mongoexpress-deployment-bd7cf697b-nc4h5" is waiting to start: trying and failing to pull image

Error in minikube dashboard is

Failed to pull image "docker.io/mongo": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry-1.docker.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

I tried pulling the image to my local docker cache and running

eval $(minikube docker-env)

But i keep getting this error. It doesnt see local image repository and it doesnt dowload the image by itself.

I am 100% sure it has something to do with user access on Fedora. But dont have any idea what to do, and i've been trying to fix this for a couple of days :(.

Please help, thank you

Dont know if this helps: I tried using k3s. Image pull is successful, but minikube isnt compatible with it on Fedora.

Also... If i try using docker without sudo it doesnt pull images. With sudo it pulls.

OS is Fedora, and i am using docker, kubernetes, minikube, podman as driver.

- linux version
NAME="Fedora Linux"
VERSION="35 (Workstation Edition)"
- kubectl version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.5", 
- docker version
Version:          20.10.12
- minikube version
minikube version: v1.25.2

I am trying to start up locally this yaml file

apiVersion: v1
kind: Secret
metadata:
  name: mongodb-secret
type: Opaque
data:
  mongo-root-username: dXNlcm5hbWU=
  mongo-root-password: cGFzc3dvcmQ=
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: mongodb-configmap
data:
  database_url: mongodb-service
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongoexpress-deployment
  labels:
    app: mongoexpress
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongoexpress
  template:
    metadata:
      labels:
        app: mongoexpress
    spec:
      containers:
        - name: mongo-express
          image: mongo-express
          ports:
            - containerPort: 8081
          env:
            - name: ME_CONFIG_MONGODB_ADMINUSERNAME
              valueFrom:
                secretKeyRef:
                  name: mongodb-secret
                  key: mongo-root-username
            - name: ME_CONFIG_MONGODB_ADMINPASSWORD
              valueFrom:
                secretKeyRef:
                  name: mongodb-secret
                  key: mongo-root-password
            - name: ME_CONFIG_MONGODB_SERVER
              valueFrom:
                configMapKeyRef:
                  name: mongodb-configmap
                  key: database_url
            - name: WHATEVER
              value: Someconfig
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-deployment
  labels:
    app: mongodb
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
        - name: mongodb
          image: mongo
          ports:
            - containerPort: 27017
          env:
            - name: MONGO_INITDB_ROOT_USERNAME
              valueFrom:
                  secretKeyRef:
                    name: mongodb-secret
                    key: mongo-root-username
            - name: MONGO_INITDB_ROOT_PASSWORD
              valueFrom:
                  secretKeyRef:
                    name: mongodb-secret
                    key: mongo-root-password
---
apiVersion: v1
kind: Service
metadata:
  name: mongodb-service
spec:
  selector:
    app: mongodb
  ports:
    - protocol: TCP
      port: 27017
      targetPort: 27017
---
apiVersion: v1
kind: Service
metadata:
  name: mongoexpress-service
spec:
  selector:
    app: mongoexpress-deployment
  type: LoadBalancer
  ports:
    - protocol: TCP
      port: 8081
      targetPort: 8081
      nodePort: 30000

CodePudding user response：

Based on the comments, my suggestion is to use the docker driver, since Docker has been installed in the system and is the preferred stable driver.

minikube start --driver=docker

You can also set this as the default driver.

minikube config set driver docker
minikube start

That doesn't explain why it doesn't work with podman, though.

CodePudding user response：

It is a kind of workaround for the problem, if you can pull image using docker pull then do it on all the worker nodes and then add an

ImagePullPolicy:IfNotPresent

inside the yamls where you are mentioning image name, then k8s will first check whether it is present inside the machine and if yes then directly use it

CodePudding user response：

Please double check if you mistype the path por the image.

The ImagePullBackOff error occurs when the image path is incorrect, the network fails, or the kubelet does not succeed in authenticating with the container registry. Kubernetes initially throws the ErrImagePull error, and then after retrying a few times, “pulls back” and schedules another download attempt.

An ImagePullBackoff error when it is unable to pull an image from a registry. The error might occur because the image cannot be found or because your nodes do not have permissions to pull from the registry.