When I try to create a pod in kubernetes with my image in my Harbor registry,I got an ErrImagePull Error, which looks like that:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 10s default-scheduler Successfully assigned test/test-pod to ubuntu-s-2vcpu-2gb-ams3-01-slave01
Normal Pulling 9s kubelet Pulling image "my.harbor.com/test/nginx:1.18.0"
Warning Failed 9s kubelet Failed to pull image "my.harbor.com/test/nginx:1.18.0": rpc error: code = Unknown desc = failed to pull and unpack image "my.harbor.com/test/nginx:1.18.0": failed to resolve reference "my.harbor.com/test/nginx:1.18.0": failed to do request: Head https://my.harbor.com/v2/test/nginx/manifests/1.18.0: x509: certificate signed by unknown authority
Warning Failed 9s kubelet Error: ErrImagePull
Normal BackOff 8s kubelet Back-off pulling image "my.harbor.com/test/nginx:1.18.0"
Warning Failed 8s kubelet Error: ImagePullBackOff
I think the crucial problem is that 'x509: certificate signed by unknown authority
but I really don't know what's wrong, since I copied my CA to both kubernetes master node and slave node, and they can both login to harbor and run docker pull my.harbor.com/test/nginx:1.18.0
to pull the image successfully.
I had been bothered days for this, any reply would be grateful.
CodePudding user response:
I copied the ca.crt to /etc/docker/certs.d/my.harbor.com/
This will make it work for the docker engine, which you've shown.
along with my.harbor.cert and my.harbor.com.key
I'd consider that a security violation and no longer trust the secret key for your harbor host. The private key should never need to be copied off of the host.
and I also copied the ca.crt to /usr/local/share/ca-certificates/ and run command update-ca-certificates to update.
That's the step that should have resolved this.
You can verify that you loaded the certificate with:
openssl s_client -connect my.harbor.com:443 -showcerts </dev/null
If the output for that doesn't include a message like Verification: OK
, then you didn't configure the host certificates correctly and need to double check the steps for your Linux distribution. It's important to check this on each of your nodes. If you only update the manager and pull your images from a worker, that worker will still encounter TLS errors.
If openssl
shows a successful verification, then check your Kubernetes node. Depending on the CRI, it could be caching old certificate data and need to be restarted to detect the change on the host.
As for CRI, I don't know what is it
Container Runtime Interface, part of your Kubernetes install. By default, this is containerd
on many Kubernetes distributions. containerd
and other CRI's (except for docker-shim
) will not look at the docker configuration.