Failed to pull image-CodePudding

I am following the tutorial here about kserve https://github.com/kserve/modelmesh-serving/blob/main/docs/quickstart.md

Is this my docker&k8s issue? I have spent hours trying to debug but to no avail.

I am getting the following error:


Events:
  Type     Reason     Age    From               Message
  ----     ------     ----   ----               -------
  Normal   Scheduled  4m20s  default-scheduler  Successfully assigned modelmesh-serving/modelmesh-serving-mlserver-0.x-77cc8fd548-xdgvr to minikube
  Normal   Pulling    4m18s  kubelet            Pulling image "kserve/modelmesh:v0.9.0-rc0"
  Normal   Pulled     3m18s  kubelet            Successfully pulled image "kserve/modelmesh:v0.9.0-rc0" in 59.419620166s
  Normal   Created    3m18s  kubelet            Created container mm
  Normal   Started    3m17s  kubelet            Started container mm
  Normal   Pulling    3m17s  kubelet            Pulling image "seldonio/mlserver:0.5.2"
  Warning  Failed     68s    kubelet            Failed to pull image "seldonio/mlserver:0.5.2": rpc error: code = Unknown desc = context deadline exceeded
  Warning  Failed     68s    kubelet            Error: ErrImagePull
  Normal   Pulling    68s    kubelet            Pulling image "kserve/modelmesh-runtime-adapter:v0.9.0-rc0"

CodePudding user response：

The image you try to pull exists and is available here. However, from what I see it's very big (2.74 GB). That probably means that you experienced a timeout while pull the image.

Depending on your Kubernetes cluster and use case you can do the following:

Try a few times and it'll pull the image at some point
Increase the --runtime-request-timeout in your kubelet
Execute docker pull seldonio/mlserver:0.5.2

CodePudding user response：

As mentioned by @Rafał Leszko, The image “seldonio/mlserver:0.5.2” you are trying to pull is very large which possibly throws the error: ErrImagePull: context deadline error. You can still pull the image with docker pull after which the scheduling succeedes.

When the timeout is exceeded, kubelet will cancel the request, throw out an error. The possible workaround is by setting or increasing the parameter --runtime-request-timeout duration via the config file(in /var/lib/kubelet) by adjusting the timeout and then pull the image by running the command docker pull imagename.

See kubelet-config-file for more information.