Client-side throttling response from kubernetes kubectl command-CodePudding

I have a 3-node ubuntu microk8s installation and it seems to be working ok. All 3 nodes are management nodes.

On only one of the nodes, I get an error message and associated delay whenever I use a kubectl command. It looks like this:

$ time kubectl get pods
I0324 03:49:44.270996  514696 request.go:665] Waited for 1.156689289s due to client-side throttling, not priority and fairness, request: GET:https://127.0.0.1:16443/apis/authentication.k8s.io/v1?timeout=32s
NAME                      READY   STATUS    RESTARTS      AGE
sbnweb-5f9d9b977f-lw7t9   1/1     Running   1 (10h ago)   3d3h
shell-6cfccdbd47-zd2tn    1/1     Running   0             6h39m

real    0m6.558s
user    0m0.414s
sys     0m0.170s

The error message always shows a different URL each time. I tried looking up the error code (I0324) and haven't found anything useful.

The other two nodes don't show this behavior. No error message and completes the request in less than a second.

I'm new to k8s so I am not sure how to diagnose this kind of problem. Any hints on what to look for would be greatly appreciated.

CodePudding user response：

Here's a good write-up about the issue. For some cases rm -rf ~/.kube/cache will remove the issue.