I'm currently trying to understand the underlying logic of how an external storage provisioner works in Kubernetes.

I have Minikube installed, and can see that the standard StorageClass resource has the field provisioner: k8s.io/minikube-hostpath. I am also aware of the Pod running this provisioner controller, storage-provisioner, which runs on the kube-system namespace. I know that when a PVC requests some storage from the StorageClass, it uses the storage-provisioner Pod to create and supply this storage.

What I want to know is how the Kubernetes API knows this is correct Pod to use given the provisioner field in the StorageClass.

What I know so far is that the StorageClass has the label addonmanager.kubernetes.io/mode=EnsureExists and the storage-provisioner Pod has the label addonmanager.kubernetes.io/mode=Reconcile. These are different, so the Kubernetes API can't be using these as selectors to map them.

I had a look at all the other API resources using these labels and found the following with addonmanager.kubernetes.io/mode=EnsureExists:

• rolebinding.rbac.authorization.k8s.io/system:persistent-volume-provisioner
• role.rbac.authorization.k8s.io/system:persistent-volume-provisioner

I then found the following with addonmanager.kubernetes.io/mode=Reconcile:

• endpoints/k8s.io-minikube-hostpath
• serviceaccount/storage-provisioner

I used kubectl describe on the Roles resource and noticed it has a PolicyRule for the endpoints/k8s.io-minikube-hostpath. I did the same for the Rolebinding resource and noticed it has serviceaccount/storage-provisioner under Subjects.

My assumption is that this associates addonmanager.kubernetes.io/mode=EnsureExists with addonmanager.kubernetes.io/mode=Reconcile, which associates the provisioner: k8s.io/minikube-hostpath field with the storage-provisioner, thereby allowing the Kubernetes API to identify the correct Pod to use.

I'm not that well-versed in the inner-workings of the Kubernetes API, so I have a feeling I am wrong. If so how does this work?

Thank you :).

CodePudding user response：

Turns out I was looking at this all wrong. What actually happens is the storage-provisioner Pod itself is the controller, and this controller watches for any PVC requests to any StorageClass that has the provisioner: k8s.io/minikube-hostpath field; provisioning volumes respectively.

This can all be seen in the actual code implementation of the controller at https://github.com/kubernetes/minikube/blob/master/pkg/storage/storage_provisioner.go

First at line 37 we have const provisionerName = "k8s.io/minikube-hostpath"

Then at line 139 we have pc := controller.NewProvisionController(clientset, provisionerName, hostPathProvisioner, serverVersion.GitVersion) which creates the Controller.

I looked further within the source code of this function at https://github.com/kubernetes-sigs/sig-storage-lib-external-provisioner/blob/52106c755f34bc764626e2e75f08352d9c3edce5/controller/controller.go

The controller's control loops start at line 809, func (ctrl *ProvisionController) Run(ctx context.Context) {

Then it calls runClaimWorker() at line 856. I just followed the bouncing ball from here until it eventually creates the provision by calling the Provision function on line 1426, volume, result, err := ctrl.provisioner.Provision(ctx, options)

This function exists at line 64 here: https://github.com/kubernetes-sigs/sig-storage-lib-external-provisioner/blob/52106c755f34bc764626e2e75f08352d9c3edce5/examples/hostpath-provisioner/hostpath-provisioner.go#L64

Quite interesting!

CodePudding user response：

Actually Kubernetes API Server keeps record of all Kubernetes Objects. Kubernetes objects are identified by their kind,name and namespace triplet(unique identifier). A Kubernetes object is a "record of intent"--once you create the object, the Kubernetes system will constantly work to ensure that object exists.

Official docs Kubernetes Objects