Kubectl how to work with different clusters (contexts) at the same time-CodePudding

In this case, I have multiple Kubernetes clusters and want to work on different clusters at the same time. (Will keep it as 2 clusters to make it simple)

As described in Kubernetes documentation I have configured two clusters (will call them dc1-main and dc2-main)

I'm logging into a node where kubectl is, with an application support user (e.g. appuser)

At the same time on two sessions to the management server I logged in with appuser.

In this case, I want to use kubectl to manage one context on each session.

But, if I set the active context as below, both sessions to the server reflect get the change as both are referring to the the same config file (which has both contexts)

kubectl config use-context dc1-main

Or the other option in the document is to pass the context with the command as an argument. Which makes the command quite complicated.

kubectl --context="dc2-main" get nodes

I'm looking at an easy way to change this quickly to change the context without affecting the other session. Which could be most likely an environment variable. Not so sure if this is the easiest though.

I went through the kubectl project GitHub and found a change has been requested long time ago for something similar to this and talking about env variables.

Any better suggestions?

CodePudding user response：

The standard Kubernetes client libraries support a $KUBECONFIG environment variable. This means that pretty much every tool supports it, including Helm and any locally-built tools you have. You can set this to a path to a cluster-specific configuration. Since it's an environment variable, each shell will have its own copy of it.

export KUBECONFIG="$HOME/.kube/dc1-main.config"
kubectl get nodes

In your shell dotfiles, you can write a simple shell function to set this

kubecfg() {
  export KUBECONFIG="$HOME/.kube/$1.config"
}

In my use I only have one context (user/host/credentials) in each kubeconfig file, so I pretty much never use the kubectl config family of commands. This does mean that, however you set up the kubeconfig file initially, you either need to repeat those steps for each cluster or split out your existing kubeconfig file by hand (it's YAML so it's fairly doable).

# specifically for Amazon Elastic Kubernetes Service
kubecfg dc1-main
aws eks update-kubeconfig --name dc1-main ...
kubecfg dc2-main
aws eks update-kubeconfig --name dc2-main ...

Tools that want to write the configuration also use this variable, which for me mostly comes up if I want to recreate my minikube environment. You may find it useful to chmod 0400 "$KUBECONFIG" to protect these files once you've created them.

CodePudding user response：

Thought of adding my quick workaround as an answer.

Creating two aliases as below resolved the problem partially.

alias kdc1='kubectl --context="dc1-main"'
alias kdc2='kubectl --context="dc2-main"'

Which gives me a quick two commands to access two contexts.

But the problem is if I use commands like helm.

CodePudding user response：

Not the finest solution but one of the fastest for me was to create VM or remote instance (like free m2.micro AWS) and act on it via SSH\VM UI having multiple configurations.

However you can take a look to this THIS INSTRUCTION, where you can export various kubeconfigs in different shell sessions.