K3s Allow Unauthenticated Access to OIDC Endpoints-CodePudding

Goal

I want to allow unauthenticated access to the following OIDC endpoints in my K3s cluster (from other pods inside the cluster mostly but access from outside is also acceptable):

Problem

By default, Kubernetes requires an authorization token for accessing those endpoints and despite my efforts to enable unauthenticated access, I cannot seem to get the unauthenticated access to work.

What I have tried

According to the Kubernetes documentation on Service account issuer discovery, one must create a ClusterRoleBinding that maps the ClusterRole system:service-account-issuer-discovery to the Group system:unauthenticated.

I also found this helpful example for a different use-case but they're exposing the OIDC endpoints anonymously just like I want to do: OIDC issuer discovery for Kubernetes service accounts

Based on both of those, I created my ClusterRoleBinding with:

kubectl create clusterrolebinding service-account-issuer-discovery-unauthenticated --clusterrole=system:service-account-issuer-discovery --group=system:unauthenticated

This results in the following spec:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  creationTimestamp: "2022-11-28T15:24:41Z"
  name: service-account-issuer-discovery-unauthenticated
  resourceVersion: "92377634"
  uid: 75402324-a8cf-412f-923e-a7a87ed082c2
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:service-account-issuer-discovery
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:unauthenticated

I also confirmed RBAC is enabled by running this which showed an entry:

kubectl api-versions | grep 'rbac.authorization.k8s.io/v1'

Unfortunately, despite creating that ClusterRoleBinding, it still seems to be requiring an authorization token to access the OIDC endpoints. For example, when I call from outside the cluster:

curl -vk https://my-cluster-hostname:6443/.well-known/openid-configuration

Output:

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {

  },
  "status": "Failure",
  "message": "Unauthorized",
  "reason": "Unauthorized",
  "code": 401
}

Accessing from the Ingress controller inside the pod (by execing into it), I get the same error.

curl -vk https://kubernetes.default.svc/.well-known/openid-configuration

I am using K3s for this cluster... Is there something special about K3s that I have not accounted for? Do I need to do something to get this ClusterRoleBinding to take effect? Something else I missed?

CodePudding user response：

The issue was that the --anonymous-auth api server setting is set to false by default.

I was able to adjust this with my already-installed K3s server nodes by editing the systemd unit for the k3s service. Step-by-step guide:

Edit the systemd unit with:

sudo vim /etc/systemd/system/k3s.service

You'll see the K3s unit:

[Unit]
Description=Lightweight Kubernetes
Documentation=https://k3s.io
Wants=network-online.target
After=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
Type=notify
EnvironmentFile=-/etc/default/%N
EnvironmentFile=-/etc/sysconfig/%N
EnvironmentFile=-/etc/systemd/system/k3s.service.env
KillMode=process
Delegate=yes
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
TimeoutStartSec=0
Restart=always
RestartSec=5s
ExecStartPre=/bin/sh -xc '! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service'
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s \
    server \
        '--cluster-init' \
        '--disable=traefik' \
        '--write-kubeconfig-mode' \
        '644' \

Add the last two lines to the ExecStart:

ExecStart=/usr/local/bin/k3s \
    server \
        '--cluster-init' \
        '--disable=traefik' \
        '--write-kubeconfig-mode' \
        '644' \
        '--kube-apiserver-arg' \
        '--anonymous-auth=true' \

Reload the systemd unit:

sudo systemctl daemon-reload

Finally, restart the k3s service:

sudo systemctl restart k3s

References:

Control Plane Execution and Arguments - You can see the default for K3s there
API discovery roles - also mentions this flag at the top