Pod assigned node role instead of service account role on AWS EKS-CodePudding

Before I get started I have seen questions this and this, and they did not help.

I have a k8s cluster on AWS EKS on which I am deploying a custom k8s controller for my application. Using instructions from eksworkshop.com, I created my service account with the appropriate IAM role using eksctl. I assign the role in my deployment.yaml as seen below. I also set the securityContext as that seemed to solve problem in another case as described here.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: tel-controller
  namespace: tel
spec:
  replicas: 2
  selector:
    matchLabels:
      app: tel-controller
  strategy:
    rollingUpdate:
      maxSurge: 50%
      maxUnavailable: 50%
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: tel-controller
    spec:
      serviceAccountName: tel-controller-serviceaccount
      securityContext:
        fsGroup: 65534
      containers:
      - image: <image name>
        imagePullPolicy: Always
        name: tel-controller
        args:
        - --metrics-bind-address=:8080
        - --health-probe-bind-address=:8081
        - --leader-elect=true
        ports:
          - name: webhook-server
            containerPort: 9443
            protocol: TCP
          - name: metrics-port
            containerPort: 8080
            protocol: TCP
          - name: health-port
            containerPort: 8081
            protocol: TCP
        securityContext:
          readOnlyRootFilesystem: true
          runAsNonRoot: true
          allowPrivilegeEscalation: false

But this does not seem to be working. If I describe the pod, I see the correct role.

AWS_DEFAULT_REGION:           us-east-1
AWS_REGION:                   us-east-1
AWS_ROLE_ARN:                 arn:aws:iam::xxxxxxxxx:role/eksctl-eks-tel-addon-iamserviceaccount-tel-t-Role1-3APV5KCV33U8
AWS_WEB_IDENTITY_TOKEN_FILE:  /var/run/secrets/eks.amazonaws.com/serviceaccount/token
Mounts:
  /var/run/secrets/eks.amazonaws.com/serviceaccount from aws-iam-token (ro)
  /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-6ngsr (ro)

But if I do a sts.GetCallerIdentityInput() from inside the controller application, I see the node role. And obviously i get an access denied error.

caller identity: (go string) { Account: "xxxxxxxxxxxx", Arn: "arn:aws:sts::xxxxxxxxxxx:assumed-role/eksctl-eks-tel-nodegroup-voice-NodeInstanceRole-BJNYF5YC2CE3/i-0694a2766c5d70901", UserId: "AROAZUYK7F2GRLKRGGNXZ:i-0694a2766c5d70901" }

This is how I created by service account

eksctl create iamserviceaccount --cluster ${EKS_CLUSTER_NAME} \
--namespace tel \
--name tel-controller-serviceaccount \
--attach-policy-arn arn:aws:iam::xxxxxxxxxx:policy/telcontrollerRoute53Policy \
--override-existing-serviceaccounts --approve

I have done this successfully in the past. The difference this time is that I also have role & role bindings attached to this service account. My rbac.yaml for this SA.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: tel-controller-role
  labels:
    app: tel-controller
rules:
- apiGroups: [""]
  resources: [events]
  verbs: [create, delete, get, list, update, watch]
- apiGroups: ["networking.k8s.io"]
  resources: [ingressclasses]
  verbs: [get, list]
- apiGroups: ["", "networking.k8s.io"]
  resources: [services, ingresses]
  verbs: [create, get, list, patch, update, delete, watch]
- apiGroups: [""]
  resources: [configmaps]
  verbs: [create, delete, get, update]
- apiGroups: ["coordination.k8s.io"]
  resources: ["leases"]
  verbs: [get, create, update]
- apiGroups: [""]
  resources: [pods]
  verbs: [get, list, watch, update]
- apiGroups: ["", "networking.k8s.io"]
  resources: [services/status, ingresses/status]
  verbs: [update, patch]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tel-controller-rolebinding
  labels:
    app: tel-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: tel-controller-role
subjects:
- kind: ServiceAccount
  name: tel-controller-serviceaccount
  namespace: tel

What am I doing wrong here? Thanks.

PS: I am deploying using kubectl

PPS: from go.mod I am using github.com/aws/aws-sdk-go v1.44.28

CodePudding user response：

So the Go SDK has 2 methods to create a session. Apparently I was using the deprecated method to create my session (session.New). Using the recommended method to create the new session, session.NewSession solved my problem.