GCS read write access from the GKE Pod without credentials/auth-CodePudding

We created the GKE cluster with the public endpoint. The service account of the GKE cluster and Node pool has the following roles.

"roles/compute.admin",
"roles/compute.viewer",
"roles/compute.securityAdmin",
"roles/iam.serviceAccountUser",
"roles/iam.serviceAccountAdmin",
"roles/resourcemanager.projectIamAdmin",
"roles/container.admin",
"roles/artifactregistry.admin",
"roles/storage.admin"

The node pool of the GKE cluster has the following OAuth scopes

"https://www.googleapis.com/auth/cloud-platform",
"https://www.googleapis.com/auth/devstorage.read_write",

The private GCS bucket has the same service account as the principal, with the storage admin role.

When we try to read/write in this bucket from a GKE POD, we get the below error.

# Read
AccessDeniedException: 403 Caller does not have storage.objects.list access to the Google Cloud Storage bucket

# Write
AccessDeniedException: 403 Caller does not have storage.objects.create access to the Google Cloud Storage object

We also checked this

NodePool Security:

CodePudding user response：

Seems like you are trying to use the Node Service Account to authenticate to GCS. You need to passe the Service Account Key to the app you are calling the API from as described in this doc

If you want Keyless authentication, my advice is to use Workload identity

CodePudding user response：

This is for all who are looking for an answer to implement this solution via terraform. Please refer below:

Create a Kubernetes service account with annotation

resource "kubernetes_manifest" "service_account" {
  manifest = {
    "apiVersion" = "v1"
    "kind"       = "ServiceAccount"
    "metadata" = {
      "name"      = "KSA_NAME"
      "namespace" = "NAME"
      "annotations" = {
        "iam.gke.io/gcp-service-account" = "GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com"
      }
    }
    "automountServiceAccountToken" = true
  }
}

Create an IAM policy binding to allow the Kubernetes service account to act as the IAM service account.

resource "google_service_account_iam_binding" "service-account-iam" {
  service_account_id = "projects/PROJECT_ID/serviceAccounts/GSA_NAME@GSA_PROJECT.iam.gserviceaccount.com"
  role               = "roles/iam.workloadIdentityUser"
  members            = [
    "serviceAccount:${var.project_id}.svc.id.goog[NAMESPACE/KSA-NAME]",
  ]
}

Add service account to the deployment/statefulset manifest like this

spec:
  serviceAccount: KSA-NAME
  containers:

NOTE:

GSA_PROJECT and PROJECT_ID are the same if you are using the same project for all objects.
K8s service account is created via resource "kubernetes_manifest" method because there is an open issue for it to create via resource "kubernetes_service_account" method where it is throwing an error stating. Waiting for default secret of "NAMESPACE/KSA_NAME" to appear