I have used the Kafka cluster (with replication) click to deploy container from Google on kubernetes. How do I expose the brokers so I can consume from an external consumer? I'm very new to kunernetes.

I have tried exposing the broker nodes with a load balancer but the external ip given

I opened the port with a firewall rule

But when connecting from my consumer it throws an error about being disconnected

Any help would be great, I can provide more info if asked.

CodePudding user response：

You cannot use a load balancer. Kafka clients must talk directly to the brokers.

Firewall is a good step, but you need to ensure each of the brokers are exposed properly via advertised.listeners within and outside of the VPC. Refer blog post.

Alternative, within GKE, you can run Strimzi Operator, which handles Kubernetes resources for you with regard to the Kafka cluster.