My application produces data to Kafka and uses Zookeeper to do leader election. Now that Kafka won't be needing Zookeeper anymore, I need to move away from it too, because I don't want to keep it in my infrastructure just for that. So, does Kafka provide an API to achieve leader election? Or I would have to do it manually, creating a topic and coming up with my own logic? My application have many instances but only one of them should produce to Kafka at the same time, so I need a kind of distributed lock and I was hoping to do it in an easy way using Kafka itself or k8s, avoiding have to add another piece to my infrastructure.
CodePudding user response:
does Kafka provide an API to achieve leader election
It will use a Raft protocol internal to the brokers, yes,
I doubt it is intended to be multi-purpose for external usage, like Zookeeper.
distributed lock ... k8s