I am trying to produce message using kafka-console-producer from apache-kafka binary and consume from consumer setup in spring boot. Consumer uses avro schema.

When message is produced in json format, my consumer is throwing exception - “not able to serialize”.

I found a solution for this to use “Confluent Platform 7.1”, which has kafka-avro-console-producer. It supports avro but it is an enterprise edition.

Is there a way to produce/consume messages with avro schema using apache-kafka itself with kafka-console-producer?

CodePudding user response：

kafka-console-producer only accepts UTF8 strings, by default. Internally, it defaults to use StringSerializer

The kafka-avro-console-consumer wraps the other, and it's source-available, not Enterprise. You'd need to download at least the Confluent kafka-avro-serializer JAR file(s) and dependencies to even produce Avro data with it, but this also will require you to use the Schema Registry.

If you simply want to produce Avro binary data with no Registry, you can use Avro BinaryEncoder class, however, this will require you to also need your own deserializer in any consumer, rather than using ones provided by Confluent (again, free, not Enterprise)