I am new to Kafka and testing it with Java.

As far I see the messages I produce/consume with Kafka are strings.

Is there some possiblity to do some easy binding to structured data (e.g. a list or map) or is the best way to do so in serializing/deserializing it manually (e.g. in using JSON)?

CodePudding user response：

As far as I see

Unclear what examples you are finding, but Kafka stores bytes. StringSerializer is only one of many Serializer interfaces available and takes a (UTF-8) string to bytes.

do some easy binding to structured data (e.g. a list or map)

Confluent and Apicurio, for example, provide Avro and/or Protobuf serializers for binary serialization of Java objects that integrate with their Schema Registry products.

serializing/deserializing it manually (e.g. in using JSON)?

There is already a built-in JsonSerializer based on Jackson for Kafka clients to use or build into a Serde in Streams. (requires a dependency on connect-api, not just kafka-clients).

Spring-Kafka also has its own JsonSerializer class.

CodePudding user response：

I think what you're looking for is the Streams API: https://kafka.apache.org/documentation/streams/

There are some great explanations if you're new to Kafka, which gives a nice introduction to the whole domain.

I suspect you will eventually end up with at the "Data Type and Serialization" section here: https://kafka.apache.org/31/documentation/streams/developer-guide/datatypes.html, so when you feel comfortable with the topic go look at that.