Home > database >  Using Multiple Kafka Listeners to handle specific data types on a shared topic
Using Multiple Kafka Listeners to handle specific data types on a shared topic

Time:10-12

I have two kafka listeners A & B that are each in separate services listening to the same topic. Listener A only cares about data types 1, 2 & 3 while Listener B only cares about data type 3. I was considering using individual @KafkaHandler methods to handle the different data types. However, my concern is about what will happen if listener B receives data types 1 or 2? Is an error thrown ? Does it safely ignore.

Also wanted to double confirm but, messages of the same type like data type 3 should be consumed by both listeners so long as the consumer group-id is different.

@KafkaListener
public class listenerA {
    @KafkaHandler
    public void dataType1(Object1 data) { ... }
    @KafkaHandler
    public void dataType2(Object2 data) { ... }
    @KafkaHandler
    public void dataType3(Object3 data) { ... }
}
@KafkaListener
public class listenerB {
    @KafkaHandler
    public void dataType3(Object3 data) { ... }
}

CodePudding user response:

You can add a RecordFilterStragegy in the second listener container to discard unwanted types.

See https://docs.spring.io/spring-kafka/docs/current/reference/html/#filtering-messages

  • Related