For transactions that have been started by a listener container, we need to set the same transaction id prefix for all app instances. For producing-only transaction, we need to set a different value for each instance instead.

I'm using Spring Cloud Stream Kafka Binder in the app which has both types of transactions, and the property spring.cloud.stream.kafka.binder.transaction.transaction-id-prefix is used for creating a common transaction manager.

I wonder how to make it all work properly because it seems that you can't have it both ways at the same time.

CodePudding user response：

That requirement is no longer necessary if your broker is 2.5 or later and you are using a currently supported version of spring-cloud-stream (3.2.x) and spring-kafka (2.8.x).

You must now use a unique prefix even for consumer-initiated transactions.

If you are using an older broker or an unsupported spring-kafka version that does not use EOSMode V2 (BETA) by default, you can configure two binders.

https://docs.spring.io/spring-kafka/docs/current/reference/html/#transaction-id-prefix

This problem (different rules for transactional.id) has been eliminated when EOSMode.BETA is being used (with broker versions >= 2.5); see Exactly Once Semantics.