I am trying to understand the similarities and distinctions between RetryTemplate and @RetryableTopic.

RetryTemplate is defined inside KafkaListenerContainerFactory itself and @RetryableTopic annotation is added together with @KafkaListener. Is this more of a stylistic difference or do they have two different purposes?

CodePudding user response：

RetryTemplate in the container factory is deprecated because the error handlers now support backoff and exception classification.

The difference between normal retries (template or error handlers) is that a retried record blocks consumption of other records in the partition until the record is successfully processed or discarded. This is important if strict ordering is required.

@RetryableTopic on the other hand is non-blocking retry in that the failed record is moved to a different topic each time it fails. This allows processing of subsequent records during retries. Useful if strict ordering is not important.

See Non-Blocking Retries https://docs.spring.io/spring-kafka/docs/2.8.4-SNAPSHOT/reference/html/#retry-topic

CodePudding user response：

Those are different approaches to retrying failed processing records.

The RetryTemplate approach has been deprecated - it relies on retrying the record in memory, which could trigger partition rebalancing if the total time of the retrials is greater than the session.timeout.ms property.

You can use a DefaultErrorHandler to achieve blocking retrials - if record processing fails, the framework will seek to that offset and receive the record again - this reduces the chance of a rebalance being triggered while waiting for the retrials compared to the previous method. No other records will be processed until this record either succeeds or exhausts the retrials, which is important if you need to keep record ordering.

The @RetryableTopic approach uses one or more topics to achieve non-blocking retrial, meaning if one record fails, it'll be forwarded to another topic for the retrials, and the main topic will continue processing the remaining records. In this approach, you lose ordering guarantees in exchange for not having failed record processing retrials slow down other records' processing.