I have a service with the following architecture.

1. It consumes messages from one Kafka topic.
2. It reads records from

   We're using Locust for load testing. The question is how to test Enricher for the maximum load? I could send messages to Kafka topic. But as far as I understood, I would test Kafka itself but not the Enricher. Because the service cannot consume more messages that it is able to proceed. By the way, Aerospike interaction is also important. I have to be sure that the amount of reads won't break the cluster.

   So, what are the common practices to test Kafka consumer in such scenario?

   CodePudding user response：

   But as far as I understood, I would test Kafka itself but not the Enricher.

   Why do you say that? If you put a bunch of messages on the queue, and the purpose of your application is to process those messages, then that sounds like an entirely reasonable approach.

   Such a test is of course dependant on Kafka being fast (faster than your application), but that is most likely the case.

   Specifically, have a look at https://github.com/SvenskaSpel/locust-plugins/blob/master/examples/kafka_ex.py

   But you’ll want to measure the processing time somewhere else (either in your app, or by comparing the time a message was placed on the input topic to when it arrived in the output topic)

   Like with most queue/batch-oriented applications your performance is probably best expressed in transactions/s or the amount of time it takes to process X number of messages (latency may also be important but usually less so)

   CodePudding user response：

   Possible using Torque as is suggest in Finding Order in Chaos: How We Automated Performance Testing with Torque.