I am creating a Spring Boot application with Kafka support. I have created a Producer and a Consumer and everything is working like a charm. In runtime I have one consumer. I wanted to have many consumers, the same number of partitions of my topic.
How to create many consumers?
I have been searching in documentation but with no success.
Thank you !
You need to create a ConcurrentKafkaListenerContainerFactory
and set the concurrency parameter. It creates 1 or more KafkaMessageListenerContainers
based on concurrency. If the ContainerProperties
is configured with TopicPartitions, the TopicPartitions are distributed evenly across the instances.
For example
ConcurrentKafkaListenerContainerFactory<String, String> factory = new ConcurrentKafkaListenerContainerFactory<>();
factory.setConcurrency(12);
If you are using
@KafkaListener
, it has had aconcurrency
property (since 2.2) which overrides the container factory's default setting.In this case, the clientId is generated dynamically for each listener ?