Tune Batch Indexing Kafka Partitions
The first batch indexing variable that should be determined is the minimum number of Kafka partitions required.
Use the following formula to determine the minimum number of Kafka partitions for
the enrichment topology
Num Partitions=Max( t/p,t/c)
where
t
is the desired throughputp
is the maximum throughout using a single producerc
is the maximum throughput using a single consumer