Tune Enrichment Kafka Partitions

The first enrichment variable that should be determined is the minimum number of Kafka partitions required.

Use the following formula to determine the minimum number of Kafka partitions for the enrichment toplogy

Num Partitions=Max( t/p,t/c)

where

t is the desired throughput
p is the maximum throughout using a single producer
c is the maximum throughput using a single consumer

	Note
You can estimate the minimum number of partitions requres based on the original calculations performed for parser topologies. However, this may or may not be suitable as the message size increases as it progresses through Metron topologies. This increased message size can affect throughput. For this reason, we recommend that you perform the same steps to calculate the value for the enrichment topic.

Note

You can estimate the minimum number of partitions requres based on the original calculations performed for parser topologies. However, this may or may not be suitable as the message size increases as it progresses through Metron topologies. This increased message size can affect throughput. For this reason, we recommend that you perform the same steps to calculate the value for the enrichment topic.