Performance & Scalability

The Kafka and Cloudera SQL Stream Builder integration enables you to use the Kafka-specific syntax to customize your SQL queries based on your deployment and use case.

You can achieve high performance and scalability with Cloudera SQL Stream Builder, but the proper configuration and design of the source Kafka topic is critical. Cloudera SQL Stream Builder can read a maximum of one thread per Kafka partition. You can achieve the highest performance configuration when setting the Cloudera SQL Stream Builder threads equal to or higher than the number of Kafka partitions.

If the number of partitions is less than the number of Cloudera SQL Stream Builder threads, then Cloudera SQL Stream Builder has idle threads and messages show up in the logs indicating as such. For more information about Kafka partitioning, see the Kafka partitions documentation.