Learn how you can set up Kafka KRaft in CDP Public Cloud.
Kafka KRaft in CDP is implemented in the form of a Kafka service role. The role is called KRaft Controller. KRaft mode can be set up for a Kafka service by provisioning KRaft Controller service roles for the Kafka service and configuring various Kafka service properties. This can be done when installing a new cluster or when adding the Kafka service to an existing cluster.
If you are installing a new cluster, you complete the following steps during Step 8: Set Up a Cluster Using the Wizard of the CDP Private Cloud Base Installation Guide. Specifically, the KRaft Controller roles are added on the Assign Roles page, the properties are configured on the Review Changes page of the Add Cluster - Configuration wizard.
If you are adding a new Kafka service to an existing cluster, you complete the following steps while using the Add a Service wizard. KRaft Controller roles are added on the Assign Roles page, properties are configured on the Review Changes page of the wizard.
- KRaft requires an odd number of controllers to function. As a result, ensure that you always deploy an odd number of KRaft Controller service roles. Service setup fails if you try to deploy an even number of roles.
- KRaft can function with a single KRaft Controller role instance, but a minimum of three is recommended. Deploying a Kafka service with a single KRaft Controller is only recommended for development and testing purposes.
- Cloudera recommends that you deploy the KRaft Controller service roles on dedicated hosts.
- If deployment on dedicated hosts is not feasible, or if you are deploying a lightweight cluster where high availability is not a requirement, you can colocate the controllers on the same hosts as the brokers. In general, if your deployment can tolerate the simultaneous failure of two colocated nodes, then deploying the controllers and brokers on the same hosts is a viable option.
- The hardware and deployment recommendations that exist for ZooKeeper also hold true for KRaft Controllers. For more information, see Performance considerations.
- Deploy an odd number of KRaft Controller service roles on your hosts.
Clear the Enable Delegation Tokens Kafka property.
Delegation token based authentication is not supported with KRaft and must be disabled.
- Set the Kafka Metadata Store Service Kafka property to KRaft.