Cloudera Docs

Enabling the Kafka service operations

Describes the steps that configure the Apache Kafka service operations for your Cloudera Observability On-Premises environment. The properties are enabled and added in Cloudera Manager.

Steps that enable the Kafka service operations for the Cloudera Observability On-Premises environment by safely adding Kafka properties in Cloudera Manager.
  • Cloudera Observability On-Premises requires at least three Apache Kafka Broker hosts. Verify that you have the hostnames for these instances.
  • Ensure that you have added Apache Kafka service to the cluster in Cloudera Manager.
  1. In a supported web browser on the Cloudera Observability On-Premises cluster, log in to Cloudera Manager.
  2. In Cloudera Manager, select Clusters, KAFKA, and then click the Configuration tab.
  3. Set the Java heap size, partitions, and retention values and record the TLS/SSL port number by doing the following:
    1. Search for the Java Heap Size of Broker property and in the Kafka Broker Default Group field, set the value to a minimum of 4 GB. Cloudera recommends a setting of 8 GB.
    2. Search for the Default Number of Partitions property and in the Kafka (Service-Wide) field, set the value to 48.
    3. Increase the Kafka default retention times by doing the following:
      • Search for the Offset Retention Time property and in the Kafka (Service-Wide) field, increase the number of days to 14.
      • Search for the Data Retention Time property and in the Kafka Broker Default Group field, increase the number of days to 14.
      • Search for the Data Retention Check Interval property and in the Kafka Broker Default Group field, increase the time to 1 hr.
    4. Search for the TLS/SSL Port property and do one of the following:
      • If the Kafka Broker Default Group field contains a value, record and store the port number somewhere safe as you will be required to supply this value when you set the OBSERVABILITY service properties.
      • If the Kafka Broker Default Group field is empty, search for the TCP Port property and from the Kafka Broker Default Group field, record and store its port number somewhere safe as you will be required to supply this value when you set the OBSERVABILITY service properties.
    5. Click Save Changes.
  4. Restart the KAFKA services, by doing the following:
    1. Select the Status tab and then from the Actions menu, select Deploy Client Configuration.
    2. In the Deploy Client Configuration message, confirm deployment by clicking Deploy Client Configuration.
    3. Monitor the progress of the client's configuration deployment until you see the successfully deployed client configuration message.
    4. Click Close.
  5. Optional: To restart stale services do the following:
    1. Back in the Cloudera Manager Home page, select the Status tab.
    2. Locate a Stale Configuration: Restart needed indicator. As shown in the image below:


    3. Restart the service by clicking the service's vertical ellipses icon, selecting Restart, and in the Stale Configurations page, clicking Restart Stale Services.
    4. In the Restart Stale Services page, select the Re-deploy client configuration check box and click Restart Now.
    5. Monitor the restart progress until the All requested services successfully restarted message appears and then click Finish.