Cloudera Manager requires Kafka client configuration files to perform rolling restart
checks. These files are generated automatically. However, if your Kafka service has custom
listeners configured, you must manually update these client configuration files. Otherwise, the
rolling restart check might fail.
When Cloudera Manager executes a rolling restart check, it uses the
kafka-topics
tool to gather information about the brokers, topics, and
partitions. The kafka-topics
tool requires a valid client configuration
file to run. Cloudera Manager automatically generates two configuration files for this
purpose. One is used for the kafka-topics
commands initiated before the
brokers are stopped, the other, after brokers are restarted.
If your Kafka service is configured to use custom listeners, you must manually update the
configuration files generated by Cloudera Manager. This is required because Cloudera Manager
might not be able to automatically extract the information required to establish a
connection with the Kafka service when custom listeners are configured. The client
configuration files can be updated using advanced security snippets.
- In Cloudera Manager, select the Kafka service.
- Go to Configuration.
- Manually update the client configuration files used during rolling restart
checks.
This can be done by adding a valid client configuration to the following advanced
configuration snippets:
- Kafka Broker Advanced Configuration Snippet (Safety Valve) for
rolling_restart_check_before_stop_admin_client_configs.properties
- Kafka Broker Advanced Configuration Snippet (Safety Valve) for
rolling_restart_check_after_start_admin_client_configs.properties
Ensure that you add the same client configuration to both snippets. The client
configuration you add must contain all properties that are required to establish a
connection with the brokers. The client configuration you add here is similar to any
other client configuration you create for Kafka command line tools. However, this
specific configuration accepts the bootstrap.servers
property. Use this
property to specify your custom host:port pairs that you use as your custom
listeners.
The following client configuration example is for a Kafka service that
has both TLS/SSL and Kerberos enabled. You can use this example as a template and make
changes as needed. For more client configuration examples, see the Securing Apache Kafka
publication in the
Streams Messaging
documentation.
bootstrap.servers=[***HOST***]:[***PORT***]
security.protocol=SASL_SSL
ssl.client.auth=none
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka
sasl.jaas.config=com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true storeKey=true keyTab="[***PATH TO KEYTAB***]" principal="[***KERBEROS PRINCIPAL***]";
ssl.keystore.location=[***PATH TO KEYSTORE.JKS***]
ssl.key.password=[***PASSWORD***]
ssl.keystore.password=[***PASSWORD***]
ssl.keystore.type=jks
ssl.truststore.location=[***PATH TO TRUSTSTORE.JKS***]
ssl.truststore.type=jks
ssl.truststore.password=[***PASSWORD***]
- Click Save Changes.
The client configuration files used by Cloudera Manager during rolling restart
checks are configured.
Enable and configure rolling restart checks. Complete Configuring rolling restart
checks.