Managing Multiple Kafka Versions

Previously, Cloudera distributed Kafka as a parcel (CDK) separate from CDH. Installation of the separate Kafka required Cloudera Manager 5.4 or higher. For a list of available parcels and packages, see CDK Powered By Apache Kafka® Version and Packaging Information Kafka bundled along with CDH.

As of CDH 6, Cloudera distributes Kafka bundled along with CDH. Kafka is in the parcel itself. Installation requires Cloudera Manager 6.0 or higher. For installation instructions for Kafka using Cloudera Manager, see Cloudera Installation Guide.

Cloudera recommends that you deploy Kafka on dedicated hosts that are not used for other cluster roles.

To ensure there is no downtime during an upgrade, these instructions describe performing a rolling upgrade.

Before upgrading, ensure that you set inter.broker.protocol.version and log.message.format.version to the current Kafka version (see Kafka and CM/CDH Supported Versions Matrix), and then unset them after the upgrade. This is a good practice because the newer broker versions can write log entries that the older brokers cannot read. If you need to rollback to the older version, and you have not set inter.broker.protocol.version and log.message.format.version, data loss can occur.

From Cloudera Manager on the cluster to upgrade:

1. Explicitly set the Kafka protocol version to match what's being used currently among the brokers and clients.

   Update server.properties on all brokers as follows:

   1. Choose the Kafka service.
   2. Click Configuration.
   3. Use the Search field to find the Kafka Broker Advanced Configuration Snippet (Safety Valve) configuration property.
   4. Add the following properties to the safety valve:

      inter.broker.protocol.version = current_Kafka_version
      log.message.format.version = current_Kafka_version

      Make sure you enter full Kafka version numbers with three values, such as 0.10.0. Otherwise, you'll see an error similar to the following:

      2018-06-14 14:25:47,818 FATAL kafka.Kafka$:
      java.lang.IllegalArgumentException: Version `0.10` is not a valid version
              at kafka.api.ApiVersion$$anonfun$apply$1.apply(ApiVersion.scala:72)
              at kafka.api.ApiVersion$$anonfun$apply$1.apply(ApiVersion.scala:72)
              at scala.collection.MapLike$class.getOrElse(MapLike.scala:128)
                      

   5. Save your changes. The information is automatically copied to each broker.
2. Upgrade CDH. See Upgrading the CDH Cluster.

   Do not restart the Kafka service, select Activate Only and click OK.

3. Perform a rolling restart.

   Select Rolling Restart or Restart based on the downtime that you can afford.

At this point, the brokers are running in compatibility mode with older clients. It can run in this mode indefinitely. If you do upgrade clients, after all clients are upgraded, remove the Safety Valve properties and restart the cluster.

The table below links to the upstream Apache Kafka documentation for detailed upgrade instructions. It is recommended that you read the instructions for your specific upgrade to identify any additional steps that apply to your specific upgrade path.