Cloudera Docs

Client and broker compatibility across Kafka versions

An overview on client and broker version compatibility.

Maintaining compatibility across different Kafka clients and brokers is a common issue. Mismatches among client and broker versions can occur as part of any of the following scenarios:

  • Upgrading your Kafka cluster without upgrading your Kafka clients.
  • Using a third-party application that produces to or consumes from your Kafka cluster.
  • Having a client program communicating with two or more Kafka clusters (such as consuming from one cluster and producing to a different cluster).
  • Using Spark as a Kafka consumer.
In these cases, it is important to understand client/broker compatibility across Kafka versions. Here are general rules that apply:
  • Changes in either the major part or the minor part of the Kafka version major.minor determines whether the client and broker are fully compatible. If there are differences, some features might not be available.
  • Differences among the maintenance versions are not considered when determining compatibility.
  • All clients and brokers later than version 1.0.0 are considered compatible. Additionally, all client and broker versions shipped with Cloudera Runtime are also considered compatible. If you are using an older client version with a newer broker version, some limitations on the available features may apply, but the client and broker will be able to communicate with each other.