Kafka Public APIs

Continue reading:

What is a Public API

The following parts of Apache Kafka in CDH are considered as public APIs:

  • Kafka wire protocol format: the format itself might change, but brokers will be able to use the old format as long as documentation and upgrade instructions are followed properly.
  • Binary log format: the format itself might change, but brokers will be able to use the old format as long as documentation and upgrade instructions are followed properly.
  • Interfaces and classes in the following packages:
    • org/apache/kafka/common/serialization
    • org/apache/kafka/common/errors
    • org/apache/kafka/clients/producer
    • org/apache/kafka/clients/consumer
  • Command-line admin tools: arguments, except ZooKeeper related options, that are subject to change and/or removal.
  • HttpMetricsReporter: existing fields will stay backward compatible, but new fields may be introduced. The only public API of HttpMetricsReporter is the /api/metrics REST endpoint. For a list of supported metrics, see Kafka Metrics.
  • Properties, excluding their default values
  • Config file content and format, and the effect of configuration attributes
  • Endpoints

What is NOT a public API

There are structures that third parties might regard as an interface but Cloudera Kafka distributions do not consider them public APIs. In general, any API that is not listed as public in the What is a Public API section should be considered private, and client code should not rely on behavior/data content or format. Some examples are:

  • Data structures in ZooKeeper: the content and format what Kafka stores in ZooKeeper are internal implementation details.
  • Authorizer interface: the only supported authorizer in CHD is the Sentry one.
  • AdminClient: it is a new and rapidly evolving part of Kafka, so Cloudera can’t provide the same guarantees as for other interfaces.
  • Interfaces marked with the @Evolving or @Unstable annotations in the Kafka source code
  • Index files generated by Kafka
  • Application log file content and format (for example what Log4J/SLF4J/… produces)
  • Any classes used for testing
  • Relying on transitive dependencies: any dependency pulled in by Kafka
  • Any other interfaces not listed above
  • Anything that Cloudera does not support, even if it fits the definition of a public API