CDH 6 includes Apache Kafka as part of the core package. The documentation includes improved contents for how to set up, install, and administer your Kafka ecosystem. For more information, see the Cloudera Enterprise 6.0.x Apache Kafka Guide. We look forward to your feedback on both the existing and new documentation.

Issues Fixed in CDK 3.1.0 Powered By Apache Kafka

CDK 3.1.0 Powered By Apache Kafka fixes the following issues:

Authenticated Kafka clients may impersonate other users

Authenticated Kafka clients may impersonate any other user via a manually crafted protocol message with SASL/PLAIN or SASL/SCRAM authentication when using the built-in PLAIN or SCRAM server implementations in Apache Kafka.

Note that the SASL authentication mechanisms that apply to this issue are neither recommended nor supported by Cloudera. In Cloudera Manager (CM) there are four choices: PLAINTEXT, SSL, SASL_PLAINTEXT, and SASL_SSL. The SASL/PLAIN option described in this issue is not the same as SASL_PLAINTEXT option in CM. That option uses Kerberos and is not affected. As a result it is highly unlikely that Kafka is susceptible to this issue when managed by CM unless the authentication protocol is overridden by an Advanced Configuration Snippet (Safety Valve).

Products affected: CDK Powered by Apache Kafka

Releases affected: CDK 2.1.0 to 2.2.0, CDK 3.0

Users affected: All users

Detected by: Rajini Sivaram (rsivaram@apache.org)

Severity (Low/Medium/High):8.3 (High) (CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:H/A:H)

Impact:Privilege escalation.

CVE:CVE-2017-12610

Immediate action required: Upgrade to a newer version of CDK Powered by Apache Kafka where the issue has been fixed.

Addressed in release/refresh/patch: CDK 3.1, CDH 6.0 and higher

Knowledge article: For the latest update on this issue see the corresponding Knowledge article: TSB 2018-332: Two Kafka Security Vulnerabilities: Authenticated Kafka clients may impersonate other users and and may interfere with data replication

Authenticated clients may interfere with data replication

Authenticated Kafka users may perform an action reserved for the Broker via a manually created fetch request interfering with data replication, resulting in data loss.

Products affected: CDK Powered by Apache Kafka

Releases affected: CDK 2.0.0 to 2.2.0, CDK 3.0.0

Users affected: All users

Detected by: Rajini Sivaram (rsivaram@apache.org)

Severity (Low/Medium/High):6.3 (Medium) (CVSS:3.0/AV:N/AC:L/PR:L/UI:N/S:U/C:L/I:L/A:L)

Impact:Potential data loss due to improper replication.

CVE:CVE-2018-1288

Immediate action required: Upgrade to a newer version of CDK Powered by Apache Kafka where the issue has been fixed.

Addressed in release/refresh/patch: CDK 3.1, CDH 6.0 and higher

Knowledge article: For the latest update on this issue see the corresponding Knowledge article: TSB 2018-332: Two Kafka Security Vulnerabilities: Authenticated Kafka clients may impersonate other users and and may interfere with data replication

Upstream Issues Fixed

The following upstream issues are fixed in CDK 3.1.0 Powered By Apache Kafka:

  • KAFKA-6739: Down-conversion fails for records with headers.
  • KAFKA-6185: Selector memory leak with high likelihood of OOM in case of down conversion.
  • KAFKA-6134: High memory usage on controller during partition reassignment.
  • KAFKA-6119: Silent Data Loss in Kafka011 Transactional Producer.
  • KAFKA-6116: Major performance issue due to excessive logging during leader election.
  • KAFKA-6093: Replica dir not deleted after topic deletion.
  • KAFKA-6042: Kafka Request Handler deadlocks and brings down the cluster..
  • KAFKA-6026: KafkaFuture timeout fails to fire if a narrow race condition is hit.
  • KAFKA-6015: NPE in RecordAccumulator.
  • KAFKA-6012: NoSuchElementException in markErrorMeter during TransactionsBounceTest.
  • KAFKA-6004: Enable custom authentication plugins to return error messages to clients.
  • KAFKA-6003: Replication Fetcher thread for a partition with no data fails to start.
  • KAFKA-5987: Kafka metrics templates used in document generation should maintain order of tags.
  • KAFKA-5970: Deadlock due to locking of DelayedProduce and group.
  • KAFKA-5960: Producer uses unsupported ProduceRequest version against older brokers.
  • KAFKA-5959: NPE in NetworkClient.
  • KAFKA-5957: Producer IllegalStateException due to second deallocate after aborting a batch.
  • KAFKA-5879: Controller should read the latest IsrChangeNotification znodes when handling IsrChangeNotification event.
  • KAFKA-5829: Speedup broker startup after unclean shutdown by reducing unnecessary snapshot files deletion.
  • KAFKA-5790: SocketServer.processNewResponses should not skip a response if exception is thrown.
  • KAFKA-5767: Kafka server should halt if IBP < 1.0.0 and there is log directory failure.
  • KAFKA-5752: Delete topic and re-create topic immediate will delete the new topic's timeindex.
  • KAFKA-5708: Update Jackson dependencies (from 2.8.5 to 2.9.x).
  • KAFKA-5630: Consumer poll loop over the same record after a CorruptRecordException.
  • KAFKA-5610: KafkaApis.handleWriteTxnMarkerRequest can return UNSUPPORTED_FOR_MESSAGE_FORMAT error on partition emigration.
  • KAFKA-5600: Group loading regression causing stale metadata/offsets cache.
  • KAFKA-5556: KafkaConsumer.commitSync throws IllegalStateException: Attempt to retrieve exception from future which hasn't failed.
  • KAFKA-5417: Clients get inconsistent connection states when SASL/SSL connection is marked CONNECTED and DISCONNECTED at the same time.
  • KAFKA-4669: KafkaProducer.flush hangs when NetworkClient.handleCompletedReceives throws exception.