Kafka
HDP 2.3.4.7 provides Kafka 0.9.0 with the patches specified below. No new additional Apache patches have been included in this release.
HDP 2.3.4 provided Kafka 0.9.0 and the following Apache patches:
KAFKA-2803: Add hard bounce system test for Kafka Connect.
KAFKA-2812: improve consumer integration tests.
KAFKA-2862: Fix MirrorMaker's message.handler.args description.
KAFKA-2872: unite sink nodes with parent nodes in addSink.
KAFKA-2877: handle request timeout in sync group.
KAFKA-2878: Guard against OutOfMemory in Kafka broker.
KAFKA-2879: Make MiniKDC test service slightly more generic.
KAFKA-2880: consumer should handle disconnect/timeout for metadata requests.
KAFKA-2881: Improve Consumer Configs and API Documentation.
KAFKA-2882: Add constructor cache for Snappy and LZ4 Output/Input streams in Compressor.java.
KAFKA-2892: Consumer Docs Use Wrong Method.
KAFKA-2899: improve logging when unexpected exceptions thrown in reading local log.
KAFKA-2906: Fix Connect javadocs, restrict only to api subproject, and clean up javadoc warnings.
KAFKA-2913: missing partition check when removing groups from cache.
KAFKA-2942: inadvertent auto-commit when pre-fetching can cause message loss.
KAFKA-2950: Fix performance regression in the producer.
HDP 2.3.2 provided Kafka 0.8.2, with no additional Apache patches.
HDP 2.3.0 provided Kafka 0.8.2 and the following Apache patches:
KAFKA-1005 Shutdown consumer at the end of consumer performance test.
KAFKA-1416 Unify sendMessages in TestUtils
KAFKA-1461 Replica fetcher thread does not implement any back-off behavior
KAFKA-1499 trivial follow-up (remove unnecessary parentheses)
KAFKA-1501 Let the OS choose the port in unit tests to avoid collisions
KAFKA-1517 Messages is a required argument to Producer Performance Test
KAFKA-1546 Automate replica lag tuning;
KAFKA-1634 Bump up Offset Commit Request to v2 to add global retention and remove per-partition commit timestamp
KAFKA-1664 Kafka does not properly parse multiple ZK nodes with non-root chroot
KAFKA-1683 add Session concept in SocketServer.PlainTextTransportLayer fixes.
KAFKA-1684 Kerberos/SASL implementation.
KAFKA-1688 Adding all public entities for adding a pluggable authorizer to Kafka.
KAFKA-1755 Reject compressed and unkeyed messages sent to compacted topics
KAFKA-1809 Refactor brokers to allow listening on multiple ports and IPs
KAFKA-1824 ConsoleProducer - properties key.separator and parse.key no longer work
KAFKA-1845 KafkaConfig should use ConfigDef
KAFKA-1852 Reject offset commits to unknown topics
KAFKA-1863 Add docs for possible thrown exception in Callback;
KAFKA-1865 Add a flush() method to the producer.
KAFKA-1866 LogStartOffset gauge throws exceptions after log.delete()
KAFKA-1910 Refactor new consumer and fixed a bunch of corner cases / unit tests
KAFKA-1926 Replace kafka.utils.Utils with o.a.k.common.utils.Utils
KAFKA-1961 Prevent deletion of _consumer_offsets topic
KAFKA-1973 Remove the accidentally created LogCleanerManager.scala.orig
KAFKA-1982 (add missing files) change kafka.examples.Producer to use the new java producer
KAFKA-1982 change kafka.examples.Producer to use the new java producer
KAFKA-1986 Request failure rate should not include invalid message size and offset out of range
KAFKA-1988 Fix org.apache.kafka.common.utils.Utils.abs and add Partitioner.toPositive
KAFKA-1989 New purgatory design; patched by Yasuhiro Matsuda
KAFKA-1990 Add unlimited time-based log retention
KAFKA-1992 checkEnoughReplicasReachOffset doesn't need to get requiredAcks
KAFKA-1994 Evaluate performance effect of chroot check on Topic creation
KAFKA-1996 Fix scaladoc error.
KAFKA-1997 Refactor MirrorMaker based on KIP-3;
KAFKA-2002 Mx4JLoader doesn't disable when kafka_mx4jenable=false.
KAFKA-2009 Fix two minor bugs in mirror maker.
KAFKA-2013 benchmark test for the purgatory
KAFKA-2016 RollingBounceTest takes long
KAFKA-2024 Log compaction can generate unindexable segments.
KAFKA-2033 Small typo in documentation
KAFKA-2034 sourceCompatibility not set in Kafka build.gradle
KAFKA-2039 Update Scala to 2.10.5 and 2.11.6
KAFKA-2042 Update topic list of the metadata regardless of cluster information;
KAFKA-2043 CompressionType is passed in each RecordAccumulator append
KAFKA-2044 Support requests and responses from o.a.k.common in KafkaApis
KAFKA-2047 Move the stream creation into concurrent mirror maker threads
KAFKA-2048 Change lock synchronized to inLock() for partitionMapCond
KAFKA-2050 Avoid calling .size() on linked list.
KAFKA-2056 Fix transient testRangePartitionAssignor failure
KAFKA-2088 kafka-console-consumer.sh should not create zookeeper path when no brokers found and chroot was set in zookeeper.connect.
KAFKA-2090 Remove duplicate check to metadataFetchInProgress
KAFKA-2096 Enable keepalive socket option for broker to prevent socket leak
KAFKA-2099 BrokerEndPoint file, methods and object names should match
KAFKA-2104 testDuplicateListeners() has a typo
KAFKA-2109 Support retries in KafkaLog4jAppender
KAFKA-2112 make overflowWheel volatile
KAFKA-2113 TestPurgatoryPerformance does not compile using IBM JDK
KAFKA-2114 Unable to change min.insync.replicas default.
KAFKA-2115 Error updating metrics in RequestChannel
KAFKA-2117 Use the correct metadata field for reading offset struct
KAFKA-2118 Cleaner cannot clean after shutdown during replaceSegments.
KAFKA-2119 ConsumerRecord key() and value() methods should not have throws Exception
KAFKA-2121 Close internal modules upon client shutdown
KAFKA-2122 Remove controller.message.queue.size Config
KAFKA-2128 kafka.Kafka should return non-zero exit code when caught exception.
KAFKA-2131 Update new producer javadocs with correct documentation links
KAFKA-2138 Fix producer to honor retry backoff
KAFKA-2140 Improve code readability
KAFKA-527 Compression support does numerous byte copies;