4.3. Hadoop Common/HDFS 2.6.0

HDP 2.2.4 provides Apache Hadoop Core 2.6.0 and the following additional Apache patches:

  • HDFS-3107. Introduce truncate.

  • HDFS-7009. Active NN and standby NN have different live nodes.

  • HDFS-7056. Snapshot support for truncate.

  • HDFS-7058. Tests for truncate CLI

  • HDFS-7263. Snapshot read can reveal future bytes for appended files.

  • HDFS-7425. NameNode block deletion logging uses incorrect appender.

  • HDFS-7443. Datanode upgrade to BLOCKID_BASED_LAYOUT fails if duplicate block files are present in the same volume

  • HDFS-7470. SecondaryNameNode need twice memory when calling reloadFromImageFile.

  • HDFS-7489. Incorrect locking in FsVolumeList#checkDirs can hang datanodes.

  • HDFS-7503. Namenode restart after large deletions can cause slow processReport

  • HDFS-7606. Fix potential NPE in INodeFile.getBlocks().

  • HDFS-7634. Disallow truncation of Lazy persist files.

  • HDFS-7638. Small fix and few refinements for FSN#truncate.

  • HDFS-7643. Test case to ensure lazy persist files cannot be truncated.

  • HDFS-7655. Expose truncate API for Web HDFS.

  • HDFS-7659. Truncate should check negative value of the new length

  • HDFS-7676. Fix TestFileTruncate to avoid bug of HDFS-7611.

  • HDFS-7677. DistributedFileSystem#truncate should resolve symlinks.

  • HDFS-7707. Edit log corruption due to delayed block removal again.

  • HDFS-7714. Simultaneous restart of HA NameNodes and DataNode can cause DataNode to register successfully with only one NameNode.

  • HDFS-7733. NFS: readdir/readdirplus return null directory attribute on failure.

  • HDFS-7738. Revise the exception message for recover lease; add more truncate tests such as truncate with HA setup, negative tests, truncate with other operations and multiple truncates.

  • HDFS-7760. Document truncate for WebHDFS.

  • HDFS-7831. Fix the starting index and end condition of the loop in FileDiffList.findEarlierSnapshotBlocks().

  • HDFS-7843. A truncated file is corrupted after rollback from a rolling upgrade.

  • HDFS-7885. Datanode should not trust the generation stamp provided by client.

  • HADOOP-926. Do not fail job history iteration when encounting missing directories.

  • HADOOP-941: Addendum patch.

  • HADOOP-11321. copyToLocal cannot save a file to an SMB share unless the user has Full Control permissions.

  • HADOOP-11368. Fix SSLFactory truststore reloader thread leak in KMSClientProvider.

  • HADOOP-11381. Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack

  • HADOOP-11381. Revert. (Fix findbugs warnings in hadoop-distcp, hadoop-aws, hadoop-azure, and hadoop-openstack.)

  • HADOOP-11412. POMs mention "The Apache Software License" rather than "Apache License".

  • HADOOP-11412. Revert. (POMs mention "The Apache Software License" rather than "Apache License"

  • HADOOP-11490. Expose truncate API via FileSystem and shell command.

  • HADOOP-11509. change parsing sequence in GenericOptionsParser to parse -D parameters before -files.

  • HADOOP-11510. Expose truncate API via FileContext.

  • HADOOP-11523. StorageException complaining " no lease ID" when updating FolderLastModifiedTime in WASB.

  • HADOOP-11579. Documentation for truncate.

  • HADOOP-11595. Add default implementation for AbstractFileSystem#truncate.

  • MAPREDUCE-6230. Fixed RMContainerAllocator to update the new AMRMToken service name properly.

  • YARN-570. Time strings are formated in different timezone.

  • YARN-2246. Made the proxy tracking URL always be http(s)://proxy addr:port/proxy/<appId> to avoid duplicate sections.

  • YARN-2571. RM to support YARN registry

  • YARN-2683. registry config options: document and move to core-default

  • YARN-2837. Support TimeLine server to recover delegation token when restarting.

  • YARN-2917. Fixed potential deadlock when system.exit is called in AsyncDispatcher

  • YARN-2964. RM prematurely cancels tokens for jobs that submit jobs (oozie).

  • YARN-3103. AMRMClientImpl does not update AMRM token properly.

  • YARN-3207. Secondary filter matches entites which do not have the key being filtered for.

  • YARN-3227. Timeline renew delegation token fails when RM user's TGT is expired.

  • YARN-3239. WebAppProxy does not support a final tracking url which has query fragments and params.

  • YARN-3251. Fixed a deadlock in CapacityScheduler when computing absoluteMaxAvailableCapacity in LeafQueue.

  • YARN-3269. Yarn.nodemanager.remote-app-log-dir could not be configured to fully qualified path.