HDP-2.3.4 Release Notes
Also available as:
PDF

Fixed Issues

[Important]Important

Hortonworks strongly recommends that all users running HDP 2.3.4 upgrade to HDP 2.3.4.7.

Fixed issues represents selected issues that were previously logged via Hortonworks Support, but are now addressed in the current release. These issues may have been reported in previous versions within the Known Issues section; meaning they were reported by customers or identified by Hortonworks Quality Engineering team.

Potential Data Loss

None.

Security

Hortonworks Bug ID

Apache JIRA

Component

Summary

BUG-28778HIVE-9013HiveHive set command exposes metastore db password
BUG-35952RANGER-680RangerDefault policies for KMS repo
BUG-44378RANGER-657RangerRanger-Solr: Issue with Mapping Principals to user names "solr"
BUG-45035RANGER-661RangerPlugin receives empty policy list though the service has policies
BUG-45039RANGER-663RangerRace condition during policy update causes policy to get in an bad state
BUG-45431HDFS-9175HDFSChange scope of 'AccessTokenProvider.getAccessToken()' and 'CredentialBasedAccessTokenProvider.getCredential()' abstract methods to public
BUG-45489HIVE-11995HiveRemove repetitively setting permissions in insert/load overwrite partition
BUG-45948 KafkaCommand line script to recursively change zookeeper ACLs for Kafka zookeeper znodes
BUG-46149HIVE-10528HiveHiveserver2 in HTTP mode is not applying auth_to_local rules
BUG-46207FALCON-1027FalconAdded trusted proxy support to the Falcon REST API
BUG-46556RANGER-714RangerRestrict group and role data for only ADMIN users
BUG-47553RANGER-608RangerDenied access to HDFS file/directory is not audited in Ranger
BUG-47606RANGER-749RangerRanger KMS to support multiple KMS instances with keys across multiple clusters
BUG-47791 HueDjango Vulnerabilities in Hue
BUG-47823HADOOP-12049Hadoop CommonControl http authentication cookie persistence via configuration. HADOOP-12049 introduces a new configuration setting to control the persistence of the HTTP cookie passed to the client during authentication with Hadoop HTTP consoles.
BUG-49158HADOOP-12617Hadoop CommonSPNEGO authentication request to non-default realm gets default realm name inserted in target server principal

Incorrect Results

Hortonworks Bug ID

Apache JIRA

Component

Summary

BUG-41313HIVE-11432HiveHive macro give same result for different arguments
BUG-44489HIVE-11892HiveUDTF run in local fetch task does not return rows forwarded during GenericUDTF.close()
BUG-44539SQOOP-2399SqoopSqoop import into HBase - BigDecimalSplitter java.lang.ArrayIndexOutOfBoundsException
BUG-45496HIVE-11517HiveVectorized auto_smb_mapjoin_14.q produces different results
BUG-45582 HueCluster running Hue 2.6.1 Null values are being returned as 'None' and they are converted to blank value while exporting as CSV file
BUG-46167PHOENIX-2313Phoenixorg.apache.phoenix.schema.TypeMismatchException Error while querying a table that has an index with a Boolean
BUG-46507HIVE-12230Hivecustom UDF configure() not called in Vectorization mode
BUG-47209 Rangercheck_db_connnection in Ambari setup_ranger.py does not properly escape before passing to shell

Stability

Hortonworks Bug ID

Apache JIRA

Component

Summary

BUG-26978HDFS-6481HDFSDatanodeManager#getDatanodeStorageInfos() should check the length of storageIDs
BUG-27659HIVE-12556HiveCtrl-C in beeline doesn't kill Tez query on HS2
BUG-30068HIVE-12444HiveGlobal Limit optimization on ACID table without base directory may throw exception
BUG-42203HIVE-11499HiveDatanucleus leaks classloaders when used using embedded metastore with HiveServer2 with UDFs
BUG-42415HIVE-11422HiveJoin a ACID table with non-ACID table fail with MR
BUG-43592HIVE-12204HiveTez queries stopped running with ApplicationNotRunningException
BUG-43603HIVE-12262HiveSession log dir cannot be created in some cases
BUG-44212HIVE-12084HiveHive queries with ORDER BY and large LIMIT fails with OutOfMemoryError Java heap space
BUG-44549PHOENIX-2118PhoenixRemove/modify usages of Guava StopWatch and deprecated ComparisonChain methods
BUG-45090ZOOKEEPER-1952ZooKeeperzookeeper.log.file property is not respected; log output goes only to the zookeeper.out
BUG-45179PIG-4635Pig, TezPig 0.15 on Tez 0.7 NPE
BUG-45238HADOOP-12089Hadoop CommonStorageException complaining "no lease ID" when updating FolderLastModifiedTime in WASB
BUG-45240HADOOP-12239Hadoop CommonStorageException complaining "no lease ID" when updating FolderLastModifiedTime in WASB
BUG-45346 KafkaSleep value for KILL command is set to 0.1 in install_kafka.sh
BUG-45361HBASE-14594HBaseUse new DNS API introduced in HADOOP-12437
BUG-45465HIVE-11977HiveHive should handle an external avro table with zero length files present
BUG-45664 KafkaMemory leak in Kafka Broker caused by leak in instance of ConcurrentHashMap/socketContainer
BUG-45688KAFKA-2012KafkaKafka index file corruption
BUG-45717STORM-1030StormSynchronization error between HiveBolt and heartbeat thread causing missed HiveWriter
BUG-45943HBASE-14207HBaseRegion was hijacked and remained in transition when RS failed to open a region and later regionplan changed to new RS on retry
BUG-45952HBASE-14317HBaseStuck FSHLog: bad disk (HDFS-8960) and can't roll WAL
BUG-45955HBASE-14361HBaseReplicationSink should create Connection instances lazily
BUG-46328YARN-2910YARNFSLeafQueue can throw ConcurrentModificationException
BUG-46607HADOOP-12484Hadoop CommonSingle File Rename Throws Incorrectly In Potential Race Condition Scenarios
BUG-47136HADOOP-11685Hadoop CommonStorageException complaining "no lease ID" during HBase distributed log splitting
BUG-47179HADOOP-12508Hadoop CommonDelete fails with exception when lease is held on blob
BUG-47180HADOOP-12533Hadoop CommonIntroduce FileNotFoundException in WASB for read and seek API
BUG-47480FALCON-1595FalconFalcon server loses ability to communicate with HDFS over time
BUG-47483HIVE-11698HiveAdd additional test for PointLookupOptimizer
BUG-47563HIVE-12364HiveDistcp job fails when run under Tez
BUG-47757HIVE-12156Hive, Hive Viewexpanding view doesn't quote reserved keyword
BUG-47910HIVE-12250HiveZooKeeper connection leaks in Hive's HBaseHandler.
BUG-48437HIVE-12476HiveMetastore NPE on Oracle with Direct SQL
BUG-49159STORM-1030StormStorm HiveBolt sendHeartBeat flag is set to false after connectivity is lost with Hive database, causing heartbeats to stop until worker is restarted

Query Failure

None.

Upgrade

Hortonworks Bug ID

Apache JIRA

Component

Summary

BUG-44874RANGER-677RangerAfter upgrade, Ranger Admin fails to render policies referring to groups that contain "." in name
BUG-44884RANGER-706Ranger2.2 to 2.3 upgrade timeout while migrating Ranger
BUG-46275AMBARI-13578Ambari, RangerHive Server Start times out due to ranger http calls
BUG-46387 Ambari, RangerUpgrade: Ranger needs ldap bind password after upgrade
BUG-46390 Ambari, RangerPatched Ranger fails during RU
BUG-48155AMBARI-14153RangerHandle install failures/patch status during ranger upgrade and make detailed logs available

Usability

Hortonworks Bug ID

Apache JIRA

Component

Summary

BUG-38913AMBARI-11634KafkaKafka Service Check seems to complete successfully, however the stdout indicates an error creating a previously created topic
BUG-43333YARN-4009, YARN-2513Tez, YARNFunctional standalone UI (hosted in ATS) for Tez
BUG-43578HIVE-11919HiveHive Union Type Mismatch
BUG-44481HDFS-9082HDFSChange the log level in WebHdfsFileSystem.initialize() from INFO to DEBUG
BUG-44568YARN-291, YARN-313YARNSupport NM resource dynamic reconfiguration with CLI.
BUG-45264HADOOP-12437HDFSHDFS and YARN configs for Kerberized Dual Homing
BUG-45396ATLAS-198AtlasAtlas Dashboard Requires Internet Connectivity
BUG-45569HIVE-12235HiveImprove beeline logging for dynamic service discovery
BUG-45583HIVE-11960Hivebraces in join conditions are not supported
BUG-45722HADOOP-12350Hadoop CommonWASB Logging: Improve WASB Logging around deletes, reads and writes
BUG-47306RANGER-712RangerAdd a samples project to Ranger to enable clients and VARs to write extensions to Ranger
BUG-47475HIVE-10592HiveORC file dump in JSON format
BUG-48014HDFS-8829HDFSMake SO_RCVBUF and SO_SNDBUF size configurable for DataTransferProtocol sockets and allow configuring auto-tuning
BUG-48022STORM-1203StormWorker metadata file creation doesn't use storm.log.dir config
BUG-48187MAPREDUCE-6478MapReduceAdd an option to skip cleanupJob stage or ignore cleanup failure during commitJob().
BUG-49030 HueNo “permission denied” message is displayed in hue file browser when user with no permissions uploads a file.

Performance

Hortonworks Bug ID

Apache JIRA

Component

Summary

BUG-44879RANGER-681RangerUsersync interval between cycles is set to 1ms
BUG-46319PIG-4703PigTezOperator.stores shall not ship to backend
BUG-47415HIVE-12354HiveMapJoin with double keys is slow on MR
BUG-47481HIVE-11461HiveTransform flat AND/OR into IN struct clause
BUG-47485HIVE-11573HivePointLookupOptimizer can be pessimistic at a low nDV
BUG-47904HIVE-12312HiveExcessive logging in PPD code

Other

Hortonworks Bug ID

Apache JIRA

Component

Summary

BUG-44329HDFS-9063HDFSHDFS - Correctly handle snapshot path for getContentSummary
BUG-44544HADOOP-11098Hadoop CommonMax Non Heap Memory default changed between JDK7 and 8
BUG-45083HIVE-10752HCatalog, PigPig scripts that involve more than 66 columns return null results when MapReduce is the execution engine.
BUG-45687 Kafkakafka.out is not rotating causing it to grow very large in size
BUG-46243AMBARI-12678Ambari, KafkaInstalling additional Kafka broker on node with hostname that comes alphabetically before existing broker fails to start
BUG-47431HADOOP-12334Hadoop CommonChange Mode Of Copy Operation of HBase WAL Archiving to bypass Azure Storage Throttling after retries
BUG-47612HADOOP-12551Hadoop CommonIntroduce FileNotFoundException for open and getFileStatus API's in WASB
BUG-47731HDFS-8099HDFSChange "DFSInputStream has been closed already" message to debug log level
BUG-47820HBASE-14788HBaseSplitting a region does not support the hbase.rs.evictblocksonclose config when closing source region
BUG-49167HDFS-9434HDFSRecommission a datanode with 500k blocks may pause NN for 30 seconds
BUG-49324HDFS-9397HDFSFix typo for readChecksum() LOG.warn in BlockSender.java
BUG-49952

YARN-3987

YARN-3604

YARNBackport YARN-3604/YARN-3987 to 2.3-maint