Fixed Issues
Fixed issues represents selected issues that were previously logged via Hortonworks Support, but are now addressed in the current release. These issues may have been reported in previous versions within the Known Issues section; meaning they were reported by customers or identified by Hortonworks Quality Engineering team.
Incorrect Results
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-62638 | PHOENIX-3608 | Phoenix query dropping/ignoring additional query conditions |
| BUG-67879 | HIVE-15331 | Decimal: most multiplication returns incorrect results (NULL) |
| BUG-68526 | HIVE-15493 | Output of outer join query is different between HDP 2.3 and HDP 2.5 |
| BUG-69200 | HIVE-13936 | Incorrect result when a query having left outer join with subquery and hive.optimize.ppd enabled |
| BUG-69892 | HIVE-15338 | Wrong result from vectorized DATEDIFF with scalar parameter of type DATE/TIMESTAMP |
| BUG-73355 | SQOOP-3014 | Sqoop with HCatalog import loose precision for large numbers that does not fit into double |
| BUG-76290 | HIVE-16050 | Union of null with non-null |
Other
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-65326 | RANGER-1355 | RangerAuthorizer for Hive MSCK command |
| BUG-69874 | PHOENIX-3505 | (fenton) NPE on phoenix jdbc statement close |
| BUG-70505 | HADOOP-12666, HADOOP-12738, HADOOP-12875, HADOOP-13037, HADOOP-13242, HADOOP-13257, HADOOP-13392, HADOOP-13393, HADOOP-13956, HADOOP-13962, HADOOP-14017, HADOOP-14049, HDFS-11240 | Support Microsoft Azure Data Lake - as a file system in Hadoop |
| BUG-70961 | N/A | .Hive-staging folders created from Spark hiveContext are not getting cleaned up |
| BUG-76643 | HADOOP-14048 | REDO operation of WASB#AtomicRename should create placeholder blob for destination folder |
Performance
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-58370 | HIVE-13878 | Vectorized text about 2x slower on this simple query |
| BUG-62071 | HBASE-16675 | HBase normalizer default heuristic result in too may splits |
| BUG-65712 | HDFS-10569, HDFS-9484 | (fenton) NNThroughputBenchmark$BlockReportStats should not send empty block reports |
| BUG-67729 | HBASE-16773 | Integrate HBASE-16773 AccessController should access local region if possible |
| BUG-68699 | HDFS-11047 | Remove deep copies of FinalizedReplica to alleviate heap consumption on DataNode |
| BUG-69510 | STORM-2197 | NimbusCLient connections leak due to leakage in ThriftCLient |
| BUG-70717 | HADOOP-12185 | NetworkTopology is not efficient adding/getting/removing nodes |
| BUG-71848 | MAPREDUCE-6724 | Single shuffle to memory must not exceed Integer#MAX_VALUE |
Potential Data Loss
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-66878 | SPARK-16664 | Include SPARK-16664 in HDP |
| BUG-68473 | HIVE-15202 | compactions can step on each other causing data loss |
| BUG-73796 | HBASE-16649 | Port HBASE-16649 - Truncate table with splits preserved can cause both data loss and truncated data appeared again |
Query Failure
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-45149 | HIVE-11985, HIVE-15714 | CREATE EXTERNAL TABLE with Hive metastore in Oracle fails with ORA-01461 |
| BUG-67793 | HIVE-15488 | Vectorizer should avoid vectorizing non-primitive types |
| BUG-72178 | HIVE-16065 | [LLAP]: Vectorized tpc-ds query11 with UNION causes ClassCastException ...DecimalColumnVector cannot be cast to ...LongColumnVector |
| BUG-74311 | HIVE-13872 | Queries failing with java.lang.ClassCastException when vectorization is enabled |
| BUG-77383 | HIVE-16245 | Vectorization: Does not handle non-column key expressions in MERGEPARTIAL mode -- tpcds query11 fails with a ClassCastException - DoubleColumnVector to LongColumnVector |
Security
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-65326 | RANGER-1355 | RangerAuthorizer for Hive MSCK command |
| BUG-68038 | N/A | Integrate ZK patch for ZK C cli |
| BUG-68973 | N/A | Kafkaspout (old API) failing with secure kafka in HDP 2.5 |
| BUG-69197 | PHOENIX-3004 | Phoenix Query Server only accepts SPNEGO-authenticated clients from server's realm |
| BUG-70551 | ACCUMULO-4534 | (2.5-maint) SCA : XML External Entity Injection (Input Validation and Representation, Data Flow) |
| BUG-71078 | ATLAS-1402 | Sanitize user inputs taken in atlas UI |
| BUG-74097 | ATLAS-1508 | Atlas AD authentication Active Directory Authentication should authenticate on sAMAccountName attribute |
| BUG-74129 | RANGER-1338 | Ranger Plugin failed to download policy when JaasConfig alone is used to set the UGI instead of Principal/Keytab |
| BUG-74172 | PHOENIX-1754 | Phoenix jdbc url can not support keytab PATH in window file system |
| BUG-76127 | PHOENIX-3652 | User requires all Create privilege on 'default' namespace to connect using sqlline.py - Fenton |
| BUG-76645 | HBASE-14148 | Web UI Framable Page |
Stability
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-43023 | HDFS-9038 | dfs.datanode.du.reserved is not considered while computing available space (HDFS-9038) |
| BUG-51037 | ZOOKEEPER-1936 | Regionservers exit for FATAL exception during initialization |
| BUG-53884 | HIVE-14170, HIVE-7224 | beeline OOM on large results with default params |
| BUG-58185 | HIVE-15208 | Drilldown link on HS2 UI for a running query results in an NPE |
| BUG-64503 | HBASE-14798 | Integrate HBASE-14798 NPE reporting server load causes regionserver abort; causes TestAcidGuarantee to fail |
| BUG-64573 | HBASE-16429 | Integrate HBASE-16429 FSHLog: deadlock if rollWriter called when ring buffer filled with appends |
| BUG-64977 | YARN-5476 | Pig-HA stuck on nonexistent application id |
| BUG-65709 | HDFS-10569 | A bug causes OutOfIndex error in BlockListAsLongs |
| BUG-65799 | RANGER-1173 | Improve Ranger database schema import logic and make it more robust |
| BUG-65984 | N/A | (fenton) - Increase the kite sdk version to 1.1 for fixing Sqoop parquet support issues |
| BUG-66078 | HIVE-15181 | TxnHandler::performTimeout gets stuck in an infinite loop |
| BUG-66387 | STORM-2042 | Nimbus client connections not closed properly causing connection leaks (Storm-2042) |
| BUG-68046 | HADOOP-11780 | Prevent IPC reader thread death |
| BUG-68047 | HDFS-10609 | Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications |
| BUG-68060 | STORM-2158 | OutOfMemoryError in Nimbus' SimpleTransportPlugin |
| BUG-68077 | HDFS-10301 | BlockReport retransmissions may lead to storages falsely being declared zombie |
| BUG-68526 | HIVE-15493 | Output of outer join query is different between HDP 2.3 and HDP 2.5 |
| BUG-68737 | YARN-5718 | TimelineClient and other YARN places shouldn't overwrite hdfs client retry setting which affect NN failed-over case. |
| BUG-68946 | TEZ-3566 | HiveServer2 OOM - lots of DFSClient instances retained |
| BUG-68960 | MAPREDUCE-6684, MAPREDUCE-6797 | High contention on scanning of user directory under immediate_done in Job History Server |
| BUG-69235 | AMBARI-18936 | DataNode JVM heap settings should include CMSInitiatingOccupancy |
| BUG-69387 | YARN-4109 | [Fenton] Excessive logging happening while scheduler UI page is visited |
| BUG-69393 | N/A | Falcon feed fails to aws s3 with hdp 2.5; Oozie joda time version does not meet required jar version 2.2 or later |
| BUG-69396 | OOZIE-2705 | Oozie Spark action ignores spark.executor.extraJavaOptions and spark.driver.extraJavaOptions |
| BUG-69659 | N/A | Oozie job fails with NPE at LauncherMapper#setRecoveryId |
| BUG-69892 | HIVE-15338 | Wrong result from vectorized DATEDIFF with scalar parameter of type DATE/TIMESTAMP |
| BUG-70331 | HADOOP-13863 | Hadoop - Azure: Add a new SAS key mode for WASB |
| BUG-70423 | HBASE-17023 | Region left unassigned due to AM and SSH each thinking others would do the assignment work |
| BUG-70674 | HADOOP-13565, HADOOP-13890 | Access NameNode WebUI and got "GSSException: No valid credentials provided" error when using JDK 1.8.0_60. However, no problem on same cluster if uses jdk-1.7.0_65 |
| BUG-70718 | HADOOP-13831 | HADOOP-13831 - Correct check for error code to detect Azure Storage Throttling and provide retries |
| BUG-70957 | N/A | Spark Thrift Server failing after 24hrs due to delegation token renewal failure |
| BUG-71261 | HIVE-15645 | LLAP jobs won't run because AMs are launched in the wrong queue |
| BUG-71771 | YARN-5555 | Yarn RM UI % of Queue at application level is wrong |
| BUG-71825 | HADOOP-12017 | Hadoop archives command should use configurable replication factor when closing |
| BUG-71848 | MAPREDUCE-6724 | Single shuffle to memory must not exceed Integer#MAX_VALUE |
| BUG-72167 | TEZ-3575 | RM have started forwarding origin. Use that in AMWebController for CORS support |
| BUG-73010 | N/A | FullTableBackupProcedure#toStringClassDetails() should protect against empty table list |
| BUG-73376 | TEZ-3494 | Support relative url for tez-ui.history-url.base config |
| BUG-73728 | STORM-2360 | Storm hive bolt throws NoSuchMethodError and NullPointerException |
| BUG-74157 | HIVE-15579 | Port HADOOP_PROXY_USER to fenton |
| BUG-74311 | HIVE-13872 | Queries failing with java.lang.ClassCastException when vectorization is enabled |
| BUG-74312 | HIVE-15542 | Getting NPE running a simple hive join in Tez engine after HDP upgrade |
| BUG-74953 | RANGER-1379 | Seeing HTTP 404 in the logs for PolicyRefresher intermittently for different services |
| BUG-76495 | HIVE-16014 | HiveMetastoreChecker should use hive.metastore.fshandler.threads instead of hive.mv.files.thread for pool size |
| BUG-76739 | HADOOP-14150 | Implement getHomeDirectory() method in NativeAzureFileSystem |
| BUG-76956 | HDFS-11461 | DataNode Disk Outlier Detection |
| BUG-76957 | HDFS-11477 | Simplify file IO profiling configuration |
Supportability
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-66650 | HDFS-6184 | Capture NN's thread dump when it fails over |
| BUG-74475 | YARN-6145 | YARN-6145 Improve log message on fail over |
| BUG-74594 | YARN-4496, YARN-5711, YARN-6145 | Connection refused exception call stack creating panic among end users |
| BUG-74953 | RANGER-1379 | Seeing HTTP 404 in the logs for PolicyRefresher intermittently for different services |
| BUG-77461 | HDFS-11292 | Log lastWrittenTxId info in logSyncAll |
Usability
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-54415 | PHOENIX-2783, PHOENIX-2790, PHOENIX-930 | Creating secondary index with duplicate name causes table inaccessible |
| BUG-58753 | PHOENIX-3360 | Our default secondary index configuration is wrong for RPC controller |
| BUG-61383 | HIVE-14600 | LLAP daemon does not shutdown if registration fails |
| BUG-64005 | HADOOP-13552 | Hdfs prints WARN messages during file creation causing issues for Storm |
| BUG-64511 | HDFS-9618 | Fix mismatch between log level and guard in BlockManager#computeRecoveryWorkForBlocks |
| BUG-64573 | HBASE-16429 | Integrate HBASE-16429 FSHLog: deadlock if rollWriter called when ring buffer filled with appends |
| BUG-64977 | YARN-5476 | Pig-HA stuck on nonexistent application id |
| BUG-65680 | TEZ-3433 | Tez UI: Searching using wrong ID causes error in all DAGs page |
| BUG-66066 | ACCUMULO-4424, ACCUMULO-4456 | (fenton) Do not wait to start Thrift servers until lock is acquired |
| BUG-66253 | SPARK-17612 | Spark2-Hive: ParseException for `DESC ~ PARTITION` |
| BUG-66359 | SLIDER-1169 | Slider not honoring zookeeper quorum values passed |
| BUG-66649 | HIVE-14933 | LLAP may fail to start if Python dependencies like argparse are not there |
| BUG-67340 | PHOENIX-2930 | Cannot resolve columns aliased to its own name |
| BUG-68080 | HDFS-11054 | Suppress verbose log message in BlockPlacementPolicyDefault |
| BUG-68146 | SPARK-18160 | spark.files & spark.jars should not be passed to driver in yarn mode |
| BUG-68690 | HIVE-15120 | Hive is not using Storage Based Authorization while dropping external tables. |
| BUG-68895 | PHOENIX-3241 | [PHOENIX-3241] Convert_tz doesn't allow timestamp data type |
| BUG-69590 | N/A | Oozie allowing workflow names up to 40 characters |
| BUG-70061 | N/A | Pig job statuses remain 'Running' for succeeded jobs in query history |
| BUG-70717 | HADOOP-12185 | NetworkTopology is not efficient adding/getting/removing nodes |
| BUG-70753 | RANGER-698 | Ranger policy should support variables like $user |
| BUG-72376 | N/A | Kafka cli command fails when JMX port is added |
| BUG-72548 | HADOOP-13988 | WebHDFS through Knox does not work with TDE |
| BUG-73351 | SQOOP-2408, SQOOP-2582 | Parquet file support for Sqoop for HDP 2.4.0 |
| BUG-73352 | SQOOP-3003 | Sqoop import fails to query with split-by/boundary-query using Oracle Date/Timestamp |
| BUG-73357 | SQOOP-2995 | backward incompatibility introduced by Custom Tool options |
| BUG-73358 | SQOOP-3033 | Sqoop option --skip-dist-cache is not saved as a parameter when saving Sqoop Job |
| BUG-73359 | SQOOP-2999 | Sqoop ClassNotFoundException (org.apache.commons.lang3.StringUtils) is thrown when executing Oracle direct import map task |
| BUG-74129 | RANGER-1338 | Ranger Plugin failed to download policy when JaasConfig alone is used to set the UGI instead of Principal/Keytab |
| BUG-74700 | HBASE-17460 | enable_table_replication can not perform cyclic replication of a table |
| BUG-75000 | HDFS-11391 | Cannot write to WebHDFS with username starting with numbers |
| BUG-75116 | HBASE-17648 | Integrate HBASE-17648 HBase Table-level synchronization fails between two secured(kerberized) cluster |
| BUG-75586 | PHOENIX-3686 | De-couple PQS's use of Kerberos to talk to HBase and client authentication |
| BUG-75764 | SQOOP-2349 | Unable read from a locked DB2 AS400 table with "transaction.isolation=TRANSACTION_READ_UNCOMMITTED" |
| BUG-75952 | N/A | hive variables being set as conf variables from connection url |