Fixed Issues
Fixed issues represents selected issues that were previously logged via Hortonworks Support, but are now addressed in the current release. These issues may have been reported in previous versions within the Known Issues section; meaning they were reported by customers or identified by Hortonworks Quality Engineering team.
Incorrect Results
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-65370 | HIVE-14652 | HIVE-14652 : incorrect results for not in on partition columns |
| BUG-65908 | HIVE-14595 | Backport HIVE-14595: TimestampWritable::setTimestamp gives wrong result when 2nd VInt exists |
| BUG-66084 | HIVE-14530 | Backport HIVE-14530: Union All query returns incorrect results |
| BUG-66121 | HIVE-14251 | Backport HIVE-14251 (incorrect results) |
| BUG-66335 | SQOOP-3014 | Sqoop with HCatalog import loose precision for large numbers that does not fit into double |
| BUG-66624 | HIVE-14519 | Backport HIVE-14519: Multi insert query bug |
| BUG-66626 | HIVE-7239 | Backport HIVE-7239: Fix bug in HiveIndexedInputFormat implementation that causes incorrect query result when input backed by Sequence/RC files |
| BUG-67780 | HIVE-14590 | Backport HIVE-14590: CBO (Calcite Return Path) Incorrect result set when limit is present in one of the union branches |
| BUG-68306 | HIVE-14959 | Backport distinct + windowing related patches |
| BUG-68344 | HIVE-14251, HIVE-14590 | Backport HIVE-14251 |
Other
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-67349 | N/A | Apache Spark 2.0.1 Release |
Performance
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-62071 | HBASE-16675 | HBase normalizer default heuristic result in too may splits |
| BUG-65355 | HIVE-12077, HIVE-14693 | Backport HIVE-12077: MSCK Repair table should fix partitions in batches |
| BUG-65379 | RANGER-1145, RANGER-1146, RANGER-1161, RANGER-1162 | Ranger policy engine optimizations |
| BUG-65711 | HDFS-10569, HDFS-9484 | NNThroughputBenchmark$BlockReportStats should not send empty block reports |
| BUG-66654 | HADOOP-13436 | Override equals and hashCode to avoid connection leakage |
| BUG-66876 | OOZIE-2626 | Oozie workflow processing becomes slow after the increase of rows in WF_JOBS and WF_ACTIONS tables when the metastore is running on SQL Server |
| BUG-68306 | HIVE-14959 | Backport distinct + windowing related patches |
| HIVE-850 | HIVE-14589 | WASB LLAP locality |
| HIVE-874 | HIVE-14462, HIVE-14511 | Port msck fixes to Erie |
Potential Data Loss
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-66878 | SPARK-16664 | Include SPARK-16664 in HDP |
| BUG-68438 | HIVE-13083 | Writing HiveDecimal to ORC can wrongly suppress present stream |
Query Failure
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-65399 | HIVE-14702 | NPE in LlapSignerImpl.serializeAndSign() when using spark-llap after HS2 idle for a few days |
| BUG-65476 | HIVE-14400 | Backport HIVE-14400: Handle concurrent insert with dynamic partition |
| BUG-65942 | HIVE-14621 | Backport HIVE-14621: LLAP: memory.mode = none has NPE |
| BUG-66258 | HIVE-14715 | Backport HIVE-14715: Hive throws NumberFormatException with query with Null value |
| BUG-68378 | HIVE-15099 | PTFOperator.PTFInvocation didn't properly reset the input partition |
| HIVE-836 | HIVE-14698 | Datanucleus MSSQLServerAdapter generates incorrect syntax for OFFSET-FETCH clause |
| HIVE-837 | HIVE-14674 | Create external table via beeline fails with Metaexception: Incorrect syntax near the keyword 'with' |
| HIVE-866 | HIVE-14726 | Delete from an ACID table throws IndexOutOfBoundsException |
| HIVE-897 | HIVE-14783 | Delete from an ACID table fails on setting hive.optimize.sort.dynamic.partition true |
| HIVE-905 | HIVE-14819 | HIVE2: select query using udfmax does not finish map job. |
Security
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-65704 | YARN-3982 | Banned.users from container-executor.cfg are not actually banned |
| BUG-65717 | RANGER-1100 | When column-masking is set for an user, update operations should be denied |
| BUG-66193 | YARN-4591 | YARN Web UIs should provide a robots.txt |
| BUG-66764 | RANGER-1170 | Address keyadmin user restriction |
| BUG-67853 | KNOX-762 | Knox Dispatch to Service with Kerberos and Wire Encryption Fails |
| BUG-68901 | N/A | HiveCli Exec Access Control is looking for permissions with wrong userID. |
| RANGER-38 | N/A | Ranger should be able to authenticate with Kerberos token |
Stability
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-42602 | HIVE-13090 | Hive metastore crashes on NPE with ZooKeeperTokenStore |
| BUG-51037 | ZOOKEEPER-1936 | Regionservers exit for FATAL exception during initialization |
| BUG-65466 | HADOOP-8751, HDFS-9276 | Failed to Update HDFS Delegation Token for long running application in HA mode |
| BUG-65708 | HDFS-10569 | A bug causes OutOfIndex error in BlockListAsLongs |
| BUG-65709 | HDFS-10569 | A bug causes OutOfIndex error in BlockListAsLongs |
| BUG-66086 | HIVE-14607 | ORC split generation failed with exception: java.lang.ArrayIndexOutOfBoundsException: 1 |
| BUG-66148 | HIVE-14656 | Clean up driver instance in get_splits |
| BUG-66206 | HIVE-14591 | HS2 is shut down unexpectedly during the startup time |
| BUG-66387 | STORM-2042 | Nimbus client connections not closed properly causing connection leaks |
| BUG-66439 | HIVE-14774 | Canceling query using Ctrl-C in beeline might lead to stale locks |
| BUG-66654 | HADOOP-13436 | Override equals and hashCode to avoid connection leakage |
| BUG-66666 | HBASE-16721 | Concurrency issue in WAL unflushed seqId tracking |
| BUG-66893 | HIVE-14773 | NPE aggregating column statistics for date column in partitioned table |
| BUG-67260 | KNOX-754, KNOX-756 | Knox regression in support of encoding for non-ascii chars in Location header |
| BUG-68060 | STORM-2158 | OutOfMemoryError in Nimbus' SimpleTransportPlugin |
| BUG-68261 | HIVE-14648 | LLAP: Avoid private pages in the SSD cache |
| HADOOP-1040 | HADOOP-13403, HADOOP-13459, HADOOP-13513 | Port WASB performance improvement from HADOOP 13403 |
| HBASE-364 | N/A | HMaster shuts down with FATAL error with 'error or interrupted while splitting logs in wasb' |
| HBASE-365 | N/A | IntegrationTestRegionReplicaReplication fails with timeout in case of Humboldt cluster |
| HBASE-370 | N/A | IntegrationTestTimeBoundedRequestsWithRegionReplicas failed with 'AssertionError: Verification failed with error code 1' due to read failures |
| HBASE-376 | HBASE-16604 | IntegrationTestBigLinkedList - Verify part failed with 'Found an undefined node. Undefined count=1553472' |
| HBASE-381 | N/A | HBase major_compact action is failing |
| HBASE-409 | HBASE-16824 | Synchronization Issue between Syncer and Writer causing exception during Hbase Log rollover |
| HIVE-761 | hive-12976 | MetaStoreDirectSql doesn't batch IN lists in all cases |
| HIVE-877 | HIVE-14624 | Short name registration of LLAP broken on HDInsight clusters |
| HIVE-879 | HIVE-14739 | Deadlock while running Hive queries |
| HIVE-908 | HIVE-14814 | Hive2: Hive.loadDynamicPartitions (transactional )fails with Nullpointer Exception at hive.ql.exec.MoveTask. nul |
| MAPREDUCE-84 | N/A | History Server is shutdown with NPE |
| OOZIE-330 | N/A | 'insert overwrite' query intermittently failed |
| OOZIE-332 | HADOOP-12954 | Oozie workflow start failed org.apache.oozie.action.ActionExecutorException: JA001: sometoken |
| PHOENIX-111 | CALCITE-1458 | HDP 2.5 Phoenix Query Server is broken |
| RANGER-51 | RANGER-1173 | RANGER_ADMIN install failure |
| SLIDER-20 | N/A | testAMKilledWithStateAMStartedAgentsStarted fails intermittently in ADLS |
| SPARK-172 | N/A | HbaseSource, DataType and CompositeKey example fails with TableNotEnabledException: shcExampleTable |
| YARN-220 | N/A | Timeline Service failed with " cause: java.io.IOException: /atshistory/active does not exist" |
Supportability
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-65661 | HIVE-13756 | Map failure attempts to delete reducer _temporary directory on multi-query pig query |
| BUG-66667 | HBASE-16678 | MapReduce jobs do not update counters from ScanMetrics |
| BUG-67261 | HIVE-14762, HIVE-14922 | Backport logging related patches |
| HIVE-826 | HIVE-13941 | Propagate better error messages for schematool failures during Hive metastore start |
| STORM-121 | STORM-2127 | Update eventhub client dependency for Storm-Eventhub |
Upgrade
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-66735 | N/A | Storm nimbus doesn't start after upgrading to Ambari 2.4.1 |
| ZEP-18 | N/A | Zeppelin not able to install on Spark2 |
Usability
| Hortonworks Bug ID | Apache JIRA | Summary |
|---|---|---|
| BUG-58753 | PHOENIX-3360 | Our default secondary index configuration is wrong for RPC controller |
| BUG-61383 | HIVE-14600 | LLAP daemon does not shutdown if registration fails |
| BUG-64511 | HDFS-9618 | Fix mismatch between log level and guard in BlockManager#computeRecoveryWorkForBlocks |
| BUG-65051 | RANGER-1167 | Page not found (404) after click on OK in admin create policy audit |
| BUG-65206 | ATLAS-1140, ATLAS-1141 | Atlas UI improvements & fix |
| BUG-65250 | HDFS-10764 | Customer's NN failed over with corrupted edits log, Secondary NN appears to fail to start with null pointer |
| BUG-65286 | ATLAS-1147 | UI: column name doesn't show up in schema tab for hive table |
| BUG-65384 | PIG-4976 | streaming job with store clause stuck if the script fail |
| BUG-65421 | HIVE-13723 | Executing join query on type Float using Thrift Serde will result in Float cast to Double error |
| BUG-65498 | HIVE-14411 | selecting Hive on Hbase table may cause FileNotFoundException |
| BUG-65520 | HIVE-14690 | Query fail when hive.exec.parallel=true, with conflicting session dir |
| BUG-65659 | HIVE-14457 | Partitions in encryption zone are still trashed though an exception is returned |
| BUG-65680 | TEZ-3433 | Tez UI: Searching using wrong ID causes error in all DAGs page |
| BUG-65704 | YARN-3982 | Banned.users from container-executor.cfg are not actually banned |
| BUG-65758 | HIVE-14436 | Error hit after enabling hive.optimize.skewjoin |
| BUG-65861 | SQOOP-3033 | Sqoop option --skip-dist-cache is not saved as a parameter when saving Sqoop Job |
| BUG-65901 | HIVE-13185 | orc.ReaderImp.ensureOrcFooter() method fails on small text files with IndexOutOfBoundsException |
| BUG-65980 | SQOOP-2408, SQOOP-2582 | Parquet file support for Sqoop for HDP 2.4.0 |
| BUG-65981 | SQOOP-2408, SQOOP-2582 | Parquet file support for Sqoop for HDP 2.4.0 |
| BUG-65983 | N/A | Increase the kite sdk version to 1.1 for fixing Sqoop parquet support issues |
| BUG-66065 | ACCUMULO-4424, ACCUMULO-4456 | Do not wait to start Thrift servers until lock is acquired |
| BUG-66087 | HIVE-14697 | Can not access kerberized HS2 Web UI |
| BUG-66096 | RANGER-1175 | Create policy fails for updated servicedef |
| BUG-66100 | HADOOP-12457 | fs.shell.Delete.java contains an invalid character that makes test_DFSCallsFromStreaming fail |
| BUG-66126 | RANGER-1176 | Ranger admin does not allow to create / update a policy with only delegate admin permission |
| BUG-66132 | HIVE-14149 | Hive not able to connect to S3 |
| BUG-66359 | SLIDER-1169 | Slider not honoring zookeeper quorum values passed |
| BUG-66376 | SQOOP-3003 | Sqoop import fails to query with split-by/boundary-query using Oracle Date/Timestamp |
| BUG-66428 | RANGER-1169 | Global audit settings are not honored unless at least one ranger policy exists |
| BUG-66498 | HIVE-14805 | Subquery inside a view will have the object in the subquery as the direct input |
| BUG-66577 | HIVE-11120 | ORC output format creates 0 length files if no data is written |
| BUG-66666 | HBASE-16721 | Concurrency issue in WAL unflushed seqId tracking |
| BUG-66737 | HIVE-14865 | Fix comments after HIVE-14350 |
| BUG-66784 | ATLAS-1206 | Atlas UI works in Firefox browser and NOT with IE or Chrome |
| BUG-66981 | ATLAS-1199, ATLAS-1215 | Atlas UI not loading after fresh build due to jquery-asBreadcrumbs plugin upgrade. |
| BUG-67042 | ATLAS-1192 | Atlas IE Support |
| BUG-67110 | HIVE-14448 | Queries with predicate fail when ETL split strategy is chosen for ACID tables - 2.5-maint |
| BUG-67120 | HIVE-14659 | OutputStream won't close if caught exception in funtion unparseExprForValuesClause in SemanticAnalyzer.java |
| BUG-67189 | SQOOP-2995 | Backward incompatibility introduced by Custom Tool options |
| BUG-67343 | HDFS-9008 | HDFS-9008 was not backported correctly |
| BUG-67742 | SQOOP-2999 | Sqoop ClassNotFoundException (org.apache.commons.lang3.StringUtils) is thrown when executing Oracle direct import map task |
| BUG-68436 | HIVE-14933 | LLAP may fail to start if Python dependencies like argparse are not there (erie) |
| OOZIE-329 | OOZIE-2690 | Oozie kill got NPE |
| PIG-222 | PIG-4972, PIG-4976 | Pig streaming yarn app is running forever. |
| RANGER-39 | RANGER-1167 | Ranger admin UI does not load in Internet Explorer |
| SPARK-170 | N/A | critical fixes in Spark2 |
| SPARK-175 | SPARK-17512 | Specifying remote files for Python based Spark jobs in Yarn cluster mode not working |
| STORM-117 | N/A | HDFS topologies are failing because of missing dependency |
| ZEP-1 | ZEPPELIN-1400 | Use relative path to the interpreter setting page |
| ZEP-15 | N/A | Create HiveContext instead of SqlContext in Livy interpreter |
| ZEP-16 | N/A | Zeppelin example notebooks are not visible |