Release Notes
Also available as:
PDF

Fixed Issues

Fixed issues represents selected issues that were previously logged via Hortonworks Support, but are now addressed in the current release. These issues may have been reported in previous versions within the Known Issues section; meaning they were reported by customers or identified by Hortonworks Quality Engineering team.

Incorrect Results

Hortonworks Bug IDApache JIRASummary
BUG-65370 HIVE-14652 HIVE-14652 : incorrect results for not in on partition columns
BUG-65908 HIVE-14595 Backport HIVE-14595: TimestampWritable::setTimestamp gives wrong result when 2nd VInt exists
BUG-66084 HIVE-14530 Backport HIVE-14530: Union All query returns incorrect results
BUG-66121 HIVE-14251 Backport HIVE-14251 (incorrect results)
BUG-66335 SQOOP-3014 Sqoop with HCatalog import loose precision for large numbers that does not fit into double
BUG-66624 HIVE-14519 Backport HIVE-14519: Multi insert query bug
BUG-66626 HIVE-7239 Backport HIVE-7239: Fix bug in HiveIndexedInputFormat implementation that causes incorrect query result when input backed by Sequence/RC files
BUG-67780 HIVE-14590 Backport HIVE-14590: CBO (Calcite Return Path) Incorrect result set when limit is present in one of the union branches
BUG-68306 HIVE-14959 Backport distinct + windowing related patches
BUG-68344 HIVE-14251, HIVE-14590 Backport HIVE-14251

Other

Hortonworks Bug IDApache JIRASummary
BUG-67349N/AApache Spark 2.0.1 Release

Performance

Hortonworks Bug IDApache JIRASummary
BUG-62071 HBASE-16675 HBase normalizer default heuristic result in too may splits
BUG-65355 HIVE-12077, HIVE-14693 Backport HIVE-12077: MSCK Repair table should fix partitions in batches
BUG-65379 RANGER-1145, RANGER-1146, RANGER-1161, RANGER-1162 Ranger policy engine optimizations
BUG-65711 HDFS-10569, HDFS-9484 NNThroughputBenchmark$BlockReportStats should not send empty block reports
BUG-66654 HADOOP-13436 Override equals and hashCode to avoid connection leakage
BUG-66876 OOZIE-2626 Oozie workflow processing becomes slow after the increase of rows in WF_JOBS and WF_ACTIONS tables when the metastore is running on SQL Server
BUG-68306 HIVE-14959 Backport distinct + windowing related patches
HIVE-850 HIVE-14589 WASB LLAP locality
HIVE-874 HIVE-14462, HIVE-14511 Port msck fixes to Erie

Potential Data Loss

Hortonworks Bug IDApache JIRASummary
BUG-66878 SPARK-16664 Include SPARK-16664 in HDP
BUG-68438 HIVE-13083 Writing HiveDecimal to ORC can wrongly suppress present stream

Query Failure

Hortonworks Bug IDApache JIRASummary
BUG-65399 HIVE-14702 NPE in LlapSignerImpl.serializeAndSign() when using spark-llap after HS2 idle for a few days
BUG-65476 HIVE-14400 Backport HIVE-14400: Handle concurrent insert with dynamic partition
BUG-65942 HIVE-14621 Backport HIVE-14621: LLAP: memory.mode = none has NPE
BUG-66258 HIVE-14715 Backport HIVE-14715: Hive throws NumberFormatException with query with Null value
BUG-68378 HIVE-15099 PTFOperator.PTFInvocation didn't properly reset the input partition
HIVE-836 HIVE-14698 Datanucleus MSSQLServerAdapter generates incorrect syntax for OFFSET-FETCH clause
HIVE-837 HIVE-14674 Create external table via beeline fails with Metaexception: Incorrect syntax near the keyword 'with'
HIVE-866 HIVE-14726 Delete from an ACID table throws IndexOutOfBoundsException
HIVE-897 HIVE-14783 Delete from an ACID table fails on setting hive.optimize.sort.dynamic.partition true
HIVE-905 HIVE-14819 HIVE2: select query using udfmax does not finish map job.

Security

Hortonworks Bug IDApache JIRASummary
BUG-65704 YARN-3982 Banned.users from container-executor.cfg are not actually banned
BUG-65717 RANGER-1100 When column-masking is set for an user, update operations should be denied
BUG-66193YARN-4591YARN Web UIs should provide a robots.txt
BUG-66764 RANGER-1170 Address keyadmin user restriction
BUG-67853 KNOX-762 Knox Dispatch to Service with Kerberos and Wire Encryption Fails
BUG-68901N/AHiveCli Exec Access Control is looking for permissions with wrong userID.
RANGER-38N/ARanger should be able to authenticate with Kerberos token

Stability

Hortonworks Bug IDApache JIRASummary
BUG-42602 HIVE-13090 Hive metastore crashes on NPE with ZooKeeperTokenStore
BUG-51037 ZOOKEEPER-1936 Regionservers exit for FATAL exception during initialization
BUG-65466 HADOOP-8751, HDFS-9276 Failed to Update HDFS Delegation Token for long running application in HA mode
BUG-65708 HDFS-10569 A bug causes OutOfIndex error in BlockListAsLongs
BUG-65709 HDFS-10569 A bug causes OutOfIndex error in BlockListAsLongs
BUG-66086 HIVE-14607 ORC split generation failed with exception: java.lang.ArrayIndexOutOfBoundsException: 1
BUG-66148 HIVE-14656 Clean up driver instance in get_splits
BUG-66206 HIVE-14591 HS2 is shut down unexpectedly during the startup time
BUG-66387 STORM-2042 Nimbus client connections not closed properly causing connection leaks
BUG-66439 HIVE-14774 Canceling query using Ctrl-C in beeline might lead to stale locks
BUG-66654 HADOOP-13436 Override equals and hashCode to avoid connection leakage
BUG-66666 HBASE-16721 Concurrency issue in WAL unflushed seqId tracking
BUG-66893 HIVE-14773 NPE aggregating column statistics for date column in partitioned table
BUG-67260 KNOX-754, KNOX-756 Knox regression in support of encoding for non-ascii chars in Location header
BUG-68060 STORM-2158 OutOfMemoryError in Nimbus' SimpleTransportPlugin
BUG-68261 HIVE-14648 LLAP: Avoid private pages in the SSD cache
HADOOP-1040 HADOOP-13403, HADOOP-13459, HADOOP-13513 Port WASB performance improvement from HADOOP 13403
HBASE-364N/AHMaster shuts down with FATAL error with 'error or interrupted while splitting logs in wasb'
HBASE-365N/AIntegrationTestRegionReplicaReplication fails with timeout in case of Humboldt cluster
HBASE-370N/AIntegrationTestTimeBoundedRequestsWithRegionReplicas failed with 'AssertionError: Verification failed with error code 1' due to read failures
HBASE-376 HBASE-16604 IntegrationTestBigLinkedList - Verify part failed with 'Found an undefined node. Undefined count=1553472'
HBASE-381N/AHBase major_compact action is failing
HBASE-409 HBASE-16824 Synchronization Issue between Syncer and Writer causing exception during Hbase Log rollover
HIVE-761 hive-12976 MetaStoreDirectSql doesn't batch IN lists in all cases
HIVE-877 HIVE-14624 Short name registration of LLAP broken on HDInsight clusters
HIVE-879 HIVE-14739 Deadlock while running Hive queries
HIVE-908 HIVE-14814 Hive2: Hive.loadDynamicPartitions (transactional )fails with Nullpointer Exception at hive.ql.exec.MoveTask. nul
MAPREDUCE-84N/AHistory Server is shutdown with NPE
OOZIE-330N/A'insert overwrite' query intermittently failed
OOZIE-332 HADOOP-12954 Oozie workflow start failed org.apache.oozie.action.ActionExecutorException: JA001: sometoken
PHOENIX-111 CALCITE-1458 HDP 2.5 Phoenix Query Server is broken
RANGER-51 RANGER-1173 RANGER_ADMIN install failure
SLIDER-20N/AtestAMKilledWithStateAMStartedAgentsStarted fails intermittently in ADLS
SPARK-172N/AHbaseSource, DataType and CompositeKey example fails with TableNotEnabledException: shcExampleTable
YARN-220N/ATimeline Service failed with " cause: java.io.IOException: /atshistory/active does not exist"

Supportability

Hortonworks Bug IDApache JIRASummary
BUG-65661 HIVE-13756 Map failure attempts to delete reducer _temporary directory on multi-query pig query
BUG-66667 HBASE-16678 MapReduce jobs do not update counters from ScanMetrics
BUG-67261 HIVE-14762, HIVE-14922 Backport logging related patches
HIVE-826 HIVE-13941 Propagate better error messages for schematool failures during Hive metastore start
STORM-121 STORM-2127 Update eventhub client dependency for Storm-Eventhub

Upgrade

Hortonworks Bug IDApache JIRASummary
BUG-66735 N/A Storm nimbus doesn't start after upgrading to Ambari 2.4.1
ZEP-18N/AZeppelin not able to install on Spark2

Usability

Hortonworks Bug IDApache JIRASummary
BUG-58753 PHOENIX-3360 Our default secondary index configuration is wrong for RPC controller
BUG-61383 HIVE-14600 LLAP daemon does not shutdown if registration fails
BUG-64511 HDFS-9618 Fix mismatch between log level and guard in BlockManager#computeRecoveryWorkForBlocks
BUG-65051 RANGER-1167 Page not found (404) after click on OK in admin create policy audit
BUG-65206 ATLAS-1140, ATLAS-1141 Atlas UI improvements & fix
BUG-65250 HDFS-10764 Customer's NN failed over with corrupted edits log, Secondary NN appears to fail to start with null pointer
BUG-65286 ATLAS-1147 UI: column name doesn't show up in schema tab for hive table
BUG-65384 PIG-4976 streaming job with store clause stuck if the script fail
BUG-65421 HIVE-13723 Executing join query on type Float using Thrift Serde will result in Float cast to Double error
BUG-65498 HIVE-14411 selecting Hive on Hbase table may cause FileNotFoundException
BUG-65520 HIVE-14690 Query fail when hive.exec.parallel=true, with conflicting session dir
BUG-65659 HIVE-14457 Partitions in encryption zone are still trashed though an exception is returned
BUG-65680 TEZ-3433 Tez UI: Searching using wrong ID causes error in all DAGs page
BUG-65704 YARN-3982 Banned.users from container-executor.cfg are not actually banned
BUG-65758 HIVE-14436 Error hit after enabling hive.optimize.skewjoin
BUG-65861 SQOOP-3033 Sqoop option --skip-dist-cache is not saved as a parameter when saving Sqoop Job
BUG-65901 HIVE-13185 orc.ReaderImp.ensureOrcFooter() method fails on small text files with IndexOutOfBoundsException
BUG-65980 SQOOP-2408, SQOOP-2582 Parquet file support for Sqoop for HDP 2.4.0
BUG-65981 SQOOP-2408, SQOOP-2582 Parquet file support for Sqoop for HDP 2.4.0
BUG-65983N/AIncrease the kite sdk version to 1.1 for fixing Sqoop parquet support issues
BUG-66065 ACCUMULO-4424, ACCUMULO-4456 Do not wait to start Thrift servers until lock is acquired
BUG-66087 HIVE-14697 Can not access kerberized HS2 Web UI
BUG-66096 RANGER-1175 Create policy fails for updated servicedef
BUG-66100 HADOOP-12457 fs.shell.Delete.java contains an invalid character that makes test_DFSCallsFromStreaming fail
BUG-66126 RANGER-1176 Ranger admin does not allow to create / update a policy with only delegate admin permission
BUG-66132 HIVE-14149 Hive not able to connect to S3
BUG-66359 SLIDER-1169 Slider not honoring zookeeper quorum values passed
BUG-66376 SQOOP-3003 Sqoop import fails to query with split-by/boundary-query using Oracle Date/Timestamp
BUG-66428 RANGER-1169 Global audit settings are not honored unless at least one ranger policy exists
BUG-66498 HIVE-14805 Subquery inside a view will have the object in the subquery as the direct input
BUG-66577 HIVE-11120 ORC output format creates 0 length files if no data is written
BUG-66666 HBASE-16721 Concurrency issue in WAL unflushed seqId tracking
BUG-66737 HIVE-14865 Fix comments after HIVE-14350
BUG-66784 ATLAS-1206 Atlas UI works in Firefox browser and NOT with IE or Chrome
BUG-66981 ATLAS-1199, ATLAS-1215 Atlas UI not loading after fresh build due to jquery-asBreadcrumbs plugin upgrade.
BUG-67042 ATLAS-1192 Atlas IE Support
BUG-67110 HIVE-14448 Queries with predicate fail when ETL split strategy is chosen for ACID tables - 2.5-maint
BUG-67120 HIVE-14659 OutputStream won't close if caught exception in funtion unparseExprForValuesClause in SemanticAnalyzer.java
BUG-67189 SQOOP-2995 Backward incompatibility introduced by Custom Tool options
BUG-67343 HDFS-9008 HDFS-9008 was not backported correctly
BUG-67742 SQOOP-2999 Sqoop ClassNotFoundException (org.apache.commons.lang3.StringUtils) is thrown when executing Oracle direct import map task
BUG-68436 HIVE-14933 LLAP may fail to start if Python dependencies like argparse are not there (erie)
OOZIE-329 OOZIE-2690 Oozie kill got NPE
PIG-222 PIG-4972, PIG-4976 Pig streaming yarn app is running forever.
RANGER-39 RANGER-1167 Ranger admin UI does not load in Internet Explorer
SPARK-170N/Acritical fixes in Spark2
SPARK-175 SPARK-17512 Specifying remote files for Python based Spark jobs in Yarn cluster mode not working
STORM-117N/AHDFS topologies are failing because of missing dependency
ZEP-1 ZEPPELIN-1400 Use relative path to the interpreter setting page
ZEP-15N/ACreate HiveContext instead of SqlContext in Livy interpreter
ZEP-16N/AZeppelin example notebooks are not visible