Fixed Issues
Fixed issues represents selected issues that were previously logged via Hortonworks Support, but are now addressed in the current release. These issues may have been reported in previous versions within the Known Issues section; meaning they were reported by customers or identified by Hortonworks Quality Engineering team.
Incorrect Results
Hortonworks Bug ID | Apache JIRA | Summary |
---|---|---|
BUG-108283 | RANGER-2169 | Create unique index on service and name column of x_policy table |
BUG-108346 | KNOX-1398 | WebHDFS and WebHBase APIs listed incorrectly as services that need whitelist enforcement |
BUG-108348 | KNOX-1392, KNOX-1393, KNOX-1395 | Remove use of InetAddress in Default Whitelist Creation |
BUG-108351 | KNOX-1386 | Fix NiFi service discovery |
BUG-108352 | KNOX-1385 | Whitelist test inconsistencies |
BUG-108802 | KNOX-1412 | Knox fails while executing a `curl` command using Content-Type: text/xml" "Content-Encoding: gzip" |
BUG-108882 | RANGER-2187 | External Group search fails on Ranger UI when installed with postgres |
BUG-109034 | RANGER-2114 | Backport RANGER-2114 on Atlantic-M05 |
BUG-109385 | HIVE-20406 | Nested Coalesce giving incorrect results |
BUG-109887 | HIVE-20187 | Backport HIVE-20187 |
BUG-95462 | HDFS-13205, RANGER-2183 | Incorrect path is passed to checkPermission during authorization of file under a snapshot (specifically under a subdir) after original subdir is deleted |
Other
Hortonworks Bug ID | Apache JIRA | Summary |
---|---|---|
BUG-102276 | HIVE-19739 | Use the checkpoint on table / partition to create relevant repl load tasks. |
BUG-102279 | HIVE-19739 | Repl load should not fail if Target Db already exists. |
BUG-103052 | HIVE-19564 | Vectorization: Backport HIVE-19564 to Atlantic-M05 --> Fix NULL / Wrong Results issues in Arithmetic Functions |
BUG-105695 | HIVE-20090 | TPCDS query64 has regressed in Atlantic by 5x times. |
BUG-105963 | HIVE-19408 | Show tables statement includes views and materialized views |
BUG-106249 | HIVE-20011 | Move away from append mode in proto logging hook |
BUG-106294 | HIVE-20028 | hive streaming fails with unable to abort invalid txns |
BUG-106359 | HIVE-4367 | enhance TRUNCATE syntex to drop data of external table |
BUG-107015 | N/A | TestReplicationScenariosAcidTables & TestReplicationOnHDFSEncryptedZones failing |
BUG-107018 | HIVE-19104 | TestHS2ImpersonationWithRemoteMS.testImpersonation failing |
BUG-107020 | HIVE-20299 | TestLlapSignerImpl.testSigning failing |
BUG-107023 | N/A | TestDynamicPartitionPruner.testSingleSourceMultipleFiltersOrdering1 failing with time out |
BUG-107024 | N/A | TestCustomPartitionVertex.testGetBytePayload failing due to timeout |
BUG-107025 | HIVE-19800 | TestUpgradeTool.testPostUpgrade failing |
BUG-107026 | N/A | TestWarehouseExternalDir.testManagedPaths failing |
BUG-107062 | N/A | TestSchemaToolForMetastore failing |
BUG-107135 | HIVE-20353 | Follow redirects when hive connects to a passive druid overlord/coordinator |
BUG-107180 | N/A | CLONE - UT error in storm: org.apache.storm.sql.hdfs.TestHdfsDataSourcesProvider |
BUG-107264 | N/A | TestMiniDruidCliDriver failing |
BUG-107300 | KNOX-1386 | Autodiscovery not using the correct NiFi properties |
BUG-107398 | HIVE-20151 | External table: exception while storing stats |
BUG-107516 | HIVE-19800 | PreUpgrade Tool: Support upgrade from >= 2.6.0 |
BUG-107547 | HIVE-20135 | Backport HIVE-20135: Fix incompatible change in TimestampColumnVector to default to UTC |
BUG-107548 | HIVE-20090 | Backport HIVE-20090: Extend creation of semijoin reduction filters to be able to discover new opportunities |
BUG-107553 | N/A | Vectorization: Backport HIVE-19992 to Atlantic-M05: Follow-on to HIVE-19951 --> add call to SchemaEvolution.isOnlyImplicitConversion to disable encoded LLAP I/O for ORC only when data type conversion is not implicit |
BUG-107572 | N/A | Vectorization: Backport HIVE-20174 to Atlantic-M05: Fix NULL / Wrong Results issues in GROUP BY Aggregation Functions |
BUG-107694 | HIVE-20098 | Backport HIVE-20098: Statistics: NPE when getting Date column partition statistics |
BUG-107695 | HIVE-20152 | Backport HIVE-20152: reset db state, when repl dump fails, so rename table can be done |
BUG-107713 | N/A | Vectorization: Backport HIVE-20197 to Atlantic-M05: Add DECIMAL_64 testing, add Date/Interval/Timestamp arithmetic, and add more GROUP BY Aggregation tests |
BUG-107768 | HIVE-19360 | backport: CBO: Add an "optimizedSQL" to QueryPlan object |
BUG-107866 | ATLAS-2786 | Atlas HA fails to use all the zookeeper in the environment |
BUG-108021 | HIVE-20383 | provide the correct queue name to proto logger |
BUG-108080 | HIVE-20172 | Backport HIVE-20172: StatsUpdater failed with GSS Exception while trying to connect to remote metastore |
BUG-108109 | N/A | Disable flaky spark Tests |
BUG-108201 | HIVE-20207 | Vectorization: Backport HIVE-20207 to Atlantic-M05: Fix NULL / Wrong Results issues in Filter / Compare |
BUG-108216 | HIVE-20192 | HS2 with embedded metastore is leaking JDOPersistenceManager objects. |
BUG-108263 | N/A | Fix TestJdbcWithDBTokenStore.testTokenAuth, TestJdbcWithDBTokenStoreNoDoAs.testTokenAuth failures |
BUG-108265 | HIVE-20212 | Backport HIVE-20212: Hiveserver2 in http mode emitting metric default.General.open_connections incorrectly |
BUG-108287 | HIVE-20383 | Missing pre-hook and duplicate post hook events from hive proto events hook. |
BUG-108345 | HIVE-20245 | Vectorization: Backport HIVE-20245 to Atlantic-M05: Fix NULL / Wrong Results issues in BETWEEN / IN |
BUG-108357 | HIVE-20082, HIVE-20204 | Decimal truncation for trailing zeros - port to 3.0-maint |
BUG-108424 | HIVE-20033 | PowerBI unable to connect to LLAP cluster with large Databases/Tables |
BUG-108439 | N/A | TestJdbcWithMiniLlapRow is failing on ptest |
BUG-108440 | N/A | TestHiveDruidQueryBasedInputFormat is failing on ptest |
BUG-108444 | HIVE-18201 | Subquery joins result in cross product instead of broadcast |
BUG-108511 | HIVE-20263 | Backport HIVE-20263: Typo in HiveReduceExpressionsWithStatsRule variable |
BUG-108512 | HIVE-20241 | Backport HIVE-20241: Support partitioning spec in CTAS statements |
BUG-108513 | HIVE-14493 | Backport HIVE-14493: Partitioning support for materialized views |
BUG-108558 | HIVE-20281 | Backport HIVE-20281: SharedWorkOptimizer fails with 'operator cache contents and actual plan differ' |
BUG-108566 | N/A | TPCDS query23 runs longer on hdp3.0 with varying runtimes. |
BUG-108573 | KNOX-1415 | Upgrade Jetty to 9.2.25 |
BUG-108586 | N/A | Vectorization: Backport HIVE-20294 to Atlantic-M05: Fix NULL / Wrong Results issues in COALESCE / ELT |
BUG-108642 | N/A | Update golden files for TezDriver |
BUG-108643 | HIVE-20302 | Backport HIVE-20302: LLAP: non-vectorized execution in IO ignores virtual columns, including ROW__ID |
BUG-108644 | HIVE-20314 | Backport HIVE-20314: Include partition pruning in materialized view rewriting |
BUG-108645 | HIVE-20301 | Backport HIVE-20301: Enable vectorization for materialized view rewriting tests |
BUG-108719 | HIVE-20322 | FlakyTest: TestMiniDruidCliDriver |
BUG-108720 | N/A | Add missing Apache Hadoop 3.1.1 commits to Atlantic M05 (YARN/MR) |
BUG-108816 | HIVE-20383 | Hive: ProtoLoggingHook NPE on queue name field |
BUG-108894 | HIVE-20336 | Backport HIVE-20336: Masking and filtering policies for materialized views |
BUG-108910 | N/A | Tez History API fails with ClassCastException |
BUG-108917 | N/A | Backport Druid perf enhacements |
BUG-108999 | HIVE-20347 | Backport HIVE-20347: hive.optimize.sort.dynamic.partition should work with partitioned CTAS and MV |
BUG-109060 | HIVE-20364 | Update default for hive.map.aggr.hash.min.reduction |
BUG-109168 | N/A | TestHiveCli timing out (need HIVE-20149 backport) |
BUG-109173 | N/A | Backport CALCITE-2465: Enable use of materialized views for any planner |
BUG-109245 | HIVE-20379 | Backport HIVE-20379: Rewriting with partitioned materialized views may reference wrong column |
BUG-109247 | HIVE-20382 | Backport HIVE-20382: Materialized views: Introduce heuristic to favour incremental rebuild |
BUG-109272 | HIVE-20345 | Backport HIVE-20345: Drop database may hang if the tables get deleted from a different call |
BUG-109311 | HIVE-20391 | Backport HIVE-20391: HiveAggregateReduceFunctionsRule may infer wrong return type when decomposing aggregate function |
BUG-109313 | N/A | Backport HIVE-17979 : Tez: Improve ReduceRecordSource passDownKey copying |
BUG-109319 | N/A | Backport HIVE-20177: Vectorization: Reduce KeyWrapper allocation in GroupBy Streaming mode |
BUG-109321 | N/A | Backport HIVE-20153: Count and Sum UDF consume more memory in Hive 2+ |
BUG-109703 | RANGER-2167 | Upgrade to Apache parent pom version 20 |
BUG-110109 | N/A | Remove unidecode from superset |
BUG-110124 | YARN-8726 | YARN UI2 is not accessible |
BUG-110428 | HIVE-20450 | Add replication test for LOAD command on ACID table. |
BUG-89982 | HIVE-20357 | Forward port azure db scripts to atlantic |
BUG-92690 | HIVE-19267 | Generate new event for ACID Write operations |
BUG-92700 | HIVE-19340 | Disable timeout of transactions opened by replication task at target cluster |
BUG-95678 | HIVE-17683 | Add explain locks <sql> command |
BUG-98176 | HIVE-20337 | TPCDS query16 compilation fails on cn105 when CachedStore is used. ObjectStore works fine. |
BUG-99461 | HIVE-20337 | Queries returning incorrect results using CachedStore. ObjectStore works fine. |
BUG-110970 | N/A |
For long-running SparkSQL jobs on a Kerberized cluster, some JDBC clients may randomly fail with a "no token found in cache" error after the delegation token expiry period. |
Performance
Hortonworks Bug ID | Apache JIRA | Summary |
---|---|---|
BUG-105965 | N/A | Adding contraints on TPCDS schema, changes query plan for query93 and degrades its perf by 4x times |
BUG-107255 | HIVE-20209 | Metastore connection fail 1st attempt in repl dump |
BUG-107475 | RANGER-2162 | Rangeradmin stuck with db connection issue. |
BUG-107932 | RANGER-2158 | [Atlantic-M05] - RANGER-2158 Performance improvement for REST API calls |
BUG-107990 | HIVE-20300 | VectorFileSinkArrowOperator |
BUG-108108 | HADOOP-15598 | DataChecksum calculate checksum is contented on hashtable synchronization |
BUG-108281 | RANGER-2165 | Address JPA Cache issue when policies Create, Update and Delete are done via REST API in Apache Ranger admin |
BUG-108282 | RANGER-2168 | Add service admin user through service config |
BUG-108401 | RANGER-2173 | RANGER-2173: Optimize Trie construction and Policy lookup |
BUG-108422 | HADOOP-15547 | WASB: improve listStatus performance |
BUG-108515 | HIVE-20290 | Lazy initialize ArrowColumnarBatchSerDe so it doesn't allocate buffers during GetSplits |
BUG-108638 | HIVE-20316 | Skip external table file listing for create table event |
BUG-108742 | HIVE-18624 | Hive parser is very slow in parsing nested expressions |
BUG-109693 | RANGER-2188 | Support multiple threads to build Trie and on-lookup post-setup for Trie nodes |
BUG-109694 | RANGER-2191 | Update ranger-tool with new options to control Trie |
BUG-109702 | RANGER-2186 | Increment service-specific policy and tag versions after… |
BUG-109720 | RANGER-2196 | Ensure that any explicit threads used by Ranger are marked as daemon threads |
Potential Data Loss
Hortonworks Bug ID | Apache JIRA | Summary |
---|---|---|
BUG-104681 | HIVE-19891 | insert from select overwrites the partition location path |
Security
Hortonworks Bug ID | Apache JIRA | Summary |
---|---|---|
BUG-106840 | N/A | CVE-2018-8009: Zip Slip Vulnerability - Apache Hadoop distributed cache archive vulnerability |
BUG-107549 | N/A | druid bolt fails with NoSuchMethodError |
BUG-107610 | ATLAS-2788 | Add policy permissions for tagsync user |
BUG-107863 | STORM-3052 | Backport STORM-3052: Let blobs un archive (HDP 3.0-maint) |
BUG-108451 | KNOX-1416 | Upgrade Spring Framework to 4.3.17 |
BUG-108574 | N/A | Upgrade Bouncy Castle Dependency from Pac4J |
BUG-108749 | ZEPPELIN-3686 | Update org.eclipse.jetty version to 9.4.11.v20180605 to avoid CVE issues |
BUG-108874 | N/A | Upgrade of Jackson Databind to 2.9.5 in Oozie |
BUG-108997 | N/A | BlackDuck for HWC |
BUG-109155 | N/A | Address Storm black-duck scan issues |
BUG-109209 | ATLAS-2821 | Update old atlas-simple-authz-policy.json (m0) file with relationship permission attributes |
Stability
Hortonworks Bug ID | Apache JIRA | Summary |
---|---|---|
BUG-101700 | SPARK-23007 | Add read schema suite for file-based data sources |
BUG-102162 | YARN-8595 | [UI2] Container diagnostic missing from container page |
BUG-102349 | YARN-8593, YARN-8594 | [UI2] Show the current logged in user in UI2 |
BUG-102389 | YARN-8620 | [UI2] YARN Services UI new submission failures are not debuggable |
BUG-102493 | YARN-8608 | [UI2] No information available per application appAttempt about 'Total Outstanding Resource Requests' |
BUG-103056 | N/A | [YARN-8301] Add documentation for yarn service upgrade |
BUG-104220 | RANGER-2150 | Unit test coverage for org.apache.ranger.biz.XUserMgr and UserMgr |
BUG-104727 | N/A | [YARN-8403] NM log should print Exception at ERROR log level |
BUG-104844 | YARN-8407 | [YARN-8407] Container launch exception in AM log should be printed in ERROR level |
BUG-105493 | YARN-8429 | [YARN-8429] Improve diagnostic message when artifact is not set properly |
BUG-106203 | YARN-8603 | [UI2] Latest run application should be listed first in the RM UI |
BUG-106504 | N/A | Update `spark2:atlantic` with Spark 2.3.2 RC5 |
BUG-106619 | N/A | Zengine UT failure: IllegalStateException: Queue configuration missing child queue names for root |
BUG-107236 | ATLAS-2793,, ATLAS-2794, RANGER-2166 | Tagsync start fails with CNF error for org/codehaus/jackson/xc/JaxbAnnotationIntrospector when using Atlas-REST as source |
BUG-107241 | HIVE-19860 | HiveServer2 ObjectInspectorFactory memory leak with cachedUnionStructObjectInspector |
BUG-107347 | N/A | Zeppelin, set right URL for jdbc for different transport modes |
BUG-107413 | SPARK-23679 | Spark Web UI in master=yarn mode does not proxy/redirect with YARN HA (secured) |
BUG-107477 | YARN-8418 | [YARN-8418] App local logs could leaked if log aggregation fails to initialize for the app |
BUG-107576 | HIVE-20069 | Backport HIVE-20069: Fix reoptimization in case of DPP and Semijoin optimization |
BUG-107632 | N/A | [YARN-8545] YARN native service should return container if launch failed |
BUG-107700 | HIVE-20203 | Arrow SerDe leaks a DirectByteBuffer |
BUG-107725 | N/A | Hive warehouse directory creation issue at blueprint cluster creation |
BUG-107871 | N/A | [Service Discovery] Add Support for discovering beacon service |
BUG-107922 | HADOOP-15593 | Backport HADOOP-15593 to Atlantic-M05 |
BUG-107945 | RANGER-2160 | 'Email Address' search is not working in user listing page,userRoles filters also needs to be improved. |
BUG-107955 | YARN-8571 | op/cli commands to a running service is failing in a secure cluster |
BUG-107965 | RANGER-2164 | Ranger to add default altlas policy for rangertagsync user. |
BUG-108112 | N/A | backport HIVE-19935 (race in WM resulting in a session dying) |
BUG-108146 | RANGER-2182 | Handle upgrade scenario since atlas-service def is added with new resources for relationship |
BUG-108209 | N/A | [ATSv2] [YARN-8591] NPE error while checking for entity acl in non-secure cluster |
BUG-108210 | YARN-8592 | [UI2] Resource Manager Web UI V2 shows a blank page in windows OS And Chrome browser |
BUG-108283 | RANGER-2169 | Create unique index on service and name column of x_policy table |
BUG-108349 | KNOX-1391 | Fix IllegalStateException when accessing Admin UI |
BUG-108350 | KNOX-1387, KNOX-1390 | Fix NPE in AclsAuthorizationFilter |
BUG-108354 | HIVE-20244, HIVE-20247, HIVE-20249 | backport LLAP error handling changes |
BUG-108407 | RANGER-2171 | RANGER-2171: Unit Test cases to cover policy operations from service admin user |
BUG-108583 | N/A | Issue with tutorial notebooks |
BUG-108624 | N/A | ATSv2 HBase Application does not appear when HBase app is stopped [System Service Mode] |
BUG-108632 | HIVE-20312 | HWC: Assign Arrow ChildAllocator per-task to fail-fast on memory leak |
BUG-108762 | SPARK-24948 | Spark History Server filters some applications on ADL |
BUG-108833 | N/A | Upgrade bouncy castle version to 1.6.0 to fix CVE issue |
BUG-108915 | N/A | Upgrade Jetty version to fix CVE issue in Spark |
BUG-109034 | RANGER-2114 | Backport RANGER-2114 on Atlantic-M05 |
BUG-109183 | N/A | [YARN-8667] Container Relaunch fails with "find: File system loop detected;" |
BUG-109184 | N/A | Add yarn.service.container-failure.retry.max for ats-hbase app. |
BUG-109195 | SPARK-25051 | Correctness issue with outer joins for Spark 2.3 |
BUG-109198 | N/A | Provide work around handle BUG-108844 impact on YARN |
BUG-109312 | HIVE-20393 | Semijoin Reduction : markSemiJoinForDPP behaves inconsistently |
BUG-109384 | N/A | [YARN-8679] Gridmix of HDP-3.0 is stuck |
BUG-109581 | ZEPPELIN-3741 | Do not clear "Authorization" header if Z-server is running behind proxy |
BUG-109697 | RANGER-2180 | Handle token replacement correctly when token is not defined in the request context |
BUG-109720 | RANGER-2196 | Ensure that any explicit threads used by Ranger are marked as daemon threads |
BUG-109777 | RANGER-2197 | Delegate Admin is not able to create policy |
Supportability
Hortonworks Bug ID | Apache JIRA | Summary |
---|---|---|
BUG-107437 | ZOOKEEPER-2662, ZOOKEEPER-3019 | Port JMX metrics for Zookeeper I/O to Atlantic-Next |
BUG-109698 | RANGER-2152 | Incorrect debugging information in RangerPluginClassLoader.java |
BUG-109700 | RANGER-2177 | Handle validations for duplicate configuration item during service create/edit |
BUG-109701 | RANGER-2153 | Supply the function of reverting policy history version. |
BUG-30983 | HIVE-14898 | HiveServer2 error message from knox in logs - benign error confuses users |
Upgrade
Hortonworks Bug ID | Apache JIRA | Summary |
---|---|---|
BUG-108146 | RANGER-2182 | Handle upgrade scenario since atlas-service def is added with new resources for relationship |
BUG-109209 | ATLAS-2821 | Update old atlas-simple-authz-policy.json (m0) file with relationship permission attributes |
BUG-109696 | RANGER-2184 | Update RangerAtlas authorization to authorize add/update/remove of relationships |
BUG-109728 | N/A | phoenix-sqlline launch fails on upgraded cluster |
Usability
Hortonworks Bug ID | Apache JIRA | Summary |
---|---|---|
BUG-103643 | RANGER-2161 | it will be good to mark exclude from deny and allow condition with some background color |
BUG-104455 | HIVE-19694 | create materialized view should validate if view already exists |
BUG-106469 | HIVE-20044 | [SPARK_LLAP] fixed char type output via spark + HDWC is inconsistent with that via HS2 beeline |
BUG-106681 | OOZIE-3328 | Reenable hive action in Oozie |
BUG-106688 | HIVE-20341 | Druid Needs Explicit CASTs from Timestamp to STRING when the output of timestamp function is used as String |
BUG-106818 | ATLAS-2785 | import_hive.sh with "-t" option imports all metadata instead of just that table. |
BUG-106266 | OOZIE-2769 OOZIE-3085OOZIE-3156OOZIE-3183 | |
BUG-107236 | ATLAS-2793,, ATLAS-2794, RANGER-2166 | Tagsync start fails with CNF error for org/codehaus/jackson/xc/JaxbAnnotationIntrospector when using Atlas-REST as source |
BUG-107241 | HIVE-19860 | HiveServer2 ObjectInspectorFactory memory leak with cachedUnionStructObjectInspector |
BUG-107742 | NA | Update some Ambari defaults based on 3.0-EA |
BUG-107945 | RANGER-2160 | 'Email Address' search is not working in user listing page,userRoles filters also needs to be improved. |
BUG-107965 | RANGER-2164 | Ranger to add default altlas policy for rangertagsync user. |
BUG-108114 | N/A | Backport HIVE-20240 |
BUG-108282 | RANGER-2168 | Add service admin user through service config |
BUG-108312 | N/A | Oozie doesn't work out of the box with Spark2 |
BUG-108347 | KNOX-1383, KNOX-1396, KNOX-1397 | Admin UI Fixes for Topology List Refresh and URL Validation |
BUG-108630 | HIVE-20277 | Vectorization: Backport HIVE-20277 to Atlantic-M05: Vectorization: Case expressions that return BOOLEAN are not supported for FILTER |
BUG-108738 | N/A | HWC: Python API is missing "executeUpdate" |
BUG-108882 | RANGER-2187 | External Group search fails on Ranger UI when installed with postgres |
BUG-109034 | RANGER-2114 | Backport RANGER-2114 on Atlantic-M05 |
BUG-109290 | N/A | HWC: HiveWarehouseSession.table(<tableName>) doesn't accept unqualified table name |
BUG-109309 | N/A | HWC: HiveWarehouseSession.dropTable API is broken |
BUG-109698 | RANGER-2152 | Incorrect debugging information in RangerPluginClassLoader.java |
BUG-109700 | RANGER-2177 | Handle validations for duplicate configuration item during service create/edit |
BUG-109701 | RANGER-2153 | Supply the function of reverting policy history version. |
BUG-109777 | RANGER-2197 | Delegate Admin is not able to create policy |
BUG-109784 | N/A | Misleading documentation to enable SSL on Spark2 History Server |
BUG-111169 | N/A | Oozie hive spark jobs failing because spark still use hcat type delegation tokens |
BUG-95269 | YARN-8318 | [UI2] Few Missing information on Component Instance Information page |
BUG-95462 | HDFS-13205, RANGER-2183 | Incorrect path is passed to checkPermission during authorization of file under a snapshot (specifically under a subdir) after original subdir is deleted |
BUG-95678 | HIVE-17683 | Add explain locks <sql> command |