9. Known Issues for HDP 2.2.4

Hortonworks Bug ID

Apache JIRA

Component/s

Summary

BUG-18450

HIVE-7160

Hive

Vectorization Udf: GenericUDFConcat, is not supported for String+Long

BUG-19255

 

Zookeeper

Uninstalling the ZooKeeper RPM does not remove all ZooKeeper directories

BUG-19911

 

Falcon

Ensure falcon windows installer takes secure cluster installation into account

BUG-19912

 

Falcon

Document how to configure Falcon on Kerberized HDP for Windows

BUG-19930

 

Falcon

Document entity management dashboard

BUG-20856

 

Hive

Column names are missing from join expression in Map join with CBO enabled

BUG-20866

 

Mahout

Mahout trainnb command fails with -c option

BUG-21166

 

Phoenix

testGetCaseInsensitiveDynCol fails intermittently throwing AssertionError

BUG-21254

 

HBase

hbck tool fixMeta flag not able to recreate deleted entries from hbase:meta table when just one column is deleted

BUG-21468

 

Hive

With CBO enabled queries fail with Failed to breakup Windowing invocations into Groups

BUG-22522

 

YARN

MR Job failed when run via oozie and NN and RM were being killed

BUG-22807

 

YARN

Non-secure slider app processes run with yarn user.

BUG-22841

 

HBase

HBase test failed with Insufficient permissions (user=hrt_qa@HWQE.HORTONWORKS.COM error

BUG-23260

 

Hive

BUG-22878 insert after alter table issue

BUG-23527

 

Oozie

Oozie config referred to versioned hadoop config

BUG-23590

HIVE-8247

HCatalog

Pig cursor written to Hive via HCat doesn't NULL-fill missing columns

BUG-23619

 

HBase

On Windows, tests fail complaining about Cannot run program /usr/bin/env

BUG-23655

 

Oozie

Oozie stuck on mapreduce job because workflow state isn't changing from RUNNING to KILLED

BUG-23820

 

Accumulo

BulkFileIT-testBulkFile fails due to timeout.

BUG-23830

HIVE-8263

Hive

CBO : Query query has date_dim d1,date_dim d2 and date_dim d3 but the explain has d1, d1 and d1

BUG-23874

 

Hive

Map join sizing done by auto.convert .join.noconditional task.size doesn't take into account Hash table overhead and results in OOM

BUG-23889

 

Knox

Windows security scripting should setup Knox to use ActiveDirectory for LDAP

BUG-23897

 

HBase

HBase REST server continuously restarting in secure windows env

BUG-23981

 

HBase

IOException: Invalid HFile block magic in Windows unit tests

BUG-24118

 

Oozie

HDP Windows: Update/clean up oozie's SQL Server JDBC driver copies

BUG-24136

 

Falcon

Special/utf8 chars should not be allowed in pipeline name

BUG-24517

 

Oozie

oozie needs to support Application Tag

BUG-24518

 

Tez

Tez needs to support Application Tag

BUG-24589

 

Falcon

DR - set up replication for HDFS

BUG-24590

 

Falcon

DR - set up replication for Table

BUG-24591

 

Falcon

BUG-24589 UI Mock for setting up HDFS Replication

BUG-24592

 

Falcon

BUG-24589 Rest API design for HDFS replication

BUG-24594

 

Falcon

BUG-24590 UI Mock for setup table replication

BUG-24595

 

Falcon

BUG-24590 Rest API design for set up table replication

BUG-24597

 

Falcon

DR - delete replication for HDFS

BUG-24598

 

Falcon

DR - delete replication for Table

BUG-24831

 

Tez

TEZ windows test framework needs to respect ${fs.defaultFS} in tez configs.

BUG-25049

 

Knox

Two WebHdfsHaFuncTest are unstable and should be fixed and enabled

BUG-25058

HIVE-8459

Hive

DbLockManager reader locks are locking tables in addition to partitions

BUG-25064

 

Hive

Hive : Q51 fails in CBO with "Failed to breakup Windowing invocations into Groups. At least 1 group must only depend on input columns"

BUG-25194

 

Tez

AM running out of memory for partitioned bucketed tables

BUG-25282

 

Sqoop

l .ArrayIndex OutOfBounds Exception using --map-column-hive

BUG-25352

 

HDFS

Exit code 0 when command (put/copying a file bigger than quota) fails.

BUG-25380

 

Tez

For hive to work with Virtual columns and parittion names, we need information available within a record-reader

BUG-25424

 

Hive

When table is renamed stats are lost as changes are not propagated to metastore tables TAB_COL_STATS and PART_COL_STATS

BUG-25450

 

Mahout

Integrate Mahout in RU Automation Frame work

BUG-25451

 

Accumulo

[Moved to Dal] Integrate Accumulo in RU Automation Frame work

BUG-25460

 

Storm

Integrate storm-slider in RU automation framework

BUG-25961

 

HBase

HBase tests failing with org.apache. hadoop.hbase.NotAllMeta RegionsOnline Exception: Timed out; 10000ms

BUG-26243

 

Hive

CBO : Q51 fails in explain with Failed to breakup Windowing invocations into Groups.

BUG-26307

 

Falcon

As a user, I need to be able to replicate the action of creating new database at the target Hive DB

BUG-26309

 

Falcon

As a user, I need to be able to replicate the action of dropping a database at the target Hive DB

BUG-26310

 

Falcon

As a user, I need to be able to replicate the action of altering database at the target Hive DB

BUG-26311

 

Falcon

As a user, I need to be able to replicate the action of creating new table at the target Hive DB

BUG-26312

 

Falcon

As a user, I need to be able to replicate the action of dropping a table at the target Hive DB

BUG-26313

 

Falcon

As a user, I need to be able to replicate the action of altering a table at the target Hive DB

BUG-26314

 

Falcon

As a user, I need to be able to replicate the action of inserting rows into unpartitioned table at the target Hive DB

BUG-26315

 

Falcon

As a user, I need to be able to replicate the action of inserting and overwriting rows into unpartitioned table at the target Hive DB

BUG-26316

 

Falcon

As a user, I need to be able to replicate the action of adding a partition into a table at the target Hive DB

BUG-26317

 

Falcon

As a user, I need to be able to replicate the action of dropping a partition from a table at the target Hive DB

BUG-26318

 

Falcon

As a user, I need to be able to replicate the action of altering a partition in a table at the target Hive DB

BUG-26319

 

Falcon

As a user, I need to be able to replicate the action of inserting rows into a partition of a table at the target Hive DB

BUG-26320

 

Falcon

As a user, I need to be able to replicate the action of inserting and overwriting rows into a partition of a table at the target Hive DB

BUG-26322

 

Falcon

As a user, I need to be able to replicate Date partitions at the target Hive DB

BUG-26323

 

Falcon

As a user, I need to be able to replicate Site partitions at the target Hive DB

BUG-26324

 

Falcon

As a user, I need to be able to replicate Regex partitions at the target Hive DB

BUG-26325

 

Falcon

As a user, I need to be able to replicate custom non standard partitions at the target Hive DB

BUG-26326

 

Falcon

As a user, I need to be able to replicate multiple tables at once in a database at the target Hive DB

BUG-26327

 

Falcon

As a user, I need to be able to replicate multiple table names (defined by regular expressions) at the target Hive DB

BUG-26328

 

Falcon

As a user, I need to be able to replicate Avro partitions at the target Hive DB

BUG-26329

 

Falcon

As a user, I need to be able to replicate dynamic partitions at the target Hive DB

BUG-26330

 

Falcon

As a user, I need to be able to replicate hive roles at the target Hive DB

BUG-26419

 

Tez

Tez application fails with diff client and AM version

BUG-26428

 

Ranger

XaAdmin and XaAgent jobs for Debian failing on Unix clusters.

BUG-26661

 

Phoenix

Windows runs failed with "region isn't co-located with its data" exception

BUG-26766

 

Oozie

[Rolling Upgrade] Allow Prepare Oozie.war to be done without stopping Oozie Server

BUG-26875

 

YARN

Sometimes NodeManager keeps in tracking non-exitant completed container process

BUG-26907

HADOOP-11274

YARN

[HADOOP-11274] Concurrent Modification ThanException in Configuration Copy Constructor

BUG-26944

 

Falcon, Oozie

E0803: IO error, null during scheduling feed on a cluster with wire enc on

BUG-27089

 

Knox

Knox YarnSysTest skipped in secure mode for gsInstaller test config

BUG-27091

 

Knox

Knox YarnSysTest skipped in secure mode for ambari test config

BUG-27094

 

Knox

Knox HBase tests skipped for ambari test config

BUG-27103

 

Accumulo

replication.ReplicationIT-replication RecordsAreClosedAfterGarbageCollection failed due to timeout.

BUG-27194

 

YARN

[YARN-2821] Failures encountered on Distributed Shell on windows

BUG-27256

 

HBase

Create mr based Hbase Long running application for Rollingupgrade

BUG-27479

 

Falcon

falcon idempotency is being voilated

BUG-27507

 

Hive

Hive CLI returns 0 even in case of failures

BUG-27509

HBASE-12464

HBase

test_Replica CopyTable _verifyCopy Table[tableone] fails after trying several times for 'Failed to find location, tableName=hbase:meta, row='

BUG-27513

 

Storm

Storm-slider default config has ganglia error in supervisor.log

BUG-27549

 

Knox

Rolling upgrade: knox - webHbase smoke test failed after upgrading core components

BUG-27562

 

Tez

WordCount on Tez with non-nightly failed with NPE

BUG-27582

 

Hive

Beeline Client Connections Errors - Obscure errors and exception swallow - 1

BUG-27590

 

YARN

Distributed shell job does not execute python script in Windows

BUG-27592

 

Hive

Beeline Client Connections Errors - Obscure errors and exception swallow - 4

BUG-27632

 

Zookeeper

Add feature of retry in zookeeper client

BUG-27636

 

Hive

Oracle: intemittent acid_concurrency test failures due to NoSuchLockException

BUG-27637

 

Hive

Postgres: intermittent test failure due to "ERROR: could not serialize access due to read/write dependencies among transactions"

BUG-27638

 

Falcon

DEV - As a user i need a login page when i run Falcon as a standalone app

BUG-27639

 

Falcon

DEV - As a user, i need a way to logout when logged in the standalone version of Falcon

BUG-27656

HBASE-12472, HBASE-13192

HBase

IntegrationTestBulkLoad fails on mapreduce

BUG-27708

 

Falcon

DEV - When uploading an XML of a cluster that already exists (same name) the system allows the operation

BUG-27734

 

Falcon

DEV - Column titles should have 14px size

BUG-27737

 

Falcon

DEV - When loading the dashboard, the systen should show a spinning loader for tables that are updating, and just a plain message for columns without entities

BUG-27756

 

HBase

test_Integration TestMulti GetRegion Replicas slowness

BUG-27769

 

Sqoop

[Rolling upgrade] [Test issue] Sqoop Long running job finishes before upgrade of all component finishes

BUG-27802

 

Knox

Improve reliability of Knox smoke test in system test deployment

BUG-27858

 

HBase

HBase HA tests fail with Table Namespace Manager not ready yet

BUG-27893

 

Phoenix

testCurrentDate and testCurrentTime are failing due to NTP on windows server

BUG-27906

 

Knox

Increase timeout of system test WebHCatDslSysTest-testPig

BUG-27927

 

Falcon

DEV - Correct the steps information

BUG-27928

 

Falcon

DEV - The form needs a "Target Clusters" title and "add cluster" button in step 4 of feed setup

BUG-27929

 

Falcon

DEV - Put a label before the input and add archive option in step 4 of feed setup

BUG-27933

 

Falcon

DEV - Each cluster or input container must have it's title

BUG-27936

 

Falcon

DEV - Correct vertical spacings between labels and inputs in all setups

BUG-27971

HDFS-7558

HDFS

HDFS daemons JVM pauses when the NFS client and server on the same host as other HDFS daemons

BUG-28011

 

Phoenix

testInnerJoinWithPreFilters and testJoinOverSaltedTables are failing with SequenceNotFoundException exception

BUG-28031

 

Knox

Knox rolling upgrade system test support for wire encryption

BUG-28039

 

Knox

Knox Failed Authentication Auditing

BUG-28049

 

Falcon, RelEng

lineage is not enabled for Falcon in MSI

BUG-28080

FALCON-885

Falcon

Merge falcon-885 to internal repo for Champlain-M10

BUG-28097

HIVE-8915

Hive

Log file explosion due to non-existence of COMPACTION_QUEUE table

BUG-28218

 

Hive

Hive CLI hangs

BUG-28276

 

Falcon

Falcon actions intermittently fail with java.net.Socket TimeoutException on windows

BUG-28344

 

Falcon

DEV - XML editor should only be able to change it's vertical size, not horizontal

BUG-28348

 

Falcon

DEV - Increase vertical spacing in the space for the action buttons

BUG-28372

 

Sqoop

With Teradata connector all Sqoop-Hive(hcat) tests failing with NoClassDefFoundError: org/apache/hcatalog/mapreduce/OutputJobInfo

BUG-28394

 

Falcon

DEV - Correct font sizes in forms

BUG-28427

 

Knox

NPE in ambari knox install

BUG-28614

 

Oozie

Oozie property oozie.authentication.kerberos.principal is being used for authentication of webhdfs endpoint

BUG-28664

 

Falcon

DEV - Fix layout for user generated inputs

BUG-28977

FALCON-959

Falcon

Unable to update feed's acl to a different user of different group

BUG-29261

 

Phoenix

Tests are failing with ERROR 1200 and then raising an error due to jobManager rejection

BUG-29279

 

YARN

Distributedshell App fails after unregistering the application

BUG-29287

 

Falcon

Backport https://issues.apache.org/jira/browse/FALCON-829

BUG-29427

 

Hive

Turn off Parquet Vectorization until all data types work: DECIMAL, DATE, TIMESTAMP, CHAR, and VARCHAR

BUG-29730

 

Storm

Storm Hive,Hbase, Hadoop connector should support user specified UGI in non-secure env

BUG-29864

FALCON-1101

Falcon

Cluster submission in falcon does not create an owned-by edge

BUG-29903

 

HBase

IntegrationTestMTTR slowness

BUG-30022

HIVE-7693

Hive

Invalid column ref error in order by when using column alias in select clause and using having

BUG-30055

 

Falcon

Falcon recipe should have rest api

BUG-30059

 

HBase

In case of secure HBase slider, hbase shell's grant command throws 'ERROR: DISABLED: Security features are not available’

BUG-30784

ZOOKEEPER-1952

Zookeeper

zookeeper.log.file property is not respected; log output goes only to the zookeeper.out

BUG-30984

 

Hive

Hive compaction tests failing

BUG-31349

 

Oozie

oozie job in running state is causing intermittent timeout failures

BUG-31353

 

HBase

IntegrationTest RegionReplica Perf region not serving

BUG-31426

 

Oozie, RelEng

Oozie failed to start due to "SA" schema not created

BUG-31497

 

Phoenix

[RollingUpgrade] Phoenix intermittently fails to return every column on sql queries

BUG-31522

 

Tez

You cannot search by the name of 'Queue'

BUG-31526

 

Tez

The order of columns will go back to the initial setting after you refresh a page or come back from another page.

BUG-31565

 

Tez

When you kill a job in the middle of running it, it Diagnostics shows 'killed/failed due to:null".

BUG-31583

 

Tez

Tez UI : DAG fails but Task attempts don't show any failues

BUG-31594

 

Tez

TEZ view UI Column selector should have a select all or something to make selecting columns easier

BUG-31595

 

Tez

Tez UI : Per DAG counters not getting aggregated

BUG-31598

 

Tez

Tez UI vertical scroll broken from time to time

BUG-31602

 

Tez

Tez UI :Task Tab should have Vertex name as Hyper link not Vertex ID

BUG-31623

 

Tez

Tez UI : Increase the size of table name tex box in the DAG view

BUG-31624

 

Hive

Dropping a 2000+ partition table takes over 10 minutes

BUG-31671

 

Hive

HS2 concurr tests intermittently fail with http mode on secure Openstack throwing org.apache .hive.service .auth.Http Authentication Exception: Kerberos authentication failed

BUG-31676

 

Falcon

Falcon jar uploading occasionally fails

BUG-31749

 

Ranger

Usersync NPE when object does not have userNameAttribute

BUG-31782

 

Ranger

Ranger gives error when using hyphen or space in lastname field

BUG-31822

 

HCatalog

hcat startup failed on windows

BUG-31936

 

HBase

test_hbaseBigLinkedList[Loop] TimeoutException

BUG-31943

 

HBase

test_Integration TestRegion Replicas region not serving

BUG-31949

 

Ranger

In hive plugin TABLE policy restriction for all tables {{*}} fails when UDF policy for all functions {{*}} is maintained for a database.

BUG-32026

 

Oozie

test_mapred_via_oozie_ha[oozie-nn-rm-5-jobs] FAILED

BUG-32049

 

Kafka

No Kafka Documentation Available at docs.hortonworks.com

BUG-32195

 

HBase

Need to add hbase-slider RU downgrade

BUG-32342

 

Tez

Minimum/Maximum Duration show a empty bracket next to 0 secs when you purposefully failed a job.

BUG-32416

 

Hive

Hive result mismatch in decimal digits when converting from double

BUG-32419

 

Oozie

TestBundleVersion: oozie bundle takes 1 hr 10mins to recognized a three completed/killed coordinator/wf actions

BUG-32509

 

HBase

test_runIntegration TestMTTRwith DistributedLogs due to failure in testKillRs HoldingMeta

BUG-32688

 

Storm

kafka/api/OffsetRequest is not packaged in storm-kafka-example jar

BUG-32690

 

Oozie

test_pig_via_oozie_ha[resourcemanager] exception: NotFoundException, message:java. lang.Exception: job, job_1425457888074_0001, is not found,javaClassName

BUG-32732

 

Sqoop

Netezza skip control characters issue

BUG-32735

SLIDER-799

Slider

Slider to manage strict->lax placement

BUG-32763

 

Oozie, RelEng

yum cannot find Xvfb for centos06

BUG-32768

FALCON-954

Falcon

CLONE - Secure Kerberos setup : Falcon should periodically revalidate security credentials.

BUG-32795

 

Hive

HIVE schema failed to upgrade from HDP 2.0/2.1 to HDP 2.2 with schematool

BUG-32859

HIVE-9977

Hive

Hive metastore auto major compaction not happening if data is loaded through dynamic partitioning insert

BUG-32902

 

Falcon

Falcon pom.xml should not use exact HDP versions for Hive,Oozie and Hadoop

BUG-33054

 

RelEng, Spark

Spark examples are packaged differently on Windows

BUG-33056

 

Spark

{start,stop}-thriftserver equivalent script on Windows is missing

BUG-33113

 

HBase

HBase-HA master fail over immediately

BUG-33167

 

Hive

Hive does not prevent non whitelisted config params from being set under certain condtions

BUG-33168

 

Hive

hive authorization checks on configs should not be done if authorization is diabled

BUG-33180

 

Oozie

oozie logging tests fail on MR framework with postgres 9

BUG-33278

HBASE-13239

HBase

HBASE grants at specific column level does not work for Groups

BUG-33321

 

Hue

Created tables not available under Hcatalog

BUG-33335

 

Spark

spark make-distribution.sh doesn't evaluate SPARK_HIVE on OSX

BUG-33338

 

Hive

Hiveserver2 in HTTP mode is not applying auth_to_local rules

BUG-33402 Hive

HiveServer2 fails to start without kerberos ticket in place before starting.

Workaround: In a secure environment, the admin of the cluster will need to do a Kerberos login from the shell of the user running the HiveServer2 process, before starting the server on the host.

If the user hive is going to run the HiveServer2 process and the Kerberos keytab for the principal hive (full principal name of the format: hive/<hostname>@REALM) is in /etc/security/keytabs/hive.service.keytab, then:

  1. su - hive

  2. kinit -k -t /etc/security/keytabs/hive.service.keytab hive/my_hiveserver2_hostname@MY_KERBEROS_REALM

BUG-33546

YARN-3393

YARN

Getting application(s) goes wrong when app finishes before starting the attempt

BUG-33764

 

HDP stack

HDP-Select needs to handle user-created directories in /usr/hdp

BUG-35305

 Hive

Hive insert query runs distcp for local moves when fs.hdfs.impl.disable.cache=true

BUG-35528 Ranger

Users must remove empty user entries from their MYSQL database

Error: Internal Exception: java.sql.SQLException: Connections could not be acquired from the underlying database!

BUG-35942

 YARN

Users must manually configure ZooKeeper security with ResourceManager High Availability

BUG-38088

HDFS-7553

Hadoop Common/HDFS

Issue: DataNode restart fails during Rolling Upgrade from 2.2.0-2.2.6 to 2.3.x.

Details: Not all writers have responder running, but sendOOB() tries anyway. This causes NPE and the shutdown thread dies, halting the shutdown after only shutting down DataXceiverServer

Workaround: Two possible workarounds:

  • Skip the failed datanodes, downgrade them and try to upgrade them again.

  • Manually restart the failed datanodes.

Notes: See also: BUG-38088, BUG-36247, BUG-38009

BUG-50531 Kafka

Kafka file system support

Issue: Encrypted file systems such as SafenetFS are not supported for Kafka. Index file corruption can occur.

For more information, see: Install Kafka.

RMP-2377

HIVE-7926

Hive

Sub-5-Second: LLAP Daemons and LLAP API (tech preview)

RMP-2381

 

Hive

Sub-5-Second: Lookup Query Metastore in less than 200 milliseconds (tech preview)

RMP-2430

HIVE-9792

Hive

SQL: Implement Interval types and datetime arithmetic (partial)

RMP-2448

 

Hive

Sub-5-Second: Umbrella: Execute TPC-DS query 55 in under 5 second at 30TB and Query 27 at 1 TB (tech preview)

RMP-2449

HIVE-9675

Hive

Transactions: Support BEGIN, COMMIT, ROLLBACK

RMP-2620

 

Hive

CBO: Introduce semi join reduction in Calcite

RMP-2634

 

Hive

Replication for Hive (partial)

RMP-2662

 

Hive

Sub-Second: Vectorization enhancements - MapJoin

RMP-3332

HIVE-9792

Hive

RMP-2430 SQL: Partition pruning support for Intervals

RMP-3333

HIVE-9792

Hive

RMP-2430 SQL: ORC PPD support for Intervals