HDP-2.3.2 Release Notes
Also available as:
PDF

Hive

HDP 2.3.2 provides Hive 1.2.1 and the following Apache patches:

IMPROVEMENTS

  • HIVE-11037 HiveOnTez: make explain user level = true as default

BUG FIXES

  • HIVE-10140: Window boundary is not compared correctly

  • HIVE-10453: Reverted

  • HIVE-10569: Hive CLI gets stuck when hive.exec.parallel=true; and some exception happens during SessionState.start

  • HIVE-10571: HiveMetaStoreClient should close existing thrift connection before its reconnect

  • HIVE-10620: ZooKeeperHiveLock overrides equal() method but not hashcode()

  • HIVE-10646: ColumnValue does not handle NULL_TYPE

  • HIVE-10651: ORC file footer cache should be bounded

  • HIVE-10698: query on view results fails with table not found error if view is created with subquery alias (CTE).

  • HIVE-10714: Bloom filter column names specification should be case insensitive

  • HIVE-10722: external table creation with msck in Hive can create unusable partition

  • HIVE-10726: Hive JDBC setQueryTimeout should not throw exception to make it work with JMeter

  • HIVE-10731: NullPointerException in HiveParser.g

  • HIVE-10732: Hive JDBC driver does not close operation for metadata queries

  • HIVE-10771: "separatorChar" has no effect in "CREATE TABLE AS SELECT" statement

  • HIVE-10781: HadoopJobExecHelper Leaks RunningJobs

  • HIVE-10790: orc write on viewFS throws exception

  • HIVE-10793: HIVE-11587 Hybrid Grace Hash Join: Don't allocate all hash table memory upfront

  • HIVE-10802: Table join query with some constant field in select fails

  • HIVE-10808: Inner join on Null throwing Cast Exception

  • HIVE-10835: Concurrency issues in JDBC driver

  • HIVE-10880: The bucket number is not respected in insert overwrite.

  • HIVE-10925: Non-static threadlocals in metastore code can potentially cause memory leak

  • HIVE-10963: Hive throws NPE rather than meaningful error message when window is missing

  • HIVE-10972: DummyTxnManager always locks the current database in shared mode, which is incorrect.

  • HIVE-11013: MiniTez tez_join_hash test on the branch fails with NPE (initializeOp not called?)

  • HIVE-11024: Error inserting a date value via parameter marker (PreparedStatement.setDate)

  • HIVE-11029: hadoop.proxyuser.mapr.groups does not work to restrict the groups that can be impersonated

  • HIVE-11054: Read error: Partition Varchar column cannot be cast to string

  • HIVE-11079: Fix qfile tests that fail on Windows due to CR/character escape differences

  • HIVE-11087: DbTxnManager exceptions should include txnid

  • HIVE-11090: ordering issues with windows unit test runs

  • HIVE-11095: SerDeUtils another bug ,when Text is reused

  • HIVE-11102: ReaderImpl: getColumnIndicesFromNames does not work for some cases

  • HIVE-11112: ISO-8859-1 text output has fragments of previous longer rows appended

  • HIVE-11135: Fix the Beeline set and save command in order to avoid the NullPointerException

  • HIVE-11151: Calcite transitive predicate inference rule should not transitively add not null filter on non-nullable input

  • HIVE-11152: Swapping join inputs in ASTConverter

  • HIVE-11157: Hive.get(HiveConf) returns same Hive object to different user sessions

  • HIVE-11171: Join reordering algorithm might introduce projects between joins

  • HIVE-11172: Vectorization wrong results for aggregate query with where clause without group by

  • HIVE-11174: Hive does not treat floating point signed zeros as equal (-0.0 should equal 0.0 according to IEEE floating point spec)

  • HIVE-11176: Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct cannot be cast to [Ljava.lang.Object;

  • HIVE-11193: ConstantPropagateProcCtx should use a Set instead of a List to hold operators to be deleted

  • HIVE-11198: Fix load data query file format check for partitioned tables

  • HIVE-11203: Beeline force option doesn't force execution when errors occurred in a script.

  • HIVE-11211: Reset the fields in JoinStatsRule in StatsRulesProcFactory

  • HIVE-11216: UDF GenericUDFMapKeys throws NPE when a null map value is passed in

  • HIVE-11221: In Tez mode, alter table concatenate orc files can intermittently fail with NPE

  • HIVE-11255: get_table_objects_by_name() in HiveMetaStore.java needs to retrieve table objects in multiple batches

  • HIVE-11258: The function drop_database_core() of HiveMetaStore.java may not drop all the tables

  • HIVE-11271: java.lang.IndexOutOfBoundsException when union all with if function

  • HIVE-11301: thrift metastore issue when getting stats results in disconnect

  • HIVE-11303: Getting Tez LimitExceededException after dag execution on large query

  • HIVE-11317: ACID: Improve transaction Abort logic due to timeout

  • HIVE-11320: ACID enable predicate pushdown for insert-only delta file

  • HIVE-11344: HIVE-9845 makes HCatSplit.write modify the split so that PartInfo objects are unusable after it

  • HIVE-11356: SMB join on tez fails when one of the tables is empty

  • HIVE-11357: ACID enable predicate pushdown for insert-only delta file 2

  • HIVE-11375: Broken processing of queries containing NOT (x IS NOT NULL and x 0)

  • HIVE-11407: JDBC DatabaseMetaData.getTables with large no of tables call leads to HS2 OOM

  • HIVE-11429: Increase default JDBC result set fetch size (# rows it fetches in one RPC call) to 1000 from 50

  • HIVE-11433: NPE for a multiple inner join query

  • HIVE-11442: Remove commons-configuration.jar from Hive distribution

  • HIVE-11449: HIVE-11587 "Capacity must be a power of two" error when HybridHashTableContainer memory threshold is too low

  • HIVE-11456: HCatStorer should honor mapreduce.output.basename

  • HIVE-11467: HIVE-11587 WriteBuffers rounding wbSize to next power of 2 may cause OOM

  • HIVE-11493: Predicate with integer column equals double evaluates to false

  • HIVE-11502: Map side aggregation is extremely slow

  • HIVE-11581: HiveServer2 should store connection params in ZK when using dynamic service discovery for simpler client connection string.

  • HIVE-11587: Fix memory estimates for mapjoin hashtable

  • HIVE-11592: ORC metadata section can sometimes exceed protobuf message size limit

  • HIVE-11600: Hive Parser to Support multi col in clause (x,y..) in ((..),..., ())

  • HIVE-11605: Incorrect results with bucket map join in tez.

  • HIVE-11606: Bucket map joins fail at hash table construction time

  • HIVE-11607: Export tables broken for data > 32 MB

  • HIVE-11658: Load data file format validation does not work with directories

  • HIVE-11727: (BUG-44285). Hive on Tez through Oozie: Some queries fail with fnf exception

  • HIVE-11755: Incorrect method called with Kerberos enabled in AccumuloStorageHandler

  • HIVE-11820: export tables with size of >32MB throws "java.lang.IllegalArgumentException: Skip CRC is valid only with update options"

  • HIVE-11836: ORC SARG creation throws NPE for null constants with void type

  • HIVE-11839: Vectorization wrong results with filter of (CAST AS CHAR)

  • HIVE-11849: NPE in HiveHBaseTableShapshotInputFormat in query with just count(*)

  • HIVE-11852: numRows and rawDataSize table properties are not replicated

  • HIVE-11875: JDBC Driver does not honor delegation token mechanism when readings params from ZooKeeper

  • HIVE-11897: JDO rollback can throw pointless exceptions

  • HIVE-11928: ORC footer section can also exceed protobuf message limit

  • HIVE-11936: Support SQLAnywhere as a backing DB for the hive metastore

  • HIVE-5277: HBase handler skips rows with null valued first cells when only row key is selected

  • HIVE-6727: Table level stats for external tables are set incorrectly

  • HIVE-7476: CTAS does not work properly for s3

  • HIVE-8529: HiveSessionImpl#fetchResults should not try to fetch operation log when hive.server2.logging.operation.enabled is false.

  • HIVE-9566: HiveServer2 fails to start with NullPointerException

  • HIVE-9625: Delegation tokens for HMS are not renewed

  • HIVE-9811: Hive on Tez leaks WorkMap objects

  • HIVE-9974: Sensitive data redaction: data appears in name of mapreduce job

HDP 2.3.0 provided Hive 1.2.1 and the following Apache patches:

INCOMPATIBLE CHANGES

  • HIVE-11118 Load data query should validate file formats with destination tables

NEW FEATURES

  • HIVE-10233 Hive on Tez: memory manager for grace hash join

IMPROVEMENTS

  • HIVE-11164 WebHCat should log contents of HiveConf on startup HIVE-11037 HiveOnTez: make explain user level = true as default

BUG FIXES

  • HIVE-10251 HIVE-9664 makes hive depend on ivysettings.xml (using HIVE-10251.simple.patch)

  • HIVE-10996 Aggregation / Projection over Multi-Join Inner Query producing incorrect results

  • HIVE-11028 Tez: table self join and join with another table fails with IndexOutOfBoundsException

  • HIVE-11048 Make test cbo_windowing robust

  • HIVE-11050 testCliDriver_vector_outer_join.* failures in Unit tests due to unstable data creation queries

  • HIVE-11051 Hive 1.2.0 MapJoin w/Tez - LazyBinaryArray cannot be cast to [Ljava.lang.Object;

  • HIVE-11059 hcatalog-server-extensions tests scope should depend on hive-exec

  • HIVE-11060 Make test windowing.q robust

  • HIVE-11066 Ensure tests don't share directories on FS

  • HIVE-11074 Update tests for HIVE-9302 after removing binaries

  • HIVE-11076 Explicitly set hive.cbo.enable=true for some tests

  • HIVE-11083 Make test cbo_windowing robust

  • HIVE-11104 Select operator doesn't propagate constants appearing in expressions

  • HIVE-11147 MetaTool doesn't update FS root location for partitions with space in name