HDP-2.4.3 Release Notes
Also available as:
PDF

Hive

HDP 2.4.3 provides Hive 1.2.1 and the following Apache patches:

  • HIVE-10233: Hive on tez memory manager for grace hash join.

  • HIVE-10631: create_table_core method has invalid update for Fast Stats.

  • HIVE-10632: Make sure TXN_COMPONENTS gets cleaned up if table is dropped before compaction.

  • HIVE-11037: HiveOnTez make explain user level = true as default.

  • HIVE-11118: Load data query should validate file formats with destination tables.

  • HIVE-11164: WebHCat should log contents of HiveConf on startup.

  • HIVE-11498: HIVE Authorization v2 should not check permission for dummy entity.

  • HIVE-11716: Reading ACID table from non-acid session should raise an error.

  • HIVE-11891: Add basic performance logging to metastore calls.

  • HIVE-12619: Switching the field order within an array of structs causes the query to fail.

  • HIVE-12768: Thread safety binary sortable serde decimal deserialization.

  • HIVE-13043: Reload function has no impact to function registry.

  • HIVE-13051: Deadline class has numerous issues.

  • HIVE-13175: Disallow making external tables transactional.

  • HIVE-13191: DummyTable map joins mix up columns between tables.

  • HIVE-13294: AvroSerde leaks the connection in a case when reading schema from a url.

  • HIVE-13299: Column Names trimmed of leading and trailing spaces.

  • HIVE-13303: Spill to YARN directories, not tmp, when available.

  • HIVE-13313: TABLESAMPLE ROWS feature broken for Vectorization.

  • HIVE-13338: Differences in vectorized_casts.q output for vectorized and non-vectorized runs.

  • HIVE-13358: Stats state is not captured correctly turn off stats optimizer for sampled table.

  • HIVE-13373: Use most specific type for numerical constants.

  • HIVE-13390: Add more test to ZK service discovery using MiniHS2 (SSL Certificate Hostname vulnerability fix).

  • HIVE-13392: Disable speculative execution for ACID Compactor.

  • HIVE-13394: Analyze table fails in tez on empty partitions/files/tables.

  • HIVE-13405: Fix Connection Leak in OrcRawRecordMerger.

  • HIVE-13561: HiveServer2 is leaking ClassLoaders when add jar / temporary functions are used.

  • HIVE-13592: Metastore calls map is not thread safe.

  • HIVE-13596: HS2 should be able to get UDFs on demand from metastore.

  • HIVE-13609: Fix UDTFs to allow local fetch task to fetch rows forwarded by GenericUDTF.close().

  • HIVE-13618: Trailing spaces in partition column will be treated differently.

  • HIVE-13619: Bucket map join plan is incorrect.

  • HIVE-13632: Hive failing on insert empty array into parquet table.

  • HIVE-13660: Vectorizing IN expression with list of columns throws java.lang.ClassCastException ExprNodeColumnDesc cannot be cast to ExprNodeConstantDesc.

  • HIVE-13691: No record with CQ_ID=0 found in COMPACTION_QUEUE.

  • HIVE-13693: Multi-insert query drops Filter before file output when there is a.val <> b.val.

  • HIVE-13729: FileSystem$Cache leaks in FileUtils.checkFileAccessWithImpersonation.

  • HIVE-13730: Avoid double spilling the same partition when memory threshold is set very low.

  • HIVE-13948: Incorrect timezone handling in Writable results in wrong dates in queries.

  • HIVE-13991: Union All on view fail with no valid permission on underneath table.

  • HIVE-14027: NULL values produced by left outer join do not behave as NULL.

  • HIVE-14045: (Vectorization) Add missing case for BINARY in VectorizationContext.getNormalizedName method.

  • HIVE-14132: Don't fail config validation for removed configs.

  • HIVE-14147: Hive PPD might remove predicates when they are defined as a simple expression e.g. WHERE 'a'.

  • HIVE-14167: Use work directories provided by Tez instead of directly using YARN local dirs.

  • HIVE-14178: Hive::needsToCopy should reuse FileUtils::equalsFileSystem.

  • HIVE-14262: Inherit writetype from partition WriteEntity for table WriteEntity.

  • HIVE-14277: Disable StatsOptimizer for all ACID tables.

  • HIVE-14278: Migrate TestHadoop23SAuthBridge.java from Unit3 to Unit4.

  • HIVE-14279: fix mvn test TestHiveMetaStore.testTransactionalValidation.

  • HIVE-14282: HCatLoader ToDate() exception with hive partition table ,partitioned by column of DATE datatype.

  • HIVE-14348: Alter Table Exchange Partition NullPointerException and authorization issues.

  • HIVE-9605: Remove parquet nested objects from wrapper writable objects .

HDP 2.4.2 provided Hive 1.2.1 and the following Apache patches:

  • HIVE-11097: HiveInputFormat uses String.startsWith to compare splitPath and PathToAliases.

  • HIVE-11333: ColumnPruner prunes columns of UnionOperator that should be kept.

  • HIVE-11470: NPE in DynamicPartFileRecordWriterContainer on null part-keys.

  • HIVE-11816: Upgrade groovy to 2.4.4.

  • HIVE-11841: KeyValuesInputMerger creates huge logs.

  • HIVE-12022: NPE in SARG with timestamp cast.

  • HIVE-12064: prevent transactional=false.

  • HIVE-12165: wrong result when hive.optimize.sampling.orderby=true with some aggregate functions.

  • HIVE-12189: The list in pushdownPreds of ppd.ExprWalkerInfo should not be allowed to grow very large.

  • HIVE-12352: CompactionTxnHandler.markCleaned() may delete too much.

  • HIVE-12684: NPE in stats annotation when all values in decimal column are NULLs.

  • HIVE-12742: NULL table comparison within CASE does not work as previous hive versions.

  • HIVE-12749: Constant propagate returns string values in incorrect format.

  • HIVE-12784: Group by SemanticException: Invalid column reference.

  • HIVE-12879: RowResolver of Semijoin not updated in CalcitePlanner.

  • HIVE-12947: SMB join in tez has ClassCastException when container reuse is on.

  • HIVE-13016: ORC FileDump recovery utility fails in Windows.

  • HIVE-13092: Vectorized java.lang.ClassCastException: org.apache.hadoop.hive.serde2.typeinfo.ListTypeInfo cannot be cast to org.apache.hadoop.hive.serde2.typeinfo.PrimitiveTypeInfo.

  • HIVE-13129: CliService leaks HMS connection.

  • HIVE-13144: HS2 can leak ZK ACL objects when curator retries to create the persistent ephemeral node.

  • HIVE-13151: Clean up UGI objects in FileSystem cache for transactions.

  • HIVE-13174: Remove Vectorizer noise in logs.

  • HIVE-13232: Aggressively drop compression buffers in ORC OutStreams.

  • HIVE-13233: Use min and max values to estimate better stats for comparison operators.

  • HIVE-13263: Vectorization: Unable to vectorize regexp_extract/regexp_replace " Udf: GenericUDFBridge, is not supported".

  • HIVE-13285: Orc concatenation may drop old files from moving to final path.

  • HIVE-13298: nested join support causes undecipherable errors in SemanticAnalyzer.

  • HIVE-13325: Excessive logging when ORC PPD fails type conversions.

HDP 2.4.0 provided Hive 1.2.1 and the following Apache patches:

  • HIVE-11141: Improve RuleRegExp when the Expression node stack gets huge.

  • HIVE-11291: Avoid allocation storm while doing rule matching on operator/expression trees.

  • HIVE-11310: Avoid expensive AST tree conversion to String for expressions in WHERE clause.

  • HIVE-11311: Avoid dumping AST tree String in Explain unless necessary.

  • HIVE-11328: Avoid String representation of expression nodes in ConstantPropagateProcFactory unless necessary.

  • HIVE-11330: Add early termination for recursion in StatsRulesProcFactory.evaluateExpression.

  • HIVE-11397: Parse Hive OR clauses as they are written into the AST.

  • HIVE-11405: Add early termination for recursion in StatsRulesProcFactory.evaluateExpression for OR expression.

  • HIVE-11406: Vectorization: StringExpr::compare() == 0 is bad for performance.

  • HIVE-11981: ORC Schema Evolution Issues (Vectorized, ACID, and Non-Vectorized).

  • HIVE-12625: Backport to branch-1 HIVE-11981 ORC Schema Evolution Issues (Vectorized, ACID, and Non-Vectorized).

  • HIVE-12660: HS2 memory leak with .hiverc file use.

  • HIVE-12706: Incorrect output from from_utc_timestamp()/to_utc_timestamp when local timezone has DST.

  • HIVE-12728: Apply DDL restrictions for ORC schema evolution.

  • HIVE-12766: TezTask does not close DagClient after execution.

  • HIVE-12799: Always use Schema Evolution for ACID.