What's new in Cloudera Runtime 7.1.9 SP1 CHF 16

Understand the functionalities and improvements to features of components in Cloudera Runtime 7.1.9 SP1 CHF 16.

Impala

New TCMalloc metrics

You can now monitor memory allocation more effectively by using new numeric properties from TCMalloc on the metrics page.

The following metrics are now available:

tcmalloc.current-total-thread-cache-bytes: The number of bytes used across all thread caches.
tcmalloc.central-cache-free-bytes: The number of free bytes in the central cache that are assigned to size classes.
tcmalloc.transfer-cache-free-bytes: The number of free bytes waiting to be transferred between the central cache and a thread cache.
tcmalloc.thread-cache-free-bytes: The number of free bytes currently in thread caches.

Iceberg

Query optimization for MIN, MAX, and COUNT DISTINCT on Iceberg tables

Impala now optimizes MIN(key_column), MAX(key_column), or

COUNT(DISTINCT
        key_column)

queries on partition columns of Iceberg tables by leveraging partition-level metadata.

For Iceberg tables, partition statistics are retrieved from Iceberg metadata through the Iceberg API. This optimization applies to Iceberg v1 and v2 tables and is effective when partition transforms use identity.

Apache Jira: IMPALA-11986

Allow forced predicate pushdown to Iceberg

Since IMPALA-11591, Impala has optimized query planning by avoiding predicate pushdown to Iceberg unless it is potentially beneficial. While this default behavior makes planning faster in most cases, it can miss opportunities to prune files early based on Iceberg's file-level statistics.

A new table property, impala.iceberg.push_down_hint is introduced, which allows you to force predicate pushdown for specific columns. The property accepts a comma-separated list of column names, for example, 'col_a, col_b'.

If a query contains a predicate on any column listed in the impala.iceberg.push_down_hint property, Impala tries to push down all the predicates for evaluation during the planning phase.

Apache Jira: IMPALA-14123