Fixed Issues in Impala

Review the list of Impala issues that are resolved in Cloudera Runtime 7.1.9 SP1.

CDPD-67912: Multiple fixes address performance issues with improved memory tracking for row batch serialization during KRPC shuffling

Queries shuffling large row batches were consuming untracked memory during row batch serialization, leading to higher memory usage without proper accounting.

Memory for tuple_data_ and compression scratch is now tracked using MemTrackerAllocator, and memory estimation. Added to account for outbound row batch allocation. Memory for row batch serialization is now fully tracked, leading to more accurate memory usage in queries that shuffle rows.

CDPD-67901: Support python 3.10

The impala-shell commands now works with Python 3.10 on SLES 15 operating system.

CDPD-60005: Fix for two Catalogd with force active failure

The issue has been resolved.

CDPD-67493: Catalogd performance fix

The Catalogd performance is enhanced by the following improvements done through:

• Impala now reloads events to update metadata across Catalogd replicas for refresh or invalidate commands.
• Impala now handles events thrown by Hive Metastore when a compaction is committed, allowing for automatic refreshing after compactions.
• Impala now skips redundant refresh events by processing only the first event, reducing overhead.
• Catalogd now skips preceding events after a manual table refresh, allowing it to quickly catch up with Hive Metastore events.
• Impala now efficiently processes notification events in Catalogd.
• Refined batching logic in Catalogd optimizes event processing by grouping events across different sources per table, enhancing efficiency.
• Impala now optimizes event processing by skipping unnecessary events and better handling manual refreshes to prevent disruptions.
• Catalogd's event processor now reloads file metadata specifically for truncate table operations, in Hive Metastore alter table events.
• Impala's event batching now efficiently manages events for a table, even if there are unrelated events in between.
• Impala now reloads events to update metadata across Catalogd replicas for refresh or invalidate commands.
• Impala now includes table-level metrics added to the Catalogd webUI page, such as avg-events-process-duration and events-consuming-delay.
• Impala now accurately processes subsequent ALTER_PARTITION events after creating a partition through INSERT operation, ensuring enhanced event processing stability and accuracy.