Fixed issues in Cloudera Data Warehouse on premises 1.5.5 SP4

Fixed issues in Cloudera Data Warehouse on premises

DWX-24408: Rebuilding a Database Catalog or Virtual Warehouse before upgrading leads to an error state

Previously, rebuilding a Database Catalog or Virtual Warehouse running on an older runtime version after a Cloudera Control Plane upgrade deployed the latest Helm chart configurations onto older runtime images. Because the updated Helm charts included JDK 17 JVM options incompatible with older JDK 8 images, pods such as hue-query-processor entered a CrashLoopBackOff error state.

This issue is now fixed. The Rebuild option in the UI context menu is now disabled with an explanatory tooltip whenever an upgrade is available for a Database Catalog or Virtual Warehouse. As a result, you must first upgrade the resource to the available version before rebuilding it, preventing configuration mismatches.

DWX-23933: FQDN validation incorrectly restricted full domain names to 63 characters, blocking Cloudera Data Warehouse environment activation

Previously, Cloudera Data Warehouse cluster validation incorrectly enforced a 63-character limit on the entire Fully Qualified Domain Name (FQDN) instead of individual DNS labels. As a result, environment activation failed for valid FQDNs longer than 63 characters, even when they conformed to the RFC 1035 total limit of 253 characters.

This issue is now resolved.

DWX-23912: Log collection in log router pods halts unexpectedly during log rotation

Previously, when container logs were rotated under heavy load, a race condition occurred during file rotation. This caused the log collection timer to stop, permanently halting log forwarding on the affected node until the log router pod was restarted.

This issue is now fixed. The log router configuration is updated to prevent the rotation issue and increase the log processing limit. As a result, log router pods collect and forward logs continuously and reliably without interruption.

DWX-23406: Switching resource templates in an Impala Virtual Warehouse does not update pod resource values

Previously when users updated resource templates on an Impala Virtual Warehouse the underlying pod configurations such as CPU and memory did not update to reflect the new template values. This occurred because an invalid namespace identifier was passed during the internal update workflow which prevented the Virtual Warehouse from applying the changes automatically.

This issue is now fixed.

DWX-15287: Hive DROP queries fail with an InvalidACLException when the Virtual Warehouse is stopped

Previously, running DROP DATABASE, DROP TABLE, or ALTER TABLE DROP PARTITION operations on a stopped Hive Virtual Warehouse resulted in an error:

org.apache.zookeeper.KeeperException$InvalidACLException: KeeperErrorCode = InvalidACL
          for /llap-sasl/user-hive

This occurred because Hive attempted to proactively evict LLAP executor caches while the warehouse was stopped and no query-executor pods were running.

This issue is now fixed.

Fixed issues in Cloudera Data Explorer (Hue) on Cloudera Data Warehouse on premises

DWX-22697: S3 file upload and import failures

Previously, file uploads and remote file imports failed in the File Browser and Importer when using Amazon S3 storage. This issue occurred because incomplete AWS credential configurations were incorrectly identified as active IAM-authenticated setups, causing authorization errors during file operations. This issue is resolved by updating AWS configuration parsing to validate credentials correctly before enabling S3 storage connectors.

CDPD-97902: Out of memory errors occur when opening large Parquet files in the Data Explorer file browser

Previously, opening a large or highly compressed Parquet file in the Data Explorer file browser caused out of memory errors due to unbounded file processing. This issue is now resolved by implementing file size and row count limits during Parquet file processing in Data Explorer to control memory consumption.

CDPD-101746: Performance degradation when listing many files/folders in S3

Previously, the S3 file browser became slow or unresponsive when you listed a large number of files or folders. The directory listing process triggered redundant API requests for each file, causing significant delays. This issue is now resolved by optimizing the directory listing process to retrieve file information more efficiently.

DWX-23315: Error when accessing the Impala tab in the Jobs browser

Previously, an error occurred when you clicked a query ID in the editor to open the query profile page if your environment uses an active-passive coordinator setup. This issue exists as a following changes made to support active-active coordinator configurations, preventing access to detailed query metrics in active-passive environments. This issue is now resolved.

CDPD-101177: File upload failure to S3 in Data Explorer (Cloudera Data Explorer (Hue))

Previously, file uploads to AWS S3 through Data Explorer (Cloudera Data Explorer (Hue)) might fail in both RAZ and non-RAZ environments because the storage connector incorrectly attempted to use a legacy configuration path. This issue is resolved by ensuring the storage connector uses the correct configuration path for the boto3 SDK.

CDPD-98620: Intermittent 403 authentication failures on ABFS file uploads

Previously, an intermittent file upload failed with a 403 Client Error when uploading to ABFS. This issue occurred because the system incorrectly normalized the full URL, including the security token. This process collapsed double slashes in the signature part of the token into a single slash, causing Azure to reject the request.

This issue is now resolved by splitting the URL to ensure normalization only applies to the path component while preserving the security token signature.

CDPD-101348: Download logs button in Data Explorer (Cloudera Data Explorer (Hue)) administration UI leads to incorrect URL

Previously, the Download logs button in the Data Explorer (Cloudera Data Explorer (Hue)) administration UI pointed to an incorrect or outdated URL, which prevented from downloading server logs. This issue is now resolved by updating the log download API endpoints.

CDPD-95666: Persistent loading indicator for unpermitted directories

Previously, selecting a folder or file without read permissions in Microsoft Azure environments caused a persistent loading bar to appear across file systems, including Amazon S3 and HDFS. The user interface remained in a loading state and failed to display an alert message indicating missing permissions. This issue is now fixed.

CDPD-23994: Table browser error due to mixed data type columns in Data Explorer

Previously, the table page in Data Explorer displayed a ‘<’ not supported between instances of 'str' and 'int' comparison error. This occurred within the Autocomplete and Describe APIs when columns contained mixed data types, which caused sorting and description operations to fail when both strings and integers were present. This issue is now fixed by correcting how the Autocomplete and Describe APIs handle and sort mixed data types, ensuring that the table page displays correctly without errors.

CDPD-82506: Single-column CSV import failure in Data Explorer

Previously, importing a CSV file containing only a single column in Data Explorer failed during the file format detection step. This prevented the data preview from displaying and disabled the submission process. This issue is resolved by improving file format parsing and field name inference for single-column CSV files.

CDPD-100816: Data Explorer File Browser workflow submission button failure

Previously, the Submit button component in the Data Explorer File Browser application failed to function, which prevented submitting workflows. This issue is now fixed by restoring the required path reference dependency.

DWX-24136: Trino query failure in Data Explorer

Previously, running Trino queries in Data Explorer failed with a "cannot use authentication with HTTP" error on shared Virtual Warehouse instances. This issue occurred because service discovery fallback mechanisms built an incorrect Trino coordinator URL, which omitted required authentication credentials. This issue is now resolved.

CDPD-68779: Error while browsing S3 buckets or ADLS containers from the left-assist panel

Previously, when attempting to browse S3 buckets or ADLS containers from the left-assist panel in Data Explorer without the required permissions, the system displayed a generic error message: Failed to retrieve buckets:1:0: syntax error. This issue is now fixed. When RAZ is enabled, the left-assist panel now opens the home directory if the required permissions are available. In non-RAZ environments, the left-assist panel opens the path configured in the Data Explorer configuration file.

CDPD-104785: Arbitrary file read on Data Explorer hosts through the localfile importer feature

Previously, a local file inclusion vulnerability existed on the Data Explorer host type through the localfile importer feature. This occurred because two indexer API endpoints accepted unvalidated paths, allowing unauthorized file reads through the path parameter. This issue is now fixed by blocking the localfile input format processing within the API endpoint configuration.

CDPD-100326: Trino editor Unauthorized error in Data Explorer

Previously, accessing the Trino editor in shared Data Explorer instances resulted in a 401 Unauthorized error. This issue occurred because service discovery logic incorrectly derived the Trino endpoint URL and authentication credentials from LDAP settings instead of the Trino configuration. This issue is resolved by updating service discovery endpoint resolution and restricting basic authentication usage to secure HTTPS connections.

Fixed issues in Hive on Cloudera Data Warehouse on premises

CDPD-97944: Duplicate records during minor compaction after Hive Metastore crash

Previously, if a HMS instance running the compaction initiator crashed while the HiveServer2 (HS2) workers on the same host survived, the system incorrectly reset running compaction jobs to the initiated state.

This issue is now fixed by updating the compactor cleaner to address and prevent the creation of duplicate directories when multiple jobs attempt the same compaction.

Apache Jira: HIVE-29210

CDPD-97945: Data loss during query-based minor compaction

Previously, the compactor skipped relevant delta files when aborted or open transactions existed, resulting in empty compaction directories. Consequently, when the cleaning process occurred, the original data files were removed, leading to data loss in insert-only tables.

This issue is now fixed by removing the incorrect minimum open write ID check.

Apache Jira: HIVE-29272

HIVE-27022: Improved performance for AcidHouseKeeper by moving compaction tasks to a separate timer

Compaction housekeeping tasks previously shared the same execution frequency as other AcidHouseKeeper jobs, which caused bottlenecks in the backend database on busy clusters.

This issue is resolved by moving the duplication removal components for transactions to a separate timer.

Apache Jira: HIVE-27022

HIVE-28960: Column statistics collection during compaction

Previously, column statistics were not collected during compaction if the automatic statistics gathering property was enabled.

This issue is resolved by allowing the compaction statistics updater to run independently of the automatic gathering setting.

Apache Jira: HIVE-28960

HIVE-29166: Repeated MERGE queries generate duplicate records

Previously, running a MERGE query multiple times generated duplicate records or returned incorrect results.

This issue is resolved by updating the data routing logic.

Apache Jira: HIVE-29166

HIVE-29251: Data cleanup tasks fail due to shared configuration objects

Previously, multiple cleanup tasks shared the same configuration object. When running multiple threads simultaneously, this caused a race condition where one task could overwrite the transaction settings of another.

This issue is resolved by using thread-local configuration objects. Each cleanup task now maintains its own settings, preventing interference between threads and ensuring that data is only removed after transactions are complete.

Apache Jira: HIVE-29251

HIVE-27526: Inaccurate data cleanup

Previously, the data cleanup process used inaccurate logic to check for remaining files after compaction.

This issue is resolved by ensuring the cleanup process honors the compaction high watermark.

Apache Jira: HIVE-27526

Fixed issues in Iceberg on Cloudera Data Warehouse on premises

CDPD-102654, CDPD-105436: Iceberg DELETE does not respond when the tuple cache is enabled

Previously, when the Impala tuple cache was enabled and MT_DOP was greater than 2, DELETE statements on Iceberg tables could stall indefinitely and remain in a running state. The issue occured when you ran multiple DELETE statements on the same table.

This issue is now fixed. Impala now completes Iceberg DELETE statements correctly when the tuple cache is enabled.

Apache Jira: IMPALA-14951

Fixed issues in Impala on Cloudera Data Warehouse on premises

IMPALA-14949: Catalogd deadlock during initial global invalidate metadata operation

Under heavy load or during catalogd startup (such as after a restart due to an out-of-memory event), catalogd hung and queries failed to reach the catalog server.

This issue is resolved by updating the getOrLoadTable function to wait for initial database metadata loading before taking the versionLock_ read lock.

Apache Jira: IMPALA-14949

CDPD-98207: Impala crashing on the Web UI for failed queries

Previously, Cloudera Runtime crashed when you accessed the query summary or JSON plan through the Web UI for queries that failed before execution. This occurred during scenarios such as a Create Table As Select (CTAS) failure or when admission control rejected a query.

This issue is addressed by ensuring the system correctly handles missing execution summaries. This issue is now fixed.

Apache Jira: IMPALA-14791

Fixed issues in Trino on Cloudera Data Warehouse on premises

There are no fixed issues in this release.