Cloudera Data Warehouse Private Cloud 1.5.4 SP1

Version information for Cloudera Data Warehouse Private Cloud 1.5.4 SP1 components

Cloudera Data Warehouse uses Hive, Impala, and Hue as its Runtime components and also provides integration with Cloudera Data Visualization. Review the version information of CDW Private Cloud 1.5.4 SP1 components.

Table 1. CDW Private Cloud version information
CDW component	Version
Hive	2024.0.18.3-15
Impala	2024.0.18.3-15
Hue	2024.0.18.3-15
Cloudera Data Visualization	7.2.7-b48
CDW server	1.10.0-b49
CDP CLI	0.9.126

Apache Iceberg version information

The following table shows the version of the Iceberg component in this release of CDW Private Cloud:


CDW server version	CDW Runtime version	Iceberg version
1.10.0-b49	2024.0.18.3-15	1.4.3

What's new in Cloudera Data Warehouse Private Cloud 1.5.4 SP1

Review the new features introduced in this cumulative hotfix release of Cloudera Data Warehouse Private Cloud 1.5.4 SP1.

Support for forwarding logs to your observability system

In this release, you can forward logs from environments activated in Cloudera Data Warehouse to observability and monitoring systems such as Datadog, New Relic, or Splunk. You configure a Cloudera Data Warehouse environment for these systems using the UI to provide a fluentd configuration.

Workload-aware autoscaling for Impala (General Availability)

Using workload-aware autoscaling, you can configure multiple executor groups within a single Virtual Warehouse that can independently autoscale to allow handling of different workloads in the same Virtual Warehouse. According to each query’s resource requirement, the query is scheduled on an executor group size that is appropriate for that query. For more information, see Workload aware autoscaling in Impala.

You must select the Enable workload-aware autoscaling for Impala option from Advanced Configurations to use workload-aware autoscaling. See Enabling workload-aware autoscaling for Impala.

Ability to log and manage Impala workloads (Preview)

Cloudera Data Warehouse provides you the option to enable logging Impala queries on an existing Virtual Warehouse or while creating a new Impala Virtual Warehouse. By logging the Impala queries in Cloudera Data Warehouse, you gain increased observability of the workloads running on Impala, which you can use to improve the performance of your Impala Virtual Warehouses.

This feature represents a significant enhancement to query profiling capabilities. You can have Impala archive crucial data from each query's profile into dedicated database tables known as the query history table and live query table. These tables are part of the sys database and are designed to store valuable information that can later be queried using any Impala client, providing a consolidated view of reports from previously executed queries.

For more information, see Impala workload management in Cloudera Data Warehouse (Preview).

Fixed issues in Cloudera Data Warehouse Private Cloud 1.5.4 SP1

Review the issues fixed in this service pack release of Cloudera Data Warehouse Private Cloud.

Security fixes

The following security fixes are available as part of this release:

DWX-18712: Replace Java tools for JCEKS with Go: This fix prevents CVEs resulting from the openjdk8 package on Impala autoscaler, by using a tool built in Golang (Go) language to read keys from JCEKS instead of the existing Java-based tool.
DWX-19154: Upgrade to the latest Kubernetes version: The Kubernetes package was upgraded to the latest version, 1.31.0 to help prevent CVEs.
DWX-19202/DWX-19203/DWX-19267: Move images to Chainguard: The following images are now based on the Chaiguard images to significantly reduce the CVE count:

hive

impala-autoscaler-webui-metrics

diagnostic-tools
DWX-19250: Cloudera Data Warehouse containers elevate their own privileges: This fix configures the containers in the Control Plane and sets containers[].securityContext.allowPrivilegeEscalation to "false".
DWX-19537: initContainers elevate their own privileges: This fix configures and sets initContainers[].securityContext.allowPrivilegeEscalation to "false".

Known issues in CDW Private Cloud 1.5.4 SP1

Review the issues identified in this service pack release of Cloudera Data Warehouse Private Cloud.

DWX-19016: Hue Importer displays an incorrect status message: When you create an Impala table by importing CSV files using the Hue Importer, the Importer window might display a warning indicating that the query has failed. This is an incorrect message and the table is successfully created.; None. This issue is only related to the display of an incorrect status message and does not affect the actual table creation process.
Hive compaction of Iceberg tables results in a failure: When Cloudera Data Warehouse and CDP Private Cloud Base are deployed in the same environment and use the same Hive Metastore (HMS) instance, the CDP Private Cloud Base compaction workers can inadvertently pick up Iceberg compaction tasks. Since Iceberg compaction is not yet supported in the latest CDP Private Cloud Base version, the compaction tasks will fail when they are processed by the CDP compaction workers.
In such a scenario where both Cloudera Data Warehouse and CDP Private Cloud Base share the same HMS instance and there is a requirement to run both Hive ACID and Iceberg compaction jobs, it is recommended that you use the Cloudera Data Warehouse environment for these jobs. If you want to run only Hive ACID compaction tasks, you can choose to use either the Cloudera Data Warehouse or CDP environments.; If you want to run the compaction jobs without changing the environment, it is recommended that you use Cloudera Data Warehouse. To avoid interference from CDP, change the value of the hive.compactor.worker.threads Hive Server (HS2) property to '0'. This ensures that the compaction jobs are not processed by CDP.

In Cloudera Manager, click Clusters > Hive > Configuration to navigate to the configuration page for HMS.

Search for hive.compactor.worker.threads and modify the value to '0'.

Save the changes and restart the Hive service.
DWX-19489: Concurrent Hive-Iceberg UPDATE/INSERT query fails: Concurrent UPDATE/INSERT queries on Hive Virtual Warehouses might fail intermittently with the following error:
return code 40000 from org.apache.hadoop.hive.ql.exec.MoveTask. Error committing job; Run the failed queries again.

Behavior changes in Cloudera Data Warehouse 1.5.4 SP1

Review the behavior changes introduced in this service pack release of Cloudera Data Warehouse Private Cloud.

Summary: Change in value of the query executor stack size

Before this release: The default value for the Java VM configuration for thread stack size (-Xss) resource type was set to a default value of "256k".

After this release: The query executor stack size is increased and the default value is now changed to "512k" to address query failures that were noticed during a TCP-DS benchmark run.