Cloudera Docs

October 5, 2023

This release of the Cloudera Data Warehouse (CDW) service on CDP Public Cloud introduces these changes.

Automatically backing up and restoring CDW

In earlier releases, the manual backup and restore feature replaced the in-place upgrade of AWS environments. This release adds automation to back up and restore procedures for AWS and Azure environments. You can still perform Hive and Impala in-place upgrades.

To get the supported Kubernetes version, you back up your old AWS or Azure environment and start up a new environment using the restoration process. The backup/restore feature saves your environment parameters, making it possible to recreate your environment with the same settings, URL, and connection strings you used in your previous environment.

CDP CLI Start/Stop Azure Kubernetes Service (AKS) Technical Preview

Using the Beta version of the CDP CLI, you can now start and stop a Cloudera Data Warehouse (CDW) cluster on Azure using the following commands:
  • resumeCluster <cluster ID>

    Starts the cluster.

  • suspendCluster <cluster ID>

    Stops the cluster.

The following prerequisites are required to use resumeCluster and suspendCluster :
  • You obtained a cluster ID parameter to pass with the command.
  • The cluster you want to stop is running.
  • You stopped the Virtual Warehouses and Database Catalogs in the cluster you want to stop
  • You stopped the cluster before attempting to start it.
  • The cluster you want to start or stop is not in an error state.
The Microsoft documentation describes how resumeCluster and suspendCluster starts and stops the AKS cluster, including every node pool and AKS control plane, on the cloud provider side. The stopped AKS incurs zero cost. The Postgres database, which belongs to the environment, is not stopped.

Limitations

  • The capability to start and stop an AKS cluster is not available from the UI.

  • You cannot start or stop an AKS cluster from the UI.

  • You cannot manually trigger starting or stopping an AKS cluster from the cloud provider either by using the Azure CLI or the Azure portal.

    Manually triggering start and stop on the cloud provider does not synchronize the state of the AKS instance with Cloudera control plane. The AKS continues running and the CDW UI indicates a running state.

Impala Virtual Warehouse spills temporary data to S3 automatically

When you create an Impala Virtual Warehouse, Impala is now automatically configured to write temporary data to the S3 Data Lake bucket. No configuration is required to enable the spill to S3. However, you must configure an Impala policy for permission to the scratch location on the Data Lake bucket.

Kubernetes dashboard adds Azure support and has reached GA status

From AWS or Azure environments, you can use the K8S dashboard to view the state of resources, such as CPU and memory usage, see the status of pods, and download logs. The dashboard can provide insights into the performance and health of a CDW cluster.