What's new

This section lists major features and updates for the Management Console service.

December 9, 2024

Compute Cluster enabled environments

The following changes have been introduced to compute cluster enabled environments:
  • Suspending and resuming Compute Clusters in Cloudera environments on AWS
For more information, see Suspending and resuming Compute Clusters.

October 29, 2024

Azure Single Server to Azure Flexible Server upgrade

PostgreSQL - Single Server on Microsoft Azure databases used by a Data Lakes and Data Hubs can now be upgraded to Azure Flexible Server. During the upgrade process from PosgtreSQL version 11 to PostgreSQL 14, Azure Single Server will be upgraded to Azure Flexible Server. For more information, see Upgrading Azure Single Server to Flexible Server.

Compute Cluster enabled environments

The following changes have been introduced to compute cluster enabled environments:
  • You can provide the Worker Node Subnets for compute cluster enabled environments on AWS and Azure
  • You can provide the AKS Private DNS Zone ID for compute cluster enabled environments on Azure
For more information, see Using compute cluster enabled environments on AWS and Using compute cluster enabled environments on Azure.

New configuration property for non-transparent proxy

Inbound Proxy CIDR has been introduced for configuring non-transparent proxy in Public Cloud environments to allow communication with the Kubernetes server when defining the proxy with FQDNs.

For more information, see Using a non-transparent proxy.

Receiving resource notifications

Notifications include automatically generated service and resource related alerts, such as cluster state changes and events, upgrade alerts, resource exhaustion and consumption notifications. Notifications can be received by users of a tenant who have subscribed to the resource events of a Cloudera service.

For more information, see the Receiving notifications documentation.

October 9, 2024

Support for Sweden Central region

The Sweden Central Azure region is now supported. You can register Azure environments and provision Data Hubs in that region. See updated Supported Azure regions.

September 26, 2024

This release of the Management Console service introduces the following changes:

Receiving announcements

You can subscribe to receive announcements and notifications in Cloudera Public Cloud about various events from product updates to data service specific alerts. Announcements include Cloudera product related announcements that can include updates related to End of Life (EOL), End of Support (EOS), Technical Service Bulletins (TSBs), and maintenance updates.

For more information, see the Receiving notifications documentation.

Compute Cluster enabled environments

Compute Clusters enable you to deploy a containerized platform on Kubernetes for Data Services and shared services. The Compute Cluster architecture offers simplified management, enhanced efficiency, and centralized control that leads to faster deployments, reduced configuration errors and improved system reliability. As multiple Data Services can optionally share the same Compute Cluster, it also lowers the cost of ownership.

For more information, see Using compute cluster enabled environments on AWS and Using compute cluster enabled environments on Azure.

September 3, 2024

This release of the Management Console service introduces the following changes:

Support for Private Link for Azure Flexible Server

Azure Database for PostgreSQL Flexible Server allows a highly available database to be deployed for Data Lake and Data Hub clusters. It is no longer required to use delegated subnets for Flexible Servers to be used with private access as Private Link for Azure Flexible Server is now supported in CDP.

For more information, see Using Azure Database for PostgreSQL Flexible Server and Private setup for Azure Flexible Server.

June 12, 2024

This release of the Management Console service introduces the following changes:

Support for Hyderabad and Calgary AWS regions

The Asia Pacific Hyderabad AWS region and the Calgary AWS region is now supported. You can register CDP environments and provision Data Hubs in that region. See updated Supported AWS regions.

Support for Madrid Google Cloud region

The Madrid Google Cloud region is now supported. You can register CDP environments and provision Data Hubs in that region. See updated Supported GCP regions.

May 15, 2024

This release of the Management Console service introduces the following changes:

Updating instance metadata to IMDSv2

CDP now uses IMDSv2 for accessing EC2 instance metadata on all newly created Data Lakes, FreeIPA clusters, and Data Hubs. Previously created clusters using IMDSv1 can now be updated to IMDSv2. For more information, see Updating instance metadata to IMDSv2.

April 18, 2024

This release of the Management Console service introduces the following changes:

Configuring a CMK for data encryption in Azure Database for PostgreSQL Flexible Server

You can optionally use a customer managed encryption key (CMK) for data encryption in the Azure Database for PostgreSQL Flexible Server database instance used by CDP. For more information, see Configuring a CMK for data encryption in Azure Database for PostgreSQL Flexible Server.

Disk vertical scaling in Azure (Preview)

Disk vertical scaling (that is, disk type change and resizing) is now supported by CDP for Data Lakes and Data Hubs running in Azure. Previously, only AWS support was available for this feature in CDP. For more information, see Disk Vertical Scaling - Disk Type Change and Resizing in AWS and Azure.

April 3, 2024

This release of the Management Console service introduces the following changes:

Runtime 7.2.18

Cloudera Runtime 7.2.18 is now available and can be used for registering an environment with a 7.2.18 Data Lake and creating Data Hub clusters. For more information about the new Runtime version, see Cloudera Runtime. If you need to upgrade your existing CDP environment, your upgrade path may be complex. To determine your upgrade path, refer to Upgrading to Runtime 7.2.18 documentation.

RHEL replaces CentOS as default OS

As of June 30, 2024, CentOS reaches End of Life (EOL), and consequently, Cloudera Runtime 7.2.18 supports RHEL 8 only. New deployments of Data Lakes and Data Hubs with Runtime 7.2.18 and upgrades to 7.2.18 are only possible with RHEL 8. Data Lake and Data Hub clusters running Runtime 7.2.17 support both CentOS 7 and RHEL 8. Earlier Runtime versions support CentOS 7 only. Cloudera will not publish any updates or fixes for CentOS-based images after June 2024.

As part of FreeIPA, Data Lake, and Data Hub upgrade, you have the option to upgrade the operating system (OS) on the virtual machines (VMs) from CentOS 7 to Red Hat Enterprise Linux 8 (RHEL 8). For more information, see Upgrading from CentOS to RHEL.

Discontinuation of Medium Duty Data Lake

Starting with Runtime 7.2.18, Medium Duty Data Lake is discontinued and is replaced by Enterprise Data Lake (EDL). In order for existing Data Lakes to be upgraded to Runtime 7.2.18, they must be using Enterprise or Light Duty Data Lake.

Enterprise Data Lakes are a redefined version of Medium Duty Data Lakes that still offer failure resilience, but utilize resources and allocate memory more efficiently than a Medium Duty Data Lake at the same cost. Enterprise Data Lakes can handle more intensive workloads than Medium Duty Data Lakes and when deployed in Multi-AZ mode, remain operational during an availability zone outage.

If you are using Medium Duty Data Lake and would like to upgrade to Runtime 7.2.18, you will first need to upgrade to 7.2.17 first, and then resize your Data Lake to Enterprise Data Lake. For more information, see Upgrading from Medium Duty to Enterprise Data Lake.

Support for Amazon S3 Express One Zone buckets

Starting with Runtime 7.2.18, CDP supports using S3 Express One Zone buckets for data storage. If you have additional data buckets that you would like to use for Data Hub workloads and you do not need zone redundancy, you may use S3 Express buckets, for example for faster processing of temporary data. For more information, see Using S3 Express One Zone for data storage.

Rolling upgrade support for the Data Lake

With the release of Runtime 7.2.18, rolling upgrades for certain Data Lakes are now available. Rolling upgrades for the Data Lake are limited to certain Data Lake Runtime versions and shapes. For more information, see Rolling upgrades.

March 14, 2024

This release of the Management Console service introduces the following changes:

RHEL 8 enabled by default

RHEL 8 is now used as a default operating system for all newly created Data Lake, FreeIPA, and Data Hubs clusters running Runtime 7.2.17 and newer.

Data Lake and Data Hub clusters running Runtime 7.2.17 support both CentOS 7 and RHEL 8. Earlier Runtime versions supported CentOS only, but Cloudera will not publish any updates or fixes for CentOS-based images after June 2024. Runtime 7.2.18 and newer only support RHEL.

The RHEL images, which are CIS Level 1 compliant, are provided with an embedded RHEL license that is restricted for Cloudera CDP Public Cloud use case. Cloudera’s RHEL repository connection is activated during CDP deployment and the OS is installed from that repository. Only limited OS packages are hosted in this RHEL repository, but additional OS packages can be requested by filing a support ticket with details on software name, version, and software URL (preferred) or a list of missing OS packages.

March 12, 2024

This release of the Management Console service introduces the following changes:

Modify list of AZs available for new Data Hubs

As an enhancement of the previously released functionality for launching CDP environments on Azure in multiple availability zones (AZ), you can now edit an existing environment to modify the availability zones for all newly created Data Hubs in that environment. See Deploying CDP in multiple Azure availability zones.

Data Lake root volume size was increased to 200 GB

Data Lake's root volume size was increased from 100 GB to 200 GB.

February 6, 2024

This release of the Management Console service introduces the following changes:

Tag filtering for CDP usage insights

You can now use tags to filter your usage insight based on user-level tags of clusters in your CDP environment. For more information, see CDP credit consumption and usage insights.

January 24, 2024

This release of the Management Console service introduces the following changes:

Deploying CDP in multiple Azure availability zones

You can now deploy your CDP environment, enterprise Data Lake and Data Hubs on Azure across multiple availability zones. This is an optional configuration that is not enabled by default. For more information, see Deploying CDP in multiple Azure availability zones.

January 5, 2024

This release of the Management Console service introduces the following changes:

Azure Database for PostgreSQL Flexible Server

In this release, CDP introduces Azure Database for PostgreSQL Flexible Server. New Azure environments automatically use the Flexible Server with public endpoints but as a best practice for production you should configure Private Flexible Server setup.

With the release of this feature, you must add the following permissions on the scope of the single resource group to your custom role:
"Microsoft.DBforPostgreSQL/flexibleServers/read",
"Microsoft.DBforPostgreSQL/flexibleServers/write",
"Microsoft.DBforPostgreSQL/flexibleServers/delete",
"Microsoft.DBforPostgreSQL/flexibleServers/start/action",
"Microsoft.DBforPostgreSQL/flexibleServers/stop/action",
"Microsoft.DBforPostgreSQL/flexibleServers/firewallRules/write"

For more information, see Using Azure Database for PostgreSQL Flexible Server.

Support for Azure Qatar Central region

CDP now supports launching environments and Data Hubs in Azure Qatar Central region. See Supported Azure regions.