What's new

This section lists major features and updates for the Management Console service.

May 15, 2024

This release of the Management Console service introduces the following changes:

Updating instance metadata to IMDSv2

CDP now uses IMDSv2 for accessing EC2 instance metadata on all newly created Data Lakes, FreeIPA clusters, and Data Hubs. Previously created clusters using IMDSv1 can now be updated to IMDSv2. For more information, see Updating instance metadata to IMDSv2.

April 18, 2024

This release of the Management Console service introduces the following changes:

Configuring a CMK for data encryption in Azure Database for PostgreSQL Flexible Server

You can optionally use a customer managed encryption key (CMK) for data encryption in the Azure Database for PostgreSQL Flexible Server database instance used by CDP. For more information, see Configuring a CMK for data encryption in Azure Database for PostgreSQL Flexible Server.

Disk vertical scaling in Azure (Preview)

Disk vertical scaling (that is, disk type change and resizing) is now supported by CDP for Data Lakes and Data Hubs running in Azure. Previously, only AWS support was available for this feature in CDP. For more information, see Disk Vertical Scaling - Disk Type Change and Resizing in AWS and Azure.

April 3, 2024

This release of the Management Console service introduces the following changes:

Runtime 7.2.18

Cloudera Runtime 7.2.18 is now available and can be used for registering an environment with a 7.2.18 Data Lake and creating Data Hub clusters. For more information about the new Runtime version, see Cloudera Runtime. If you need to upgrade your existing CDP environment, your upgrade path may be complex. To determine your upgrade path, refer to Upgrading to Runtime 7.2.18 documentation.

RHEL replaces CentOS as default OS

As of June 30, 2024, CentOS reaches End of Life (EOL), and consequently, Cloudera Runtime 7.2.18 supports RHEL 8 only. New deployments of Data Lakes and Data Hubs with Runtime 7.2.18 and upgrades to 7.2.18 are only possible with RHEL 8. Data Lake and Data Hub clusters running Runtime 7.2.17 support both CentOS 7 and RHEL 8. Earlier Runtime versions support CentOS 7 only. Cloudera will not publish any updates or fixes for CentOS-based images after June 2024.

As part of FreeIPA, Data Lake, and Data Hub upgrade, you have the option to upgrade the operating system (OS) on the virtual machines (VMs) from CentOS 7 to Red Hat Enterprise Linux 8 (RHEL 8). For more information, see Upgrading from CentOS to RHEL.

Discontinuation of Medium Duty Data Lake

Starting with Runtime 7.2.18, Medium Duty Data Lake is discontinued and is replaced by Enterprise Data Lake (EDL). In order for existing Data Lakes to be upgraded to Runtime 7.2.18, they must be using Enterprise or Light Duty Data Lake.

Enterprise Data Lakes are a redefined version of Medium Duty Data Lakes that still offer failure resilience, but utilize resources and allocate memory more efficiently than a Medium Duty Data Lake at the same cost. Enterprise Data Lakes can handle more intensive workloads than Medium Duty Data Lakes and when deployed in Multi-AZ mode, remain operational during an availability zone outage.

If you are using Medium Duty Data Lake and would like to upgrade to Runtime 7.2.18, you will first need to upgrade to 7.2.17 first, and then resize your Data Lake to Enterprise Data Lake. For more information, see Upgrading from Medium Duty to Enterprise Data Lake.

Support for Amazon S3 Express One Zone buckets

Starting with Runtime 7.2.18, CDP supports using S3 Express One Zone buckets for data storage. If you have additional data buckets that you would like to use for Data Hub workloads and you do not need zone redundancy, you may use S3 Express buckets, for example for faster processing of temporary data. For more information, see Using S3 Express One Zone for data storage.

Rolling upgrade support for the Data Lake

With the release of Runtime 7.2.18, rolling upgrades for certain Data Lakes are now available. Rolling upgrades for the Data Lake are limited to certain Data Lake Runtime versions and shapes. For more information, see Rolling upgrades.

March 14, 2024

This release of the Management Console service introduces the following changes:

RHEL 8 enabled by default

RHEL 8 is now used as a default operating system for all newly created Data Lake, FreeIPA, and Data Hubs clusters running Runtime 7.2.17 and newer.

Data Lake and Data Hub clusters running Runtime 7.2.17 support both CentOS 7 and RHEL 8. Earlier Runtime versions supported CentOS only, but Cloudera will not publish any updates or fixes for CentOS-based images after June 2024. Runtime 7.2.18 and newer only support RHEL.

The RHEL images, which are CIS Level 1 compliant, are provided with an embedded RHEL license that is restricted for Cloudera CDP Public Cloud use case. Cloudera’s RHEL repository connection is activated during CDP deployment and the OS is installed from that repository. Only limited OS packages are hosted in this RHEL repository, but additional OS packages can be requested by filing a support ticket with details on software name, version, and software URL (preferred) or a list of missing OS packages.

March 12, 2024

This release of the Management Console service introduces the following changes:

Modify list of AZs available for new Data Hubs

As an enhancement of the previously released functionality for launching CDP environments on Azure in multiple availability zones (AZ), you can now edit an existing environment to modify the availability zones for all newly created Data Hubs in that environment. See Deploying CDP in multiple Azure availability zones.

Data Lake root volume size was increased to 200 GB

Data Lake's root volume size was increased from 100 GB to 200 GB.

February 6, 2024

This release of the Management Console service introduces the following changes:

Tag filtering for CDP usage insights

You can now use tags to filter your usage insight based on user-level tags of clusters in your CDP environment. For more information, see CDP credit consumption and usage insights.

January 24, 2024

This release of the Management Console service introduces the following changes:

Deploying CDP in multiple Azure availability zones

You can now deploy your CDP environment, enterprise Data Lake and Data Hubs on Azure across multiple availability zones. This is an optional configuration that is not enabled by default. For more information, see Deploying CDP in multiple Azure availability zones.

January 5, 2024

This release of the Management Console service introduces the following changes:

Azure Database for PostgreSQL Flexible Server

In this release, CDP introduces Azure Database for PostgreSQL Flexible Server. New Azure environments automatically use the Flexible Server with public endpoints but as a best practice for production you should configure Private Flexible Server setup.

With the release of this feature, you must add the following permissions on the scope of the single resource group to your custom role:
"Microsoft.DBforPostgreSQL/flexibleServers/read",
"Microsoft.DBforPostgreSQL/flexibleServers/write",
"Microsoft.DBforPostgreSQL/flexibleServers/delete",
"Microsoft.DBforPostgreSQL/flexibleServers/start/action",
"Microsoft.DBforPostgreSQL/flexibleServers/stop/action",
"Microsoft.DBforPostgreSQL/flexibleServers/firewallRules/write"

For more information, see Using Azure Database for PostgreSQL Flexible Server.

Support for Azure Qatar Central region

CDP now supports launching environments and Data Hubs in Azure Qatar Central region. See Supported Azure regions.