What's new

Support for Hyderabad and Calgary AWS regions

The Asia Pacific Hyderabad AWS region and the Calgary AWS region is now supported. You can register CDP environments and provision Data Hubs in that region. See updated Supported AWS regions.

Support for Madrid Google Cloud region

The Madrid Google Cloud region is now supported. You can register CDP environments and provision Data Hubs in that region. See updated Supported GCP regions.

Updating instance metadata to IMDSv2

CDP now uses IMDSv2 for accessing EC2 instance metadata on all newly created Data Lakes, FreeIPA clusters, and Data Hubs. Previously created clusters using IMDSv1 can now be updated to IMDSv2. For more information, see Updating instance metadata to IMDSv2.

Configuring a CMK for data encryption in Azure Database for PostgreSQL Flexible Server

You can optionally use a customer managed encryption key (CMK) for data encryption in the Azure Database for PostgreSQL Flexible Server database instance used by CDP. For more information, see Configuring a CMK for data encryption in Azure Database for PostgreSQL Flexible Server.

Disk vertical scaling in Azure (Preview)

Runtime 7.2.18

Cloudera Runtime 7.2.18 is now available and can be used for registering an environment with a 7.2.18 Data Lake and creating Data Hub clusters. For more information about the new Runtime version, see Cloudera Runtime. If you need to upgrade your existing CDP environment, your upgrade path may be complex. To determine your upgrade path, refer to Upgrading to Runtime 7.2.18 documentation.

RHEL replaces CentOS as default OS

As of June 30, 2024, CentOS reaches End of Life (EOL), and consequently, Cloudera Runtime 7.2.18 supports RHEL 8 only. New deployments of Data Lakes and Data Hubs with Runtime 7.2.18 and upgrades to 7.2.18 are only possible with RHEL 8. Data Lake and Data Hub clusters running Runtime 7.2.17 support both CentOS 7 and RHEL 8. Earlier Runtime versions support CentOS 7 only. Cloudera will not publish any updates or fixes for CentOS-based images after June 2024.

As part of FreeIPA, Data Lake, and Data Hub upgrade, you have the option to upgrade the operating system (OS) on the virtual machines (VMs) from CentOS 7 to Red Hat Enterprise Linux 8 (RHEL 8). For more information, see Upgrading from CentOS to RHEL.

Discontinuation of Medium Duty Data Lake

Support for Amazon S3 Express One Zone buckets

Starting with Runtime 7.2.18, CDP supports using S3 Express One Zone buckets for data storage. If you have additional data buckets that you would like to use for Data Hub workloads and you do not need zone redundancy, you may use S3 Express buckets, for example for faster processing of temporary data. For more information, see Using S3 Express One Zone for data storage.

Rolling upgrade support for the Data Lake

With the release of Runtime 7.2.18, rolling upgrades for certain Data Lakes are now available. Rolling upgrades for the Data Lake are limited to certain Data Lake Runtime versions and shapes. For more information, see Rolling upgrades.

RHEL 8 enabled by default

RHEL 8 is now used as a default operating system for all newly created Data Lake, FreeIPA, and Data Hubs clusters running Runtime 7.2.17 and newer.

Data Lake and Data Hub clusters running Runtime 7.2.17 support both CentOS 7 and RHEL 8. Earlier Runtime versions supported CentOS only, but Cloudera will not publish any updates or fixes for CentOS-based images after June 2024. Runtime 7.2.18 and newer only support RHEL.

The RHEL images, which are CIS Level 1 compliant, are provided with an embedded RHEL license that is restricted for Cloudera CDP Public Cloud use case. Cloudera’s RHEL repository connection is activated during CDP deployment and the OS is installed from that repository. Only limited OS packages are hosted in this RHEL repository, but additional OS packages can be requested by filing a support ticket with details on software name, version, and software URL (preferred) or a list of missing OS packages.

Modify list of AZs available for new Data Hubs

As an enhancement of the previously released functionality for launching CDP environments on Azure in multiple availability zones (AZ), you can now edit an existing environment to modify the availability zones for all newly created Data Hubs in that environment. See Deploying CDP in multiple Azure availability zones.

Data Lake root volume size was increased to 200 GB

Data Lake's root volume size was increased from 100 GB to 200 GB.

Tag filtering for CDP usage insights

You can now use tags to filter your usage insight based on user-level tags of clusters in your CDP environment. For more information, see CDP credit consumption and usage insights.

Deploying CDP in multiple Azure availability zones

You can now deploy your CDP environment, enterprise Data Lake and Data Hubs on Azure across multiple availability zones. This is an optional configuration that is not enabled by default. For more information, see Deploying CDP in multiple Azure availability zones.

Azure Database for PostgreSQL Flexible Server

In this release, CDP introduces Azure Database for PostgreSQL Flexible Server. New Azure environments automatically use the Flexible Server with public endpoints but as a best practice for production you should configure Private Flexible Server setup.

"Microsoft.DBforPostgreSQL/flexibleServers/read",
"Microsoft.DBforPostgreSQL/flexibleServers/write",
"Microsoft.DBforPostgreSQL/flexibleServers/delete",
"Microsoft.DBforPostgreSQL/flexibleServers/start/action",
"Microsoft.DBforPostgreSQL/flexibleServers/stop/action",
"Microsoft.DBforPostgreSQL/flexibleServers/firewallRules/write"

For more information, see Using Azure Database for PostgreSQL Flexible Server.

Support for Azure Qatar Central region

CDP now supports launching environments and Data Hubs in Azure Qatar Central region. See Supported Azure regions.