What's new

Cloudera Runtime 7.3.1

Cloudera Runtime 7.3.1 is now available and can be used for registering an environment with a 7.3.1 Data Lake and creating Cloudera Data Hub clusters. For more information about the new Cloudera Runtime version, see Cloudera Runtime. If you need to upgrade your existing Cloudera environment, your upgrade path may be complex. To determine your upgrade path, refer to Upgrading to Runtime 7.3.1 documentation.

Compute Cluster enabled environments

Azure Single Server to Azure Flexible Server upgrade

Single Server on Microsoft Azure databases used by Data Lakes and Cloudera Data Hub clusters can now be upgraded to Azure Flexible Server. During the upgrade process from PosgtreSQL version 11 to PostgreSQL 14, Azure Single Server will be upgraded to Azure Flexible Server. For more information, see Upgrading Azure Single Server to Flexible Server.

Compute Cluster enabled environments

New configuration property for non-transparent proxy

Inbound Proxy CIDR has been introduced for configuring non-transparent proxy in Cloudera Public Cloud environments to allow communication with the Kubernetes server when defining the proxy with FQDNs.

For more information, see Using a non-transparent proxy.

Receiving resource notifications

Notifications include automatically generated service and resource related alerts, such as cluster state changes and events, upgrade alerts, resource exhaustion and consumption notifications. Notifications can be received by users of a tenant who have subscribed to the resource events of a Cloudera service.

For more information, see the Receiving notifications documentation.

Support for Sweden Central region

The Sweden Central Azure region is now supported. You can register Azure environments and provision Cloudera Data Hub clusters in that region. See updated Supported Azure regions.

Receiving announcements

You can subscribe to receive announcements and notifications in Cloudera Public Cloud about various events from product updates to data service specific alerts. Announcements include product related announcements that can include updates related to End of Life (EOL), End of Support (EOS), Technical Service Bulletins (TSBs), and maintenance updates.

For more information, see the Receiving notifications documentation.

Compute Cluster enabled environments

Compute Clusters enable you to deploy a containerized platform on Kubernetes for Data Services and shared services. The Compute Cluster architecture offers simplified management, enhanced efficiency, and centralized control that leads to faster deployments, reduced configuration errors and improved system reliability. As multiple Data Services can optionally share the same Compute Cluster, it also lowers the cost of ownership.

For more information, see Using compute cluster enabled environments on AWS and Using compute cluster enabled environments on Azure.

Support for Private Link for Azure Flexible Server

Azure Database for PostgreSQL Flexible Server allows a highly available database to be deployed for Data Lake and Cloudera Data Hub clusters. It is no longer required to use delegated subnets for Flexible Servers to be used with private access as Private Link for Azure Flexible Server is now supported in Cloudera.

For more information, see Using Azure Database for PostgreSQL Flexible Server and Private setup for Azure Flexible Server.

Support for Hyderabad and Calgary AWS regions

The Asia Pacific Hyderabad AWS region and the Calgary AWS region is now supported. You can register Cloudera environments and provision Cloudera Data Hub clusters in that region. See updated Supported AWS regions.

Support for Madrid Google Cloud region

The Madrid Google Cloud region is now supported. You can register Cloudera environments and provision Cloudera Data Hub clusters in that region. See updated Supported GCP regions.

Updating instance metadata to IMDSv2

CDP now uses IMDSv2 for accessing EC2 instance metadata on all newly created Data Lakes, FreeIPA clusters, and Cloudera Data Hub clusters. Previously created clusters using IMDSv1 can now be updated to IMDSv2. For more information, see Updating instance metadata to IMDSv2.

Configuring a CMK for data encryption in Azure Database for PostgreSQL Flexible Server

You can optionally use a customer managed encryption key (CMK) for data encryption in the Azure Database for PostgreSQL Flexible Server database instance used by Cloudera. For more information, see Configuring a CMK for data encryption in Azure Database for PostgreSQL Flexible Server.

Disk vertical scaling in Azure (Preview)

Cloudera Runtime 7.2.18

Cloudera Runtime 7.2.18 is now available and can be used for registering an environment with a 7.2.18 Data Lake and creating Cloudera Data Hub clusters. For more information about the new Cloudera Runtime version, see Cloudera Runtime. If you need to upgrade your existing Cloudera environment, your upgrade path may be complex. To determine your upgrade path, refer to Upgrading to Runtime 7.2.18 documentation.

RHEL replaces CentOS as default OS

As of June 30, 2024, CentOS reaches End of Life (EOL), and consequently, Cloudera Runtime 7.2.18 supports RHEL 8 only. New deployments of Data Lakes and Cloudera Data Hub clusters with Cloudera Runtime 7.2.18 and upgrades to 7.2.18 are only possible with RHEL 8. Data Lake and Cloudera Data Hub clusters running Cloudera Runtime 7.2.17 support both CentOS 7 and RHEL 8. Earlier Cloudera Runtime versions support CentOS 7 only. Cloudera will not publish any updates or fixes for CentOS-based images after June 2024.

As part of the upgrade process of FreeIPA, Data Lake, and Cloudera Data Hub clusters, you have the option to upgrade the operating system (OS) on the virtual machines (VMs) from CentOS 7 to Red Hat Enterprise Linux 8 (RHEL 8). For more information, see Upgrading from CentOS to RHEL.

Discontinuation of Medium Duty Data Lake

Support for Amazon S3 Express One Zone buckets

Starting with Runtime 7.2.18, CDP supports using S3 Express One Zone buckets for data storage. If you have additional data buckets that you would like to use for Cloudera Data Hub cluster workloads and you do not need zone redundancy, you may use S3 Express buckets, for example for faster processing of temporary data. For more information, see Using S3 Express One Zone for data storage.

Rolling upgrade support for the Data Lake

With the release of Cloudera Runtime 7.2.18, rolling upgrades for certain Data Lakes are now available. Rolling upgrades for the Data Lake are limited to certain Cloudera Runtime versions and shapes of Data Lakes. For more information, see Rolling upgrades.

RHEL 8 enabled by default

RHEL 8 is now used as a default operating system for all newly created Data Lake, FreeIPA, and Cloudera Data Hub cluster running Cloudera Runtime 7.2.17 and newer.

Data Lake and Cloudera Data Hub clusters running Cloudera Runtime 7.2.17 support both CentOS 7 and RHEL 8. Earlier Cloudera Runtime versions supported CentOS only, but Cloudera will not publish any updates or fixes for CentOS-based images after June 2024. Cloudera Runtime 7.2.18 and newer only support RHEL.

The RHEL images, which are CIS Level 1 compliant, are provided with an embedded RHEL license that is restricted for Cloudera Public Cloud use case. Cloudera's RHEL repository connection is activated during Cloudera deployment and the OS is installed from that repository. Only limited OS packages are hosted in this RHEL repository, but additional OS packages can be requested by filing a support ticket with details on software name, version, and software URL (preferred) or a list of missing OS packages.

Modify list of AZs available for new Cloudera Data Hub clusters

As an enhancement of the previously released functionality for launching Cloudera environments on Azure in multiple availability zones (AZ), you can now edit an existing environment to modify the availability zones for all newly created Cloudera Data Hub clusters in that environment. See Deploying CDP in multiple Azure availability zones.

Data Lake root volume size was increased to 200 GB

Data Lake's root volume size was increased from 100 GB to 200 GB.

Tag filtering for Cloudera usage insights

You can now use tags to filter your usage insight based on user-level tags of clusters in your Cloudera environment. For more information, see CDP credit consumption and usage insights.

Deploying CDP in multiple Azure availability zones

You can now deploy your Cloudera environment, enterprise Data Lake and Cloudera Data Hub clusters on Azure across multiple availability zones. This is an optional configuration that is not enabled by default. For more information, see Deploying CDP in multiple Azure availability zones.

Azure Database for PostgreSQL Flexible Server

In this release, Cloudera introduces Azure Database for PostgreSQL Flexible Server. New Azure environments automatically use the Flexible Server with public endpoints but as a best practice for production you should configure Private Flexible Server setup.

"Microsoft.DBforPostgreSQL/flexibleServers/read",
"Microsoft.DBforPostgreSQL/flexibleServers/write",
"Microsoft.DBforPostgreSQL/flexibleServers/delete",
"Microsoft.DBforPostgreSQL/flexibleServers/start/action",
"Microsoft.DBforPostgreSQL/flexibleServers/stop/action",
"Microsoft.DBforPostgreSQL/flexibleServers/firewallRules/write"

For more information, see Using Azure Database for PostgreSQL Flexible Server.

Support for Azure Qatar Central region

Cloudera now supports launching environments and Cloudera Data Hub clusters in Azure Qatar Central region. See Supported Azure regions.