December 1, 2021

New features and enhancements

[Technical Preview] CDF can now be deployed on Azure

Deploying CDF on Azure requires a specific entitlement that is granted by request only. Reach out to your Cloudera Account Representative or open a support ticket to request CDF on Azure.

Custom Processors and Controller Services can be used in NiFi flow deployments.

CDF supports deploying NiFi flows that require custom components such as processors or controller services. Users can store their custom NiFi Archive (NAR) files in an object store location in S3 or ADLS and use the Deployment Wizard or the CLI to create their flow deployments accordingly. Once configured, CDF ensures that the custom NARs are available to all NiFi pods as the deployment scales up and down.

Non-transparent proxy support in AWS environments

If you have a non-transparent proxy registered for your CDP environment, CDF reuses the existing configuration and routes its outbound traffic through the proxy.

Private EKS/AKS Kubernetes API Endpoints

CDF now supports private EKS/AKS Kubernetes API Endpoints for CDP environments which have been setup using Cloudera Connectivity Manager (CCM v2). When enabling CDF, you need to select a checkbox to create a private endpoint.

Enabling CDF for an environment with a minimum of one EKS/AKS instance

CDF can now be enabled with a minimum of one EKS/AKS instance and will scale up as required until the specified maximum number of Kubernetes nodes is reached. Previously the minimum number of nodes was set to three.

CDF instance sizing changes

CDF now uses c5.4xlarge instances on AWS and Standard_D16s_v4 on Azure to provide more efficient infrastructure usage.

NiFi node sizing changes

NiFi Node Sizes in the Deployment Wizard have also been updated to use available resources more efficiently. The new options are:

• Extra Small – 2 vCores, 4GB memory

• Small – 3 vCores, 6GB memory

• Medium – 6 vCores, 12GB memory

• Large – 12 vCores, 24GB memory

Flow deployment security certificates

Security certificates for deployments are now automatically renewed one month before their expiration. A manual restart is required to pick up the new certificates.

CDF UI guidance on using CDP CLI

The CDF UI now provides guidance on how to use the CDP CLI to enable and disable CDF, upload and delete flow definitions as well as editing existing flow deployments.

CDP Beta CLI support for editing flow deployments

The CDP Beta CLI now includes functionality to edit existing deployments allowing configuration changes such as parameter and KPI updates, scaling configuration, or NiFi version upgrades.

New events and alerts for pod scaling

Introduced deployment pod scaling alerts and events to reflect when a deployment auto scales and when scaling limits are approached.

Fixed issues

• CDPDFX-3568 – Fixed an issue where no tooltip data was shown for the Data Throughput charts in the Dashboard.

• CDPDFX-3588 – Fixed an issue where CDF did not write logging data to non us-west-1 S3 buckets.

• NIFI-9217 – Fixed an issue that prevented scale down operations due to NiFi nodes getting stuck in “offloading” mode.

• CDPDFX-3624 – Fixed an issue where deployment auto-scaling value changes were not saved.