2020

New functionality

• Data Lake metadata backup and restore: You can backup and restore the metadata maintained in the Data Lake services. Data Lake backup and restore is supported from Cloudera Runtime 7.2.1+ on AWS and Cloudera Runtime 7.2.2+ on Azure. See Backup and restore for the Data Lake.

Changed functionality

• The option to set tenant-level tags was moved from Environments > Shared Resources > Tags to a new location under Global Settings > Tags.
• The option to enable workload analytics and deployments cluster logs collection on a tenant-level was moved from Environments > Shared Resources > Telemetry to a new location under Global Settings > Telemetry.

New documentation

Updated documentation

• Azure permissions
• Renew host certificates on Data Lake and Data Hub clusters

Deprecated functionality

The following account roles are deprecated:

• EnvironmentAdmin
• EnvironmentUser

New documentation

Updated documentation

New features

New documentation

Updated documentation

New features

Behavioral changes

• The option to stop and restart a Data Lake was removed. You should stop and restart your whole environment instead, as described in Stop and restart an environment.
• The option to delete a Data Lake was removed from the Actions menu on the environment details page. When deleting an environment from the CDP UI, you no longer need to delete the environment's Data Lake separately. See updated Delete an environment documentation.
• The option to Check for Data Lake Upgrade was removed.
• The options to Enable Workload Analytics and Enable Cluster Logs Collection were removed from the Actions menu on the environments details page. You can now control these options from the Management Console > Environments > Shared Resources > Telemetry (tenant-level setting), or you can set them during environment registration (environment-level setting). For updated instructions, see Enable workload analytics and cluster logs collection.
• The option to Get FreeIPA certificate were removed from the environment's Actions menu and is now available from FreeIPA's Actions menu. To find the option, navigate to the Management Console > Environments > navigate to a specific environment > Summary > FreeIPA > Actions > Get FreeIPA certificate.

New documentation

SSH key management

A Power User can add and delete SSH keys for all users and users can add and delete their own SSH keys. For more information, see Managing SSH keys.

Choose Data Lake version at deployment

When deploying an environment, you can now select the Data Lake version. See updated instructions for Register an AWS environment and Register an Azure environment..

Edit parameters of a previously deployed environment

You can edit the following environment components:

CDP Control Plane Public API Reference

CDP Control Plane now features reference information for APIs located at CDP Control Plane Public API Documentation.You can use this reference documentation to look up descriptions and syntax for SDK functions and CLI commands.

Groups sync disabled by default

When you create a Group in CDP, the Sync Membership flag is now off by default. This will eliminate the situation when users would be removed from a CDP group if they weren’t part of the same group in the ID Provider.

Support for HDP 2.6.x Classic Clusters using CCM

Support for registering Classic clusters using the reverse SSH mechanism is extended to legacy HDP 2.6.x clusters. This will be behind the CLASSIC_CLUSTERS entitlement.

Auditing Control Plane activity

Auditing is used to collect or log evidence of activity in a system that auditors can use to both track and analyze to answer questions such as: Who made a change to the system? When did a change happen? What exactly changed? Why was a change authorized?

Edit subnets

You can now add new subnets to the configuration of a running environment. This might be useful when you add a new service such as Cloudera Machine Learning (CML) and Clouder Data Warehouse (CDW) which require their own subnet.

Improvements in Data Hub cluster scaling

A number of significant improvements were made to properly clean up cloud resources for scaling up and down Data Hub clusters. This will reduce cloud costs that would have been incurred by any orphaned instances.

Support for connecting to private subnets

CDP supports communicating with Data Lake and Data Hub workload clusters that are on private subnets. environments in Azure.

You can use Cluster Connectivity Manager (CCM) to communicate with Data Lake and Data Hub workload clusters that are on private subnets. Communication takes place over private IPs without any inbound network access rules required. CDP requires that these clusters have outbound connections to AWS NLBs hosted in Cloudera's AWS account. Workload clusters initiate an SSH tunnel to the CDP control plane, which is then used for all communication thereafter.

Proxy server (technical preview)

Proxy server works with DataLake and Datahub in both transparent and non-transparent mode. The proxy server works with Cloudera Machine Learning (CML) and Cloudera Data Warehouse (CDW) in transparent mode only. Proxy server does not currently work with CML and CDW in non-transparent proxy mode.

You can use a proxy server to control the connections that are allowed from your VPC or VNet and prevent unattended connections initiated from your environment.

Tagging CDP resources

When you create an environment or other resources shared across your cloud provider account, CDP automatically adds default tags to the Cloudera-created resources in your cloud provider account. You can also define additional custom tags at either the environment level or tenant level.

Creating environments in Azure

CDP supports creating environments in Azure.

Register your Azure account and then launch Data Hub clusters, Data Warehouse clusters, and Machine Learning workspaces within the Azure environment and determine which users have access to which resources. These clusters are attached to a Data Lake that runs within the environment and provides security and governance for all attached clusters.