Cloudera Docs

What's New

Major features and updates for the Cloudera Machine Learning service on Private Cloud.

January 24, 2023

CML on Private Cloud, version 1.5.0, has the following updates.

New features and updates
  • Kubernetes - Kubernetes supports ECS 1.23 and OCP 4.10.
  • ECS - External Image Registry & Cloudera-Default-Docker Registry are now supported on new installations of Private Cloud 1.5.0, but not on workloads upgraded from previous Private Cloud versions.
  • External Image registry - The use of an external customer ImageDocker registry for reading images is now supported. For information on registry options, see Docker repository access for OCP or for ECS. CML supports these options with some known limitations, see here for more information.
  • Cloudera Data Science Workbench (CDSW) 1.10.0 or later to CML migration (Technical Preview) - You can easily move your CDSW workloads to CML using this new migration software. The CDSW to CML migration software is in technical preview in CDP 1.5.0. Cloudera recommends that you use this process in test and development environments. It is not recommended for production deployments. For more information, see "Migrating Data Science Workbench to Machine Learning".
  • ML Runtimes - Support for Custom Runtime Addons and PBJ Runtimes added.
  • ML Runtime Addons - In Site Administration, Administrators can now change the status of Spark addons to “AVAILABLE”, “DEPRECATED” or “DISABLED”.
  • Data Science and Machine Learning - Added support for Experiments v2 with MLflow, Monitoring for Applications and Models, and Model Registry (Technical Preview).
  • Spark pushdown (Technical Preview) - Spark 2 pushdown to CDP Base functionality is supported.
  • Model Registry - The Model Registry stores and manages machine learning models and associated metadata, such as the model's version, dependencies, and performance. The registry enables MLOps and facilitates the development, deployment, and maintenance of machine learning models in a production environment.

    Note: Model Registry is not supported with R models.

  • Internal NFS Storage options on OCP: CephFS is used as the underlying storage provisioner for any new workspace using internal NFS on PVC 1.5.0. A storage class named "ocs-storagecluster-cephfs" with csi driver set to "openshift-storage.cephfs.csi.ceph.com" must exist in the cluster for new internal workspaces to get provisioned. Each workspace will have separate 1 TB internal storage.
  • Internal NFS Storage options on ECS: Any new workspace using internal NFS on 1.5.0 will use Longhorn as the underlying storage provisioner. A storage class named "longhorn" with csi driver set to "driver.longhorn.io" must exist in the cluster for new internal workspaces to get provisioned. Each workspace will have separate 1 TB internal storage.
  • Upgrades from 1.4.x with workspaces using internal NFS: On either ECS or OCP, internal workspaces running on PVC 1.4.0/1.4.1 use NFS server provisioner as the storage provisioner. These workspaces when upgraded to 1.5.0 will continue to run with the same NFS server Provisioner. However, NFS server provisioner is deprecated now and will not be supported from the 1.5.1 release onwards.
    Existing workspaces in 1.4.0/1.4.1 can be upgraded to 1.5.0 from PVC UI. After this, you can do one of the following:
    • Migrate the 1.5.0 upgraded workspace from NFS server provisioner to Longhorn (ECS) / Cephfs (OCP) if you want to continue using the same workspace in PVC 1.5.1 as well
    • Create a new 1.5.0 workspace and migrate the existing workloads to that before 1.5.1 release.
  • CML Team to CDP Group Sync - This synchronizes groups from the CDP management console to the ML workspace. See Creating a Team for more information.

November 18, 2022

CML on Private Cloud, version 1.4.1, has the following updates.

New features and updates
  • Legacy CDSW Cluster Detected - After the upgrade to Private Cloud 1.4.1, if the base cluster contains a CDSW installation, you will see a message recommending you to upgrade the cluster. Do NOT click Upgrade as this feature is not yet GA.
  • HDFS transparent encryption - Encryption at rest (HDFS transparent encryption) is supported in CML.

June 22, 2022

CML on Private Cloud, version 1.4.0, has the following updates.

New features and updates
  • Model metrics visualization - This feature allows Data Scientists and Machine Learning Engineers to monitor technical metrics relating to their running models, such as resource consumption and request throughput, within Cloudera Machine Learning.
Fixed issues
  • DSE-19937 - Fixed an issue where the pagination widget on the Session list page may not function as expected.
  • DSE-20085 - Fixed a bug where Job report recipients who subscribed to notification emails when their jobs terminated, may receive notification emails for termination statuses that they did not subscribe to.
  • DSE-19751Fixed a bug where projects may not be sorted correctly on the project list page when using the Created By field for sorting.

April 11, 2022

CML on Private Cloud, version 1.3.4, has the following updates.

New features and updates
  • ML Runtimes - ML Runtimes are now supported in CML Private Cloud. For more information, see Managing ML Runtimes.
  • Cloudera Data Visualization - Cloudera Data Visualization is now available in the default runtime.
  • GPU Taint support - GPU taints, which affect node scheduling, are now supported for both OCP and ECS clusters. For more information, see GPU node setup.

January 13, 2022

CML on Private Cloud, version 1.3.3, has the following updates.

New features and updates
  • Business User Experience - A new user role, ML BusinessUser, provides restricted access to view Applications created in CML.
  • API v2 - A new API for operations on projects, jobs, models, and applications is now generally available.
Installation notes
  • Upgrade - It is not possible to upgrade an existing ML workspace on an ECS cluster. You have to provision a new workspace.
  • Upgrade - After upgrading the ECS control plane, model and experiment building in an ML workspace might fail. See the Known Issue for more information.
  • Engines - Engine version 15-cml-2021.09-2 has been patched for CVE-2021-44228, the Apache Log4j2 vulnerability. You should use engine version 15-cml-2021.09-2 instead of version 15-cml-2021.09-1 wherever possible.

October 29, 2021

CML on Private Cloud, version 1.3.2, has the following updates.

Installation notes
  • Upgrade - Fixed an issue so that upgrading a CML workspace with ML Governance enabled works.

October 4, 2021

CML on Private Cloud, version 1.3.1, has the following new features and updates.

New features and updates
  • Embedded Container Service (ECS) is now supported.
Installation notes
  • Installation - If ECS is installed using Cloudera Docker Registries, then CML Workspace Model and Experiment building is not supported.
  • Upgrade - Upgrading a CML workspace with ML Governance enabled fails.

April 27, 2021

CML on Private Cloud, version 1.2, has the following new features and updates.

New features and updates
  • Support for OCP 4.6 and upgrading from PVC 1.1.
  • Improved non-transparent proxy support for air-gapped environments.
  • Introduced Applied ML Prototypes (AMPs).
  • Added NFS support:
    • NFS versions v3 and v4.x are supported.
    • External NFS security improvements - no_root_squash export option has been removed.
  • Support added for custom service principals (Beta).
  • Monitoring now uses CDP centralized Grafana. Added database metrics and improved alerts.
Bug fixes
  • DSE-12037 - Fixed an issue with the seamless login for Grafana.
  • DSE-14891 - Fixed an issue with broken Engine and Session log links.
  • Various security fixes.

December 16, 2020

CML on Private Cloud, version 1.1, has the following new features and updates.

  • MLOPS-216 - Production ML Support

    Model Metrics track machine model serving performance metrics. Model Governance use Apache Atlas to track builds, experiments and deployment of machine learning models.

  • DSE-10777 - UMS Integration

    MLUser and MLAdmin resource roles are now available and assignable through Environment settings.

  • DSE-12955 - Self Signed Private CA certs For custom container registries

    Customers can now use Container registries that are using self signed or private CA signed certificates. There is an option to upload the self signed or private CA signed certificates certificate during Private Cloud installation.

  • DSE-10759 - GPU support

    The OpenShift Nvidia operator is now supported for use with CML workloads.

August 17, 2020

This is the first release of CML on Private Cloud, version 1.0.

CML on Private Cloud lets you:
  • Run Machine Learning workloads on OpenShift clusters in your own data center.
  • Easily onboard a new tenant and provision an ML workspace in a shared OpenShift environment.
  • Enable data scientists to access shared data on CDP Private Cloud Base and CDW.
  • Leverage Spark-on-K8s to spin up and down Spark clusters on demand.
  • Take advantage of most CML features on public cloud, including Teams, Projects, Experiments, Models, and Applications.