Major features and updates for the Cloudera Machine Learning service on Private Cloud.
January 24, 2023
CML on Private Cloud, version 1.5.0, has the following updates.
- Kubernetes - Kubernetes is now supported on ECS 1.23 and OCP 4.10.
- ECS - External Docker Registry & Cloudera-Default-Docker Registry are now supported on new installations of Private Cloud 1.5.0, but not on workloads upgraded from previous Private Cloud versions.
- ML Runtime Addons - In Site Administration, Administrators can now change the status of Spark addons to “AVAILABLE”, “DEPRECATED” or “DISABLED”.
- Spark pushdown - Spark pushdown functionality only works with CDE 1.17 Runtime Addons.
- Model Registry - The Model Registry stores and manages machine learning models and associated metadata, such as the model's version, dependencies, and performance. The registry enables MLOps and facilitates the development, deployment, and maintenance of machine learning models in a production environment.
- External registry - The use of an external Docker registry for reading images is now supported. For information on registry options, see Docker repository access for OCP or for ECS. CML supports these options with some known limitations, see here for more information.
- Cloudera Data Science Workbench (CDSW) 1.10.0 or later to CML migration (Preview) - You can easily move your CDSW workloads to CML using this new migration software. The CDSW to CML migration software is in technical preview in CDP 1.5.0. Cloudera recommends that you use this process in test and development environments. It is not recommended for production deployments. For more information, see "Migrating Data Science Workbench to Machine Learning".
- Storage provisioner change On OCP, CephFS is used as the underlying storage
provisioner for any new internal workspace on PVC 1.5.0. A storage class named
"ocs-storagecluster-cephfs" with csi driver set to
"openshift-storage.cephfs.csi.ceph.com" must exist in the cluster for new internal
workspaces to get provisioned. Each workspace will have separate 1 TB internal storage.
On ECS, any new internal workspace on 1.5.0 will use Longhorn as the underlying storage provisioner. A storage class named "longhorn" with csi driver set to "driver.longhorn.io" must exist in the cluster for new internal workspaces to get provisioned. Each workspace will have separate 1 TB internal storage.
On either ECS or OCP, internal workspaces on PVC 1.4.0/1.4.1 use the NFS server provisioner as a storage provisioner. This server provisioner still works in 1.5.0, however, it is deprecated, and will be removed in 1.5.1.Existing workspaces in 1.4.1 need to be upgraded to 1.5.0. These workspaces use the older storage provisioner. You can do one of the following:
- Migrate the workspace to CephFS or Longhorn before 1.5.1 is released, or:
- Create a new 1.5.0 workspace, and migrate the workloads to that workspace now.
November 18, 2022
CML on Private Cloud, version 1.4.1, has the following updates.
- Legacy CDSW Cluster Detected - After the upgrade to Private Cloud 1.4.1, if the base cluster contains a CDSW installation, you will see a message recommending you to upgrade the cluster. Do NOT click Upgrade as this feature is not yet GA.
- HDFS transparent encryption - Encryption at rest (HDFS transparent encryption) is supported in CML.
June 22, 2022
CML on Private Cloud, version 1.4.0, has the following updates.
- Model metrics visualization - This feature allows Data Scientists and Machine Learning Engineers to monitor technical metrics relating to their running models, such as resource consumption and request throughput, within Cloudera Machine Learning.
- DSE-19937 - Fixed an issue where the pagination widget on the Session list page may not function as expected.
- DSE-20085 - Fixed a bug where Job report recipients who subscribed to notification emails when their jobs terminated, may receive notification emails for termination statuses that they did not subscribe to.
- DSE-19751Fixed a bug where projects may not be sorted correctly on the project list page when using the Created By field for sorting.
April 11, 2022
CML on Private Cloud, version 1.3.4, has the following updates.
- ML Runtimes - ML Runtimes are now supported in CML Private Cloud. For more information, see Managing ML Runtimes.
- Cloudera Data Visualization - Cloudera Data Visualization is now available in the default runtime.
- GPU Taint support - GPU taints, which affect node scheduling, are now supported for both OCP and ECS clusters. For more information, see GPU node setup.
January 13, 2022
CML on Private Cloud, version 1.3.3, has the following updates.
- Business User Experience - A new user role, ML BusinessUser, provides restricted access to view Applications created in CML.
- API v2 - A new API for operations on projects, jobs, models, and applications is now generally available.
- Upgrade - It is not possible to upgrade an existing ML workspace on an ECS cluster. You have to provision a new workspace.
- Upgrade - After upgrading the ECS control plane, model and experiment building in an ML workspace might fail. See the Known Issue for more information.
- Engines - Engine version 15-cml-2021.09-2 has been patched for CVE-2021-44228, the Apache Log4j2 vulnerability. You should use engine version 15-cml-2021.09-2 instead of version 15-cml-2021.09-1 wherever possible.
October 29, 2021
CML on Private Cloud, version 1.3.2, has the following updates.
- Upgrade - Fixed an issue so that upgrading a CML workspace with ML Governance enabled works.
October 4, 2021
CML on Private Cloud, version 1.3.1, has the following new features and updates.
- Embedded Container Service (ECS) is now supported.
- Installation - If ECS is installed using Cloudera Docker Registries, then CML Workspace Model and Experiment building is not supported.
- Upgrade - Upgrading a CML workspace with ML Governance enabled fails.
April 27, 2021
CML on Private Cloud, version 1.2, has the following new features and updates.
- Support for OCP 4.6 and upgrading from PVC 1.1.
- Improved non-transparent proxy support for air-gapped environments.
- Introduced Applied ML Prototypes (AMPs).
- Added NFS support:
- NFS versions v3 and v4.x are supported.
- External NFS security improvements -
no_root_squashexport option has been removed.
- Support added for custom service principals (Beta).
- Monitoring now uses CDP centralized Grafana. Added database metrics and improved alerts.
- DSE-12037 - Fixed an issue with the seamless login for Grafana.
- DSE-14891 - Fixed an issue with broken Engine and Session log links.
- Various security fixes.
December 16, 2020
CML on Private Cloud, version 1.1, has the following new features and updates.
- MLOPS-216 - Production ML Support
Model Metrics track machine model serving performance metrics. Model Governance use Apache Atlas to track builds, experiments and deployment of machine learning models.
- DSE-10777 - UMS Integration
MLUser and MLAdmin resource roles are now available and assignable through Environment settings.
- DSE-12955 - Self Signed Private CA certs For custom container registries
Customers can now use Container registries that are using self signed or private CA signed certificates. There is an option to upload the self signed or private CA signed certificates certificate during Private Cloud installation.
- DSE-10759 - GPU support
The OpenShift Nvidia operator is now supported for use with CML workloads.
August 17, 2020
This is the first release of CML on Private Cloud, version 1.0.
- Run Machine Learning workloads on OpenShift clusters in your own data center.
- Easily onboard a new tenant and provision an ML workspace in a shared OpenShift environment.
- Enable data scientists to access shared data on CDP Private Cloud Base and CDW.
- Leverage Spark-on-K8s to spin up and down Spark clusters on demand.
- Take advantage of most CML features on public cloud, including Teams, Projects, Experiments, Models, and Applications.