What's new in ML Runtimes version 2025.01.1
Review the major features and updates for ML Runtimes.
This release is available with ML Runtimes version 2025.01.1.
New Features
- ML Runtimes' base image has been updated from Ubuntu 20.04 to Ubuntu 24.04. This update enables many new features, a better experience for end users, faster and better fixes of CVEs and enhanced security to prevent malicious activities.
- The new ML Runtimes are not drop-in replacements of the previously released ML Runtimes. If you install any additional software into Cloudera AI projects, it might result in issues and might require reinstallation or recompilation.
- CUDA images are based on
nvidia/cuda:12.5.1-devel-ubuntu24.04
. The CUDA version on the Nvidia GPU Runtimes has also been also updated. - All Runtime images are now PBJ-based (Powered by Jupyter); Workbench-based Runtime images are no longer released.
Behavioral changes
- Python changes
- Workbench-based Python images have been removed, only PBJ Workbench Python images and JupyterLab Runtime images will be released going forward.
-
Python 3.7- and 3.8-based images are no longer supported.
- All ML Runtime images have updated PIP (Python package manager) package versions.
- JupyterLab changes
- JupyterLab has been migrated to PBJ (Powered by Jupyter).
node.js
has been removed.
- R changes
- Non-PBJ Workbench Runtime images are no longer available.
- Python has been updated to Python version 3.12.
- Scala changes
- Scala has been migrated to PBJ (Powered by Jupyter).
- Python has been updated to Python 3.12.
- Scala image has been updated to support Spark 3.x.
- Spark version 2.x is no longer supported (starting a Spark 2.x session will fail with an error).
- Scala has been updated to version 2.12. The Scala update might bring backward compatibility issues.
- When building a custom PBJ Scala Runtime image either use the Cloudera PBJ Scala Runtime or start from the Cloudera-provided public Dockerfiles. The custom PBJ Scala Runtime image provided by Cloudera contains necessary updates, such as the Scala chunker on the Runtime image and other necessary enhancements.
- Conda changes
- Conda has been migrated to PBJ (Powered by Jupyter).
- Python has been updated to 3.12.
- To ensure the compatibility of AMPs with ML Runtimes 2025.01.1 using JupyterLab, you need to proceed with
either of the following options:
- Switch to PBJ Workbench in the AMPs’ .project-metadata.yaml file.:
runtimes: - editor: PBJ Workbench
-
Use jobs instead of sessions for automated tasks.
Sessions are intended for interactive use, while AMP setup steps are executed in a programmed manner or automatically. Instead of running a session, change a session-based task into a job, create and execute a job.
Instead of running a session:tasks: - type: run_session name: Install Dependencies script: scripts/install_dependencies.py memory: 2 cpu: 1
Use the following example entry in the .project-metadata.yaml file for creating and executing a job:tasks: - type: create_job name: Install Dependencies entity_label: install_deps script: scripts/install_dependencies.py arguments: None cpu: 1 memory: 2 short_summary: Create job to install project dependencies. environment: TASK_TYPE: CREATE/RUN_JOB - type: run_job entity_label: install_deps short_summary: Running install dependencies job. long_summary: >- Running the job to install dependencies.
- Switch to PBJ Workbench in the AMPs’ .project-metadata.yaml file.:
- The
cdsw
Python module has been replaced by thecml
module.
Improvements
- Cloudera Copilot enhancements:
- Support for default embedding models.
- Support for additional Amazon Bedrock models: Claude v3.5 Haiku, Claude v3.5 Sonnet, Llama 3.1 405b Instruct, and Mistral Large 24.02.
- Increase in maximum tokens limit to 2048 for Cloudera AI Inference model's responses.
-
Python maintenance versions have been upgraded as follows:
- 3.9.19 -> 3.9.21
- 3.10.14 -> 3.10.16
- 3.11.9 -> 3.11.11
- 3.12.4 -> 3.12.8
Fixed issues
-
R workloads' .Libpaths() changed
Earlier the order of R workloads' .Libpaths() was:- Cloudera AI project
- CMLAddon
- Runtime image
Now R workloads that are running PBJ Runtimes have the .Libpaths() set up in this order:- Cloudera AI project
- Runtime image
- CMLAddon
Reinstall your R packages in your projects if the above detailed change of order in R workload's library path might cause any issues.