Using Conda Runtime
Users now can create their own Python or R Conda environments within their Cloudera Machine Learning Projects that they can use in the JupyterLab editor.
Environments, installed packages, and kernel specifications are persisted on the Project file system. An example flow for creating a new Python 3.10 Conda environment is the following:
conda create --name myenv python=3.10
conda activate myenv
conda install ipykernel
ipython kernel install --user --name=myenv
JupyterLab Conda Technical Preview Runtime
You might run into some known issues while using JupyterLab Conda Runtime.
- Sessions
- When starting a Notebook or a Console for a specific environment, the installed
packages will be available and the interpreter used to evaluate the contents of the
Notebook or Console will be the one installed in the environment. However, the Conda
environment is not "activated" in these sessions, therefore commands like
!which python
will return with the base Python 3.10 interpreter on the Runtime. The recommended ways to modify a Conda environments or install packages are the following:- conda commands must be used with the
-n
or--name
argument to specify the environment, for exampleconda -n myenv install pandas
- When installing packages with pip, use the
%pip
magic to install packages in the active kernel’s environment, for example%pip install pandas
- conda commands must be used with the
- Applications and Jobs
- To start an Application or Job, first create a launcher Python script containing the
following line:
!source activate <conda_env_name> && python <job / application script.py>
- Models
- Models are currently not supported for the Conda Runtime.
- Spark
- Spark is not supported in JupyterLab Notebooks and Consoles.