Creating a Project with ML Runtimes Variants
Projects create an independent working environment to hold your code, configuration, and libraries for your analysis. This topic describes how to create a project with ML Runtimes variants in Cloudera Machine Learning.
- Go to Cloudera Machine Learning and on the left sidebar, click Projects.
- Click New Project.
- If you are a member of a team, from the drop-down menu, select the Account under which you want to create this project. If there is only one account on the deployment, you will not see this option.
- Enter a Project Name.
- Select Project Visibility from one of the following options.
- Private - Only project collaborators can view or edit the project.
- Team - If the project is created under a team account, all members of the team can view the project. Only explicitly-added collaborators can edit the project.
- Public - All authenticated users of Cloudera Machine Learning will be able to view the project. Collaborators will be able to edit the project.
- Under Initial Setup, you can either create a blank project, or select one of the
following sources for your project files.
Blank - The project will contain no information from a template, local file, or Git.
Templates - Template projects contain example code that can help you get started with Cloudera Machine Learning. They are available in R, Python, PySpark, and Scala. Using a template project is not required, but it helps you start using Cloudera Machine Learning right away.
Local - If you have an existing project on your local disk, use this option to upload compressed files or folders to Cloudera Machine Learning.
Git - If you already use Git for version control and collaboration, you can continue to do so with Cloudera Machine Learning. Specifying a Git URL will clone the project into Cloudera Machine Learning. If you use a Git SSH URL, your personal private SSH key will be used to clone the repository. This is the recommended approach. However, you must add the public SSH key from your personal Cloudera Machine Learning account to the remote Git hosting service before you can clone the project.
- If you would like to configure which Runtimes are available for this particular project,
complete the following:
- Scroll down to the Runtime setup section and enter the appropriate
- Use the Basic view to select ML Runtimes based on Kernel and add all Standard Edition ML Runtimes for that kernel.
- If you would like to include the GPU-enabled variant for your selected Runtimes, click Add GPU enabled Runtime variant.
- Use the Advanced view to add ML Runtimes based on a more detailed Editor, Kernel, Edition, and Version criteria.
- Scroll down to the Runtime setup section and enter the appropriate information:
- Click Create Project. After the project is created, you can see
your project files and the list of jobs defined in your project.
Note that as part of the project filesystem, Cloudera Machine Learning also creates the following
R node_modules *.pyc .* !.gitignore
- Set or verify the ML Runtimes settings for the project.
Within the selected project, you can modify the default engine configuration:
- In the left navigation bar, click Project Settings.
- Select the Runtime/Engine tab.
- Next to Default Engine, select ML Runtime.
- Click Save Engine.
Once switched to ML Runtimes, all modules (Sessions, Jobs, Models, etc.) for the project will configure using ML Runtimes instead of Legacy Engines. Setting Engine parameters for the project will no longer be possible.
Existing and running instances (for example, Jobs, Models or Applications) previously configured with a particular Engine configuration will keep their configuration until you change the related settings. If you want to change the engine configuration for existing and running instances, you will need to update those based on the new, Runtime-based settings.
project_body = cmlapi.CreateProjectRequest( name = "project_name", description = "project_description", default_project_engine_type = "ml_runtime", project_body.visibility = "public", # or "private" or "organization" template = "Python")
You also need to specify a runtime_identifier if this is used with an ml_runtime project. Obtain a list the runtimes with the following command:
For some more examples of commands related to projects, see: Using the Projects API.