Initialize an SSH Endpoint
This topic describes how to establish an SSH endpoint for Cloudera Machine Learning.
- Create a configuration file at:
$HOME/.cdsw/config.yaml
. The contents ofconfig.yaml
should be:username: <username> url: <ML_workspace_url> auth: authtype: 1 basic: null apikey: <your_legacy_api_key>
To collect the values for these fields, first log in to your Cloudera Machine Learning Workspace using SSO:username
: The username with which you are logged into the Cloudera Machine Learning Workspace. Found in the top right corner of your Cloudera Machine Learning Workspace.url
: The complete URL used to access the Cloudera Machine Learning Workspace. For example:https://ml-<randomly-generated-cluster-name>
apikey
: Go to User Settings > API Keys. Copy the value of the Legacy API Key to this field.
- Create a local SSH endpoint to Cloudera Machine Learning. Run
the following
command:
cdswctl ssh-endpoint -p <project_name> [-c <CPU_cores>] [-m <memory_in_GB>] [-g <number_of_GPUs>] [-r <runtime ID> ]
The command uses the following defaults for optional parameters:- CPU cores: 1
- Memory: 1 GB
- GPUs: 0
For example, the following command starts a session for the usermilton
under thecustomerchurn
project with .5 cores, .75 GB of memory, 0 GPUs, and the Python3 kernel:cdswctl ssh-endpoint -p customerchurn -c 0.5 -m 0.75
To create an SSH endpoint in a project owned by another user or a team, for example
finance
, prepend the username to the project and separate them with a forward slash:
This command creates session in the projectcdswctl ssh-endpoint -p finance/customerchurn -c 0.5 -m 0.75
customerchurn
that belongs to the teamfinance
.Information for the SSH endpoint appears in the output:... You can SSH to it using ssh -p <some_port> cdsw@localhost ...
- Open a new command prompt and run the outputted command from
the previous
step:
For example:ssh -p <some_port> cdsw@localhost
You will be prompted for the passphrase for the SSH key you entered in the Cloudera Machine Learning Workspace web UI.ssh -p 9750 cdsw@localhost
Once you are connected to the endpoint, you are logged in as thecdsw
user and can perform actions as though you are accessing the terminal through the web UI. - Test the connection.If you run
ls
, the project files associated with the session you created are shown. If you runwhoami
, the command returns thecdsw
user. - Leave the SSH endpoint running as long as you want to use a local IDE.