Initialize an SSH connection to Cloudera AI for VS code
The following task describes how to establish an SSH endpoint for Cloudera AI. Creating an SSH endpoint is the first step to configuring a remote editor for Cloudera AI.
-
Log in to Cloudera AI with the CLI client using the
cdswctl login command.
cdswctl login -n [***USERNAME***] -u [***http(s)://your_cloudera_ai_workbench_url***]- Replace
[***USERNAME***]with your Cloudera AI username. - Replace
http(s)://your_cloudera_ai_workbench_urlwith the actual URL of your Cloudera AI Workbench deployment.
sample_userto the deployment located athttps://ml-dd7ab8-e7b.exu2-8y8x.cdp.cldr.com/home, you would usecdswctl login -n sample_user -u https://ml-dd7ab8-e7b.exu2-8y8x.cdp.cldr.com/homeIf your workspace uses Single Sign-On (SSO), then add your API key using the -y flag. For example:cdswctl login -n sample_user -y api_key -u https://ml-dd7ab8-e7b.exu2-8y8x.cdp.cldr.com/home - Replace
-
Create a local SSH endpoint to Cloudera AI.
Run the following command:
cdswctl ssh-endpoint -p [***USERNAME***]/[***PROJECT NAME***] [-c [***CPU CORES***]] [-m [***MEMORY IN GB***]] [-g [***NUMBER OF GPUs***]] [-r [***RUNTIME ID***] ]If the project is configured to use ML runtimes, the
-rparameter must be specified, otherwise it must be omitted. To retrieve the Runtime ID, use the following command:cdswctl runtimes listSee Using ML runtimes with cdswctl documentation page for more information.
The command uses the following defaults for optional parameters:- CPU cores: 1
- Memory: 1 GB
- GPUs: 0
For example, the following command starts a session for the logged-in usersample_userunder thecustomerchurnproject with .5 cores, .75 GB of memory, 0 GPUs, and the Python3 kernel:cdswctl ssh-endpoint -p customerchurn -c 0.5 -m 0.75To create an SSH endpoint in a project owned by another user or a team, for example
finance, prepend the username to the project and separate them with a forward slash:
This command creates session in the projectcdswctl ssh-endpoint -p finance/customerchurn -c 0.5 -m 0.75customerchurnthat belongs to the teamfinance.Information for the SSH endpoint appears in the output:... You can SSH to it using ssh -p [***PORT***] cdsw@localhost ... -
Open a new command prompt and run the outputted command from the previous step:
For example:ssh -p [***PORT***] cdsw@localhost
You will be prompted for the passphrase for the SSH key you entered in the Cloudera AI web UI.ssh -p 7847 cdsw@localhostThe public key could be rejected when the newsshkey pair is generated with a special name such asid_rsa_systest. If the public key is rejected, you must add the following information to the ~/.ssh/config file:Host * AddKeysToAgent yes StrictHostKeyChecking no IdentityFile ~/.ssh/id_rsa_cdswctlOnce you are connected to the endpoint, you are logged in as thecdswuser and can perform actions as though you are accessing the terminal through the Cloudera AI web UI. -
Test the connection.
If you run
ls, the project files associated with the session you created are shown. If you runwhoami, the command returns thecdswuser.Once you are connected, you should see something like this:$ cdswctl ssh-endpoint -p ml-at-scale -m 4 -c 2 Forwarding local port 7847 to port 2222 on session bhsb7k4eqmonap62 in project finance/customerchurn. You can SSH to the session using ssh -p 7847 cdsw@localhost -
Add an entry into your SSH config file.
For example:
$ cat ~/.ssh/config Host cdsw-public HostName localhost IdentityFile ~/.ssh/id_rsa User cdsw Port 7847HostName is alwayslocalhostand User is alwayscdsw. You get the Port number from Step 2.
