Initialize an SSH Endpoint

This topic describes how to establish an SSH endpoint for Cloudera Machine Learning.

Creating an SSH endpoint is also the first step to configuring a remote editor for an ML workspace.
  1. Create a configuration file at: $HOME/.cdsw/config.yaml. The contents of config.yaml should be:
    username: <username>
    url: <ML_workspace_url>
      authtype: 1
      basic: null
      apikey: <your_legacy_api_key>
    To collect the values for these fields, first log in to your CML workspace using SSO:
    • username: The username with which you are logged into the CML workspace. Found in the top right corner of your ML workspace.
    • url: The complete URL used to access the CML workspace. For example: https://ml-<randomly-generated-cluster-name>
    • apikey: Go to User Settings > API Keys. Copy the value of the Legacy API Key to this field.
  2. Create a local SSH endpoint to Cloudera Machine Learning. Run the following command:
    cdswctl ssh-endpoint -p <project_name> [-c <CPU_cores>] [-m <memory_in_GB>] [-g <number_of_GPUs>] [-r <runtime ID> ]

    If the project is configured to use ML runtimes, the -r parameter must be specified, otherwise it must be omitted. See Using ML runtimes with cdswctl documentation page for more information.

    The command uses the following defaults for optional parameters:
    • CPU cores: 1
    • Memory: 1 GB
    • GPUs: 0
    For example, the following command starts a session for the user milton under the customerchurn project with .5 cores, .75 GB of memory, 0 GPUs, and the Python3 kernel:
    cdswctl ssh-endpoint -p customerchurn -c 0.5 -m 0.75

    To create an SSH endpoint in a project owned by another user or a team, for example finance, prepend the username to the project and separate them with a forward slash:

    cdswctl ssh-endpoint -p finance/customerchurn -c 0.5 -m 0.75
    This command creates session in the project customerchurn that belongs to the team finance.
    Information for the SSH endpoint appears in the output:
    You can SSH to it using
        ssh -p <some_port> cdsw@localhost
  3. Open a new command prompt and run the outputted command from the previous step:
    ssh -p <some_port> cdsw@localhost
    For example:
    ssh -p 9750 cdsw@localhost
    You will be prompted for the passphrase for the SSH key you entered in the ML workspace web UI.
    Once you are connected to the endpoint, you are logged in as the cdsw user and can perform actions as though you are accessing the terminal through the web UI.
  4. Test the connection.
    If you run ls, the project files associated with the session you created are shown. If you run whoami, the command returns the cdsw user.
  5. Leave the SSH endpoint running as long as you want to use a local IDE.