Initialize an SSH Endpoint
This topic describes how to establish an SSH endpoint for Cloudera Machine Learning.
- Create a configuration file at:
$HOME/.cdsw/config.yaml. The contents ofconfig.yamlshould be:username: <username> url: <ML_workspace_url> auth: authtype: 1 basic: null apikey: <your_api_key>To collect the values for these fields, first log in to your CML workspace using SSO:username: The username with which you are logged into the CML workspace. Found in the top right corner of your ML workspace.url: The complete URL used to access the CML workspace. For example:https://ml-<randomly-generated-cluster-name>apikey: Go to User Settings > API Keys. Copy the value of the Legacy API Key to this field.
- Create a local SSH endpoint to Cloudera Machine Learning. Run
the following
command:
cdswctl ssh-endpoint -p <username>/<project_name> [-c <CPU_cores>] [-m <memory_in_GB>] [-g <number_of_GPUs>] [-r <runtime ID> ]The command uses the following defaults for optional parameters:- CPU cores: 1
- Memory: 1 GB
- GPUs: 0
For example, the following command starts a session for the usermiltonunder thecustomerchurnproject with .5 cores, .75 GB of memory, 0 GPUs, and the Python3 kernel:cdswctl ssh-endpoint -p customerchurn -c 0.5 -m 0.75To create an SSH endpoint in a project owned by another user or a team, for example
finance, prepend the username to the project and separate them with a forward slash:
This command creates session in the projectcdswctl ssh-endpoint -p finance/customerchurn -c 0.5 -m 0.75customerchurnthat belongs to the teamfinance.Information for the SSH endpoint appears in the output:... You can SSH to it using ssh -p <some_port> cdsw@localhost ... - Open a new command prompt and run the outputted command from
the previous
step:
For example:ssh -p <some_port> cdsw@localhost
You will be prompted for the passphrase for the SSH key you entered in the ML workspace web UI.ssh -p 9750 cdsw@localhostOnce you are connected to the endpoint, you are logged in as thecdswuser and can perform actions as though you are accessing the terminal through the web UI. - Test the connection.If you run
ls, the project files associated with the session you created are shown. If you runwhoami, the command returns thecdswuser. - Leave the SSH endpoint running as long as you want to use a local IDE.
