Language Technologies Institute

Access to the HPC clusters is gained through requests submitted by a contributing faculty member (often your PhD advisor). When requesting access, please follow the steps outlined below:

1. Speak with your PhD advisor to discuss your need for HPC resources. Obtain their approval and sponsorship.
2. Have your sponsoring faculty member provide the necessary information to the cluster administrator, including:
   • Full Name
   • Your ANDREW or SCS ID (see table below)
   • The cluster to which the account should be added
   • Any specific software or tools required
   • Custom shell (default: bash)
   • User account expiration date (if desired)
3. If the cluster uses SCS accounts, the faculty member or PhD advisor should send the request to help@cs.cmu.edu
4. If the cluster uses ANDREW accounts, the faculty member or PhD advisor should send the request to dvosler@cs.cmu.edu
5. Once the application has been received, the cluster administrator will provision your account.

You will need an SSH client to connect and log in to the cluster. Different clusters use different authentication sources. Refer to the following table to determine whether to use your ANDREW or SCS ID:

To log in to a cluster, use the following command, replacing <username> with your username and <mycluster> with the fully qualified domain name (FQDN) of your cluster:

 ssh <username>@<mycluster>

If you want to bypass password authentication, you can use SSH key authentication. SSH keys allow you to connect to a server or multiple servers without entering your password each time.

To generate an SSH key pair, follow these steps:

1. On your local machine, generate an SSH key:

 ssh-keygen -t ed25519 -C "[$(whoami)@$(uname -n)-$(date -I)]"

2. Automatically add your public key to the server by running the following command (use your ANDREW password):

 ssh-copy-id -i ${HOME}/.ssh/id_ed25519.pub <username>@<mycluster>.

3. You can also configure your ~/.ssh/config file to avoid typing the full cluster address every time. Add the following lines to the file:

 Host <mycluster>
 HostName <mycluster>
 User <username>
 IdentityFile ${HOME}/.ssh/id_ed25519

4. Then, you can log in with the simple command ssh mycluster.

If you would like to set or change your shell on all nodes of the cluster, including login and compute nodes, please follow these steps:

1. Send an email to help@scs.cmu.edu to request a shell change.
2. Specify your desired shell:
   • bash
   • zsh

If you require a different shell than the ones mentioned above, please notify the support team, and they will assist you accordingly.

Note: not implemented on all clusters.

You can also ssh directly to a compute node where you have a job already running by doing:

ssh -J babel babel-X-XX

To copy data to compute nodes when the data directories are not directly accessible from the login nodes, you can utilize the login node as a JumpHost, allowing direct connections to the compute nodes. Follow these steps to configure SSH and establish the connection:

1. Launch an interactive job on a compute node.
2. Determine the IP address of the compute node by executing the ip a command on the compute node itself.
3. Update your SSH configuration file located at ~/.ssh/config with the following entries:

 Host <mycluster-short-name>
   HostName <mycluster-fqdn>
   User <username>
   IdentityFile <identityfile>

 Host <mycluster>-compute-node
   HostName <node ip addr>
   User <username>
   ProxyJump <mycluster>

Replace the relevant items in < > with your actual username, cluster name and the path to your SSH private key file. For example, to connect to a node in the Babel cluster:

Host babel
  HostName babel.lti.cs.cmu.edu
  User <username>
  IdentityFile /path/to/identityfile

Host babel-compute-node
  HostName <node ip addr>
  User <username>
  ProxyJump babel

The updated configuration allows you to establish a connection to the compute node through the login node, acting as a JumpHost.

Refer to your cluster's dedicated page in the Cluster's section for more detailed information.