How to use CÉCI clusters from a terminal.

This short tutorial shows how to connect to a CÉCI cluster from a terminal, such ax XTerm, iTerm, Cygwin, etc.

Using the CÉCI key

Once you have received your private key by email, store it in a safe location. The most rational place to store it is in your .ssh folder in your home directory, along with other SSH keys you may have.

Change the permissions of the file so that only you can read it. Otherwise, your SSH client will refuse to use it.

chmod 600 ~/.ssh/id_rsa.ceci

Then, connect to a CÉCI cluster, e.g. Hmem, with

ssh -i ~/.ssh/id_rsa.ceci yourlogin@hmem.cism.ucl.ac.be

Make sure to replace mylogin with your actual login. You are then asked to type in your passphrase (the one you chose while filing in the account creation request form), and if everything worked properly, you should be greeted with the cluster's Message Of The Day:

Welcome to 
        __  __     __    __     ______     __    __    
       /\ \_\ \   /\ "-./  \   /\  ___\   /\ "-./  \   
       \ \  __ \  \ \ \-./\ \  \ \  __\   \ \ \-./\ \  
        \ \_\ \_\  \ \_\ \ \_\  \ \_____\  \ \_\ \ \_\ 
         \/_/\/_/   \/_/  \/_/   \/_____/   \/_/  \/_/ 
                 HighMemory CISM-CECI cluster

The very first time...

Upon your very first connection to a cluster, you will be greeted by a warning such as :

The authenticity of host 'hmem.cism.ucl.ac.be (130.104.1.220)' can't be established.
RSA key fingerprint is 06:54:39:a0:5c:b5:56:b3:29:9e:96:67:a0:4a:c1:ff.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'hmem.cism.ucl.ac.be,130.104.1.220' (RSA) to the list of known hosts.

This warning is normal: your SSH program warns you that it is the first time it sees that new computer. To make sure you are actually connecting to the right machine, you should compare the RSA key fingerprint shown in the message with the fingerprint announced on the cluster page.

If they match, you can proceed and type 'yes'. Your SSH program will then store that key and check, for every subsequent SSH connection, that the server to which you connect is indeed Hmem.

Tailoring your configuration

You can dramatically shorten the length of the command line used to connect to the clusters by setting your ssh configuration file properly. Assuming your login is mylogin and you are working with the Hmem cluster, add the following to your configuration file ~/.ssh/config (create it if it does not exist):

Host hmem
    HostName hmem.cism.ucl.ac.be
    User mylogin
    ForwardX11 yes
    ForwardAgent yes
    IdentityFile ~/.ssh/id_rsa.ceci

From now on, you will be able to connect to the cluster simply by issuing the following command:

ssh hmem

You can do the same for each cluster you will connect to. Type man ssh_config for a detailed list of options you can set in your configuration file.

The FowardX11 option is needed to open any host program in the client display.

With ForwardAgent the connection to the agent is automatically forwarded to the remote side. Note that, as explained in the SSH manpage, Agent forwarding should be enabled only for trusted remote hosts. It is perfectly safe to use it for the CÉCI clusters, but be cautious with other computers.

Using an SSH agent

You can further ease the process by using an SSH agent which will remember the passphrase so you do not have to type it in each time you issue the SSH command. First, make sure you have an agent running: if

ssh-add -l

responds with "Could not open a connection to your authentication agent.", start an agent with

eval $(ssh-agent)

Then, load the key with the command

ssh-add ~/.ssh/id_rsa.ceci

and enter your passphrase. From now on, in the current terminal, all ssh commands will be handled by the agent and you won't have to type your passphrase again.

Once you have an agent running, you can use the -A option of ssh to forward your agent from one computer to another. This allows you to connect, or copy files, from one cluster to another effortlessly.

You can have an ssh-agent started automatically at login by using password managing software such as Mac OS Keychain, KDE KWallet, Gnome Keyring, etc.

Going through an SSH gateway

As the clusters are not accessible from outside the university networks, you will need to either use a VPN or an SSH gateway. Going through an SSH gateway can be entirely transparent provided your client is correctly configured. Refer to the documentation provided by your local sysadmin team for the exact process.

Note that for the Tier-1 Zenobe, the procedure is explicited in the Zenobe quickstart document.

Copying files back and forth

With your agent running, copying files is easy with, for instance the scp command and the rsync command. The former works exactly like the cp command, except that it works across the network to copy files from one computer to another. A command like

scp -r mydir/ hmem:

will copy directory mydir on your personal computer to your home directory on the cluster hmem. If your login on the cluster is not the same as the one on your local computer, and you did not specify your login for hmem in your config file, make sure to specify your login on hmem with something like the following.

scp -r mydir/ mylogin@hmem:

In the case you are not using an ssh agent and your .ssh/config is not configured, you will need to add the -i .ssh/id_rsa.ceci option to the command line.

You can also use the rsync command. The later is more intelligent in the sense that it only copies what is needed to be copied in the case the target directory already exist and contains unchanged files. Click here for some examples.

Troubleshooting

Step 1. Make sure that your key file has the correct permissions:

ls -l ~/.ssh/id_rsa.ceci

should output something that starts with

-rw------

Step 2. Make sure your key and your passphrase match by issuing the following command:

ssh-keygen -yf ~/.ssh/id_rsa.ceci

If the output is 'load failed' you either have a wrong passphrase or your key file has been corrupted.

Step 3. Make sure you are connected to your university network: the following command

curl -I -k https://login.ceci-hpc.be/init/

should return something starting with

HTTP/1.1 200 OK
Status: 200 OK

Step 4. Make sure the ssh client is asking for a passphrase and not a password. Otherwise, it means you are not correctly instructing your ssh client to use the private key.

Run the ssh command with the -v option. You should find a line stating

debug1: Offering RSA public key: .ssh/id_rsa.ceci

The line immediately following line should be something like:

debug1: Server accepts key: pkalg ssh-rsa blen 277

otherwise it means the key your are using does not correspond to the one stored on the cluster. If this happens while you just updated or renewed your account, it may indicate that your new key is not fully propagated yet. Try again later.

Frequent mistakes

Here are some frequent errors you should avoid.

The permissions on your key file are not correct

If, after running ssh hmem, for instance, you see something like:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@         WARNING: UNPROTECTED PRIVATE KEY FILE!          @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
Permissions 0644 for '/home/dfr/.ssh/id_rsa.ceci' are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: /home/dfr/.ssh/id_rsa.ceci
dfr@hmem.cism.ucl.ac.be's password:

it means that Permissions 0644 for '/home/dfr/.ssh/id_rsa.ceci' are too open. Change them to 600 as explained in the first section of this document.

You did not specify the correct path to your SSH private key

If, after running ssh, you are being asked for a password directly,

$ ssh hmem
dfr@hmem.cism.ucl.ac.be's password:

it means that your SSH client did not try to use the SSH key. Make sure you either used the -i option or that your .ssh/config is properly configured and contains no typos.

You used a wrong username or tried to connect before your keys are synchronised

If, after running ssh, you are being asked for a passphrase, then a password,

$ ssh hmem
Enter passphrase for key '/home/dfr/.ssh/id_rsa.ceci': 
dfr@hmem.cism.ucl.ac.be's password:

it often means that the user name you are using is not the correct one. If you just requested or renewed your account, it could also mean that you are trying to connect with the new private key while it has not been synchronized to the cluster yet (keep in mind that the clusters are not synchronized simultaneously.)

You have too many SSH keys lying around

If the server responds with

$ ssh hmem
Received disconnect from host: 2: Too many authentication failures for dfr

that means that your client offered the server too many keys to try and access it. To make sure the client uses only the correct key (i.e. the one specified by the IdentityFile option,) you will need to add IdentitiesOnly yes to your configuration file, or pass it in argument:

$ ssh - o 'IdentitiesOnly yes' hmem

You do not have a locale set

If you can login but you see several instances of the following

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LC_CTYPE = "UTF-8",
    LANG = (unset)
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

when you connect, you should set the following environment variables:

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

for instance in your .bash_profile. Make sure to adapt if you want another language than English. You can list all locales that are available with locale -a.

© CÉCI.