Teleport is software which provides an SSH Jump Host (or Bastion host) service in a secure, modern way, with support for role-based access control and single sign-on.

Overview

The Teleport service provides:

• Single SSH hop from client systems anywhere on the internet to servers inside ECMWF (ecGate, HPC, etc)
• Re-authentication required only every 12 hours (once per day)
• Integration with standard tools such as the OpenSSH ssh client, scp, and ssh-agent
• Web-SSH interface for in-browser terminal access, with scp
• X11 and Port forwarding

The single sign-on step is performed using an application called "tsh", every 12 hours.

After that you use standard ssh or scp to connect to systems inside ECMWF.

Alternatively you can have simple terminal access in a web browser. 

Downloading tsh 

The tsh application is required to perform user authentication once every 12 hours.

tsh is open source, very portable, and has minimal dependencies.

• Download from the Gravitational website and place into your $PATH 

The binary is available for Linux 32-bit, 64-bit, and ARM, as well as Windows 64-bit and a signed package for MacOS.

MacOS users can also use homebrew for installation (brew install teleport).

User Authentication

Once every 12 hours, you will need to refresh your tokens with the tsh command, through your web browser.

First Time

Run tsh, giving the location of our gateway:

tsh login --proxy=shell.ecmwf.int:443

Your default web browser will open and you should login with your email address, ECMWF password, and then HID (ActivID) Token code.

Subsequent Occasions

tsh login

Connecting to hosts through the gateway

OpenSSH 7.3 or later has a simple command line option to connect via our gateway (shell.ecmwf.int) to the destination-host:

ssh -J username@shell.ecmwf.int username@destination-host

For example, if your username is ab0 and you wish to connect to ecgate:

ssh -J ab0@shell.ecmwf.int ab0@ecgate

The OpenSSH configuration setting for this is named ProxyJump:

Host ecgate
  User ab0
  ProxyJump ab0@shell.ecmwf.int

See the Legacy Configuration note below if your ssh client is older than 7.3.

If your connection fails after working for some time, it could be because your tokens have expired. You can check them:

$ tsh status
> Profile URL:  https://shell.ecmwf.int:443
  Logged in as: firstname.lastname@ecmwf.int
  Cluster:      shell.ecmwf.int
  Roles:        *
  Logins:       ab0
  Valid until:  2020-06-22 23:26:30 +0100 BST [EXPIRED]
  Extensions:   permit-X11-forwarding, permit-agent-forwarding, permit-port-forwarding, permit-pty

From OpenSSH 8.4 the client may refuse to connect with the cryptic message: "Connection closed by UNKNOWN port 65535".

This is because the Teleport system has to remain compatible with some old OpenSSH server versions. The problem will go away when our Bologna data centre is used instead.

The fix is to add this extra line to your OpenSSH configuration:

Host ecgate
  User ab0
  ProxyJump ab0@shell.ecmwf.int
  PubkeyAcceptedKeyTypes +ssh-rsa-cert-v01@openssh.com

Destination Hosts available

The hosts directly available through the Teleport gateway are:

• ECGATE (interactive node only)
• CCA and CCB login nodes

ssh -J ab0@shell.ecmwf.int,ab0@ecgate ab0@lxc

Configuring password-less login

Add the Teleport certificate authority to your ~/.ssh/authorized_keys file, on the relevant system at ECMWF, e.g. ecgate, cca:

curl -fs https://nexus.ecmwf.int/repository/internal-teleport-configs/prod/teleport_user_ca.pub >> ~/.ssh/authorized_keys

Terminal Access in a Web Browser

You can open a tabbed terminal in the web browser, with support for SCP upload and download.

Browse to  http://webshell.ecmwf.int/destination-host/username (replacing destination-host and username with your selection).

For example, if your username is ab0 and you wish to connect to ecgate:

http://webshell.ecmwf.int/ecgate/ab0

If you open another tab inside the web terminal, use the QUICK LAUNCH box and enter "username@destination-host:22", for example ab0@ecgate:22.

Windows Clients

There are various ways to initiate SSH from Windows 10, so it depends on your system and your preferences.

Alternatively, you can use:

Host ecgate
  User ab0
  ProxyCommand ssh -q -i c:/users/ab0/.tsh/keys/shell.ecmwf.int/firstname.lastname@ecmwf.int -W %h:%p ab0@shell.ecmwf.int
  IdentityFile c:/users/ab0/.tsh/keys/shell.ecmwf.int/firstname.lastname@ecmwf.int

# Windows currently has a bug, you need the full path to ssh or you will get:
# posix_spawn: No such file or directory

Host *.ecmwf.int
  User ab0
  ProxyCommand C:\Windows\System32\OpenSSH\ssh.exe -q -W %h:%p ab0@shell.ecmwf.int

# this assumes the SSH Agent is running, otherwise add:
# Host shell.ecmwf.int
#   IdentityFile ~/.tsh/keys/shell.ecmwf.int/firstname.lastname@ecmwf.int

{
  "guid": "{717406b0-06cb-454c-a0c4-875267fa373d}", # run "[guid]::NewGuid()" in a PowerShell to generate this
  "name": "ecGate",
  "commandline": "ssh ab0@ecgate.ecmwf.int"
  "suppressApplicationTitle": true,
  "hidden": false,
  "fontSize": 10,
  "fontFace": "Fira Code",
  "cursorShape": "filledBox",
  "cursorColor": "#073642",
  "colorScheme": "Solarized Dark",
},

Other Notes

SSH Agent is required

If you have logged in but ssh fails to connect, it may be that your SSH agent is not running.

The Agent can be started and tokens refreshed this way:

eval $(ssh-agent -s)
tsh logout
tsh login

And this will make sure the Agent continues to run in your environment:

echo 'eval $(ssh-agent -s)' >> ~/.bash_profile

SCP, X11, Agent, and Port Forwarding

SCP, Agent forwarding, X11 forwarding, and Port forwarding (including SOCKS proxy), all work through the Teleport gateway. A nice application of port forwarding with Teleport is the use of the ecflow_ui client on your local system to follow ecflow suites running at ECMWF.  See Teleport - using local ecflow_ui for more details.

For scp you can use the -o option:

scp -o ProxyJump=ab0@shell.ecmwf.int ab0@ecgate:/remote/file/path /local/file/path

Avoiding storing SSH Keys

You may wish to avoid storing SSH Keys to disk and always (and only) use the SSH Agent.

tsh  client versions 7.1.0 onwards have this feature:

-k, --add-keys-to-agent        Controls how keys are handled. Valid values are [auto no yes only]. 

X11 under macOS

$ brew cask install xquartz
# start xquartz app
$ export DISPLAY=:0
$ ssh -X ....

Legacy Configuration

For OpenSSH clients older than 7.3, the following configuration can be used:

# ~/.ssh/config file:
Host ecgate
  User ab0
  ProxyCommand /usr/bin/ssh -q -W %h:%p ab0@shell.ecmwf.int

Difficult client environments

You might not be able to download and run tsh, or access our web login service, or run the SSH Agent, from where you wish to use ssh.

Instead you can use (or copy) the identity files which tsh stores in $HOME:

# ~/.ssh/config file:
Host ecgate
  User ab0
  IdentityFile ~/.tsh/keys/shell.ecmwf.int/firstname.lastname@ecmwf.int
  ProxyCommand /usr/bin/ssh -q -i ~/.tsh/keys/shell.ecmwf.int/firstname.lastname@ecmwf.int -W %h:%p ab0@shell.ecmwf.int

This is a good way to access Teleport credentials via a shared file system from any host.

Even more difficult client environments

Recent Fedora Linux distributions (such as Fedora-33) using OpenSSH 8.4p1 no longer accept the "ssh-rsa" signature scheme using the SHA-1 hash algorithm in conjunction with the RSA public key algorithm.

As a workaround for this problem, you  may need to add ssh-rsa (rsa-sha2-256 or rsa-sha2-512 can also be used) as a PubkeyAcceptedKeyTypes to your ~/.ssh/config file:

# ~/.ssh/config file:
Host ecgate
  User          ab0
  PubkeyAcceptedKeyTypes +ssh-rsa
  IdentityFile	~/.tsh/keys/shell.ecmwf.int/firstname.lastname@ecmwf.int
  ProxyCommand /usr/bin/ssh -q -o PubkeyAcceptedKeyTypes=+ssh-rsa -i ~/.tsh/keys/shell.ecmwf.int/firstname.lastname@ecmwf.int -W %h:%p ab0@shell.ecmwf.int

Network Requirements

The service is configured to use only standard ports 22, 80, and 443, to help with access wherever users are.

Additional configuration at the local user site may be required to allow connections out. The diagram below shows the TCP ports and destination hosts used.

• The "tsh login" step uses ports 80 and 443 in order to log in to the service and obtain the client certificate.
• Your ssh client uses standard port 22 for server access.
• The web shell service also uses port 443 on the same host.