Always connected reverse SSH port forwarding with systemd

Replacing autossh, the de-facto for managing and monitoring SSH connections with systemd’s service.

Terms used

  • remote host refers to a device running in a third-party managed network i.e you have no control over any networking equipment. Its public IP may or mayn’t change
  • managed host refers to a server/device whose SSH port is reachable

Assumptions

  • remote host runs a distribution that uses systemd as an init system
  • A user named callhome on the remote host is able to SSH using public key authentication as incoming@managed host

systemd service configuration

Create a systemd service unit by adding the below mentioned config to a file called /etc/systemd/system/call-home.service.

[Unit]
Description=Forward local SSH port to remote host
After=network-online.target
Before=multi-user.target
DefaultDependencies=no

[Service]
# SSH connection uses the private key stored in this
# users home dir (~/.ssh/)
User=callhome

# SSH connection with port forwarding
# Forwards local port 22 to port 5000
ExecStart=/usr/bin/ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o ServerAliveInterval=20 -o ServerAliveCountMax=1 -o ExitOnForwardFailure=yes -N -T -R5000:localhost:22 incoming@managedhost.example.com

# wait 60 seconds before trying to restart the connection
# if it disconnects
RestartSec=60

# keep retrying no matter what
Restart=always

[Install]
WantedBy=multi-user.target

Ensure that this service starts at boot

root@raspberry-pi:~# systemctl enable call-home
Created symlink from /etc/systemd/system/multi-user.target.wants/call-home.service to /etc/systemd/system/call-home.service.
root@display1:~#

Start the service and test if port forwarding works

root@raspberry-pi:~# systemctl start call-home
# check to see if the connection was established
root@raspberry-pi:~# sudo journalctl -u call-home
Jun 25 18:03:00 raspberry-pi systemd[1]: Starting SSH reverse tunnelling...
Jun 25 18:03:00 raspberry-pi systemd[1]: Started SSH reverse tunnelling.
Jun 25 18:03:01 raspberry-pi ssh[23582]: Warning: Permanently added '1.2.3.4' (ECDSA) to the list of known hosts.

If everything worked, you should be able to connect to port 5000 on the managed host, authenticate and reach remote host,
like so:

root@ip-172-31-20-1 :~# ssh -p 5000 pi@127.0.0.1
pi@127.0.0.1's password:

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sat Jun 25 20:17:28 2016 from localhost
pi@raspberry-pi:~ $

Troubleshooting

When attempting to connect from managed host to remote host you get error ssh: connect to host 127.0.0.1 port 5000: Connection refused

  • On the remote host check if forwarding worked, like so:
pi@raspberry-pi:~ $ sudo journalctl -u call-home

Leave a Reply