Unlock VPS Full Disk Encryption

Having Full Disk Encryption for your VPS or remote server is nice, however if you don't have local terminal or out-of-band access via IPMI/iLO the only way to unlock the root partation is to use SSH on boot.

The steps outlined here will show how to configure your server to unlock your encrypted drives remotely. Steps here are for Ubuntu 16.04 and were collected from several stack exchange answers and MAN pages.

Server Configuration

Install dropbear and busybox:
sudo apt install dropbear busybox

Edit dropbear set NO_START=0 and change hostkey paths.
sudo vi /etc/default/dropbear

Should look like the following:

# disabled because OpenSSH is installed
# change to NO_START=0 to enable Dropbear
NO_START=0
# the TCP port that Dropbear listens on
DROPBEAR_PORT=22

# any additional arguments for Dropbear
DROPBEAR_EXTRA_ARGS=

# specify an optional banner file containing a message to be
# sent to clients before they connect, such as "/etc/issue.net"
DROPBEAR_BANNER=""

# RSA hostkey file (default: /etc/dropbear/dropbear_rsa_host_key)
DROPBEAR_RSAKEY="/etc/initramfs-tools/etc/dropbear/dropbear_rsa_host_key"

# DSS hostkey file (default: /etc/dropbear/dropbear_dss_host_key)
DROPBEAR_DSSKEY="/etc/initramfs-tools/etc/dropbear/dropbear_dss_host_key"

# ECDSA hostkey file (default: /etc/dropbear/dropbear_ecdsa_host_key)
DROPBEAR_ECDSAKEY="/etc/initramfs-tools/etc/dropbear/dropbear_ecdsa_host_key"

# Receive window size - this is a tradeoff between memory and
# network performance
DROPBEAR_RECEIVE_WINDOW=65536

Next, Setup directory structure.
sudo mkdir -p /etc/initramfs-tools/root/.ssh

Create new dropbear private key.
sudo dropbearkey -t rsa -f /etc/initramfs-tools/root/.ssh/id_rsa.dropbear

Convert dropbear key to openssh format.
sudo /usr/lib/dropbear/dropbearconvert dropbear openssh /etc/initramfs-tools/root/.ssh/id_rsa.dropbear /etc/initramfs-tools/root/.ssh/id_rsa

Extract the public key from dropbear formated key.
sudo dropbearkey -y -f /etc/initramfs-tools/root/.ssh/id_rsa.dropbear | grep "^ssh-rsa " > /etc/initramfs-tools/root/.ssh/id_rsa.pub

Put the public key into authorized_keys.
sudo cat /etc/initramfs-tools/root/.ssh/id_rsa.pub >> /etc/initramfs-tools/root/.ssh/authorized_keys

Create new hook script.
sudo vi /etc/initramfs-tools/hooks/crypt_unlock.sh

#!/bin/sh

PREREQ="dropbear"

prereqs() {
echo "$PREREQ"
}

case "$1" in
prereqs)
prereqs
exit 0
;;
esac

. "${CONFDIR}/initramfs.conf"
. /usr/share/initramfs-tools/hook-functions

if [ "${DROPBEAR}" != "n" ] && [ -r "/etc/crypttab" ] ; then
cat > "${DESTDIR}/bin/unlock" << EOF
#!/bin/sh
if PATH=/lib/unlock:/bin:/sbin /scripts/local-top/cryptroot; then
kill \`ps | grep cryptroot | grep -v "grep" | awk '{print \$1}'\`
# following line kill the remote shell right after the passphrase has
# been entered.
kill -9 \`ps | grep "\-sh" | grep -v "grep" | awk '{print \$1}'\`
exit 0
fi
exit 1
EOF

chmod 755 "${DESTDIR}/bin/unlock"

mkdir -p "${DESTDIR}/lib/unlock"
cat > "${DESTDIR}/lib/unlock/plymouth" << EOF
#!/bin/sh
[ "\$1" == "--ping" ] && exit 1
/bin/plymouth "\$@"
EOF

chmod 755 "${DESTDIR}/lib/unlock/plymouth"

echo To unlock root-partition run "unlock" >> ${DESTDIR}/etc/motd

fi

Make script executable.
sudo chmod +x /etc/initramfs-tools/hooks/crypt_unlock.sh

Edit initramfs.conf
sudo vi /etc/initramfs-tools/initramfs.conf

Set BUSYBOX=y

Add DROPBEAR=y right under BUSYBOX=y

Right under DEVICE= line add the following:
IP=11.111.11.214::11.111.11.222:255.255.255.240::ens160:off

Note: Keep DEVICE= unassigned otherwise it wont work.
IP= format [host ip]::[gateway ip]:[netmask]:[hostname]:[device]:[autoconf]

Now we need to disable ens160 after we are done with drop bear so the system can bring it back online using system information.

sudo vi /usr/share/initramfs-tools/scripts/init-bottom/dropbear

Add the following at the bottom of the script:
ifconfig ens160 down

Disable dropbear on boot so OpenSSH can be used.
sudo update-rc.d -f dropbear remove

Optional step, change dropbear ssh port (recommended):
sudo vi /usr/share/initramfs-tools/scripts/init-premount/dropbear

In the run_dropbear() function, append -p <port#> to the exec line.
exec /sbin/dropbear ${DROPBEAR_OPTIONS:-$PKGOPTION_dropbear_OPTION} -Fs -p 3000

Update initramfs.
sudo update-initramfs -u

Server configuration is now compete.

Client Configuration

Now we need to get some information to the client and configure the client to connect.

Copy the host key and private key to your home dirictory.

sudo cp /etc/initramfs-tools/etc/dropbear/dropbear_rsa_host_key /home/user/known_hosts.initramfs
sudo cp /etc/initramfs-tools/root/.ssh/id_rsa /home/user/id_rsa.initramfs

Change ownership so you can scp them later:
sudo chown user:user /home/user/known_hosts.initramfs
sudo chown user:user /home/user/id_rsa.initramfs

From the client (local computer), SCP the files from the server to your client (local computer).
scp user@11.111.11.214:/home/user/known_hosts.initramfs ~/.ssh/
scp user@11.111.11.214:/home/user/id_rsa.initramfs ~/.ssh/

Fix the permissions.
chmod 600 ~/.ssh/id_rsa.initramfs

Make a easy to use ssh config.
vi ~/.ssh/config

Host alias_name
    Hostname 11.111.11.214
    User root
    Port 3000
    UserKnownHostsFile ~/.ssh/known_hosts.initramfs
    IdentityFile ~/.ssh/id_rsa.initramfs

To unlock the encrypted drives on restart, you just have to run the following.
ssh alias_name

Example output:

init6@FBI:~$ ssh alias_name
The authenticity of host '[11.111.11.214]:3000 ([11.111.11.214]:3000)' can't be established.
ECDSA key fingerprint is SHA256:l62h1eAFWnIYlSrnTPfhDb9osIKEp4E9Gxw0NdHfMBQ.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[11.111.11.214]:3000' (ECDSA) to the list of known hosts.
To unlock root-partition run unlock


BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

# unlock
Please unlock disk sda5_crypt:
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  Reading all physical volumes.  This may take a while...
  Found volume group "vpn-vg" using metadata type lvm2
  /run/lvm/lvmetad.socket: connect failed: No such file or directory
  WARNING: Failed to connect to lvmetad. Falling back to internal scanning.
  2 logical volume(s) in volume group "vpn-vg" now active
cryptsetup: sda5_crypt set up successfully
Connection to 11.111.11.214 closed.

Let it finish booting up and you can ssh into the server the normal way.