I am looking for some recommendations on how to secure the data of my physical servers (against physical theft), that I am about to set up. I am new to selfhosting but have a few years of experience running Linux on a desktop.
My usecase is a simple debian(?) server at home with Paperless ngx and Tailscale for when I am away from home.
The question is how to encrypt the data while still being able to keep the server updated.
Coming from Desktop my first thought was to simply enable FDE on install. But that would mean supplying the password everytime the server needs to reboot for an update. Could someone provide some insights on how often updates to debian require a reboot?
My second thought was to use an encrypted data partition. That way the server could reboot and I could use wireguard to ssh in and open the partition even when I am away from home for a longer time.
I am open to other ideas!
You must log in or register to comment.
Unless the crook happens to be extremely nerdy or its law enforcement, already being a Linux formatted partition feels it should be enough for a rando breaking in and stealing a computer.
That being said, something like a PiKVM connected to your server (and Tailscale) could let you enable both UEFI/boot password and propt for LUKS decryption upon boot.
I wouldn’t rely on the thief not knowing how to read linux partitions. That very well may be the case, but the person they sell your hardware to will know better, considering they are in the market of purchasing used server hardware.
I self host and my threat model is the thief selling my server to someone who knows what to do with it, but not knowing how to extract encryption keys from the memory of a running server before unpluging it. That being said I haven’t figured out encryption yet so watching this thread.
I would do FDE yeah. My current laptop setup is with systemd-boot and a special initramfs that allows me to unlock it with a yubikey, with fallback to password. Fair warning, this exact configuration is not particularly easy to setup.
There are also modules which enable early network connectivity along with a SSH server, meaning you login and unlock it remotely. I have not tried this.
Debian does not frequently require rebooting under normal circumstances. Kernel updates are not that frequent, and you can usually put it off for a bit if you don’t want to deal with it.
I’m using https://github.com/dracut-crypt-ssh/dracut-crypt-ssh on some of my servers. The initrd opens an ssh port where you can login and enter the passphrase. Setting it up is non-trivial, but it works well. Haven’t tried it on Debian but there should be something similar.
This is actually really interesting, I might have to try this.
I currently use a USB stick with a key file on it that I need to plug in on boot. Something like this but it wasn’t that easy for me. https://openterprise.it/2022/07/fedora-unlock-luks-full-disk-encrypted-system-using-usb-stick/
Edit: I wonder if yours will work on my VPS… I’d love to encrypt that.
I use LUKS on my systems. I use mandos and wireguard in intramfs to connect to a mandos server to unlock LUKS during boot.
What’s mandos? I don’t find anything useful when searching for it.
https://www.recompile.se/mandos
I use this to get wireguard in intramfs. I just skip the dropbear related stuff. https://github.com/r-pufky/wireguard-initramfs
Luks FDE, and install dropbear-initramfs, configure ssh authorized_keys and rebuild initramfs. Then you can access initramfs via ssh to type luks password.
A more detailed guide for dropbear: https://www.cyberciti.biz/security/how-to-unlock-luks-using-dropbear-ssh-keys-remotely-in-linux/
If I remember correctly, the only outdated bit of information is that the IP configuration doesn’t happen anymore in the initramfs configuration but you must pass a parameter at the kernel by editing /etc/default/grub and passing
GRUB_CMDLINE_LINUX_DEFAULT="ip=192.168.x.x:::::"where 192.168.x.x is the IP address that you want at boot
You can use FDE and setup a minimal ssh server like dropbear to run at startup. This way, you can supply the password via a keyboard connected to the machine OR via ssh. This gives you a similar workflow to the data partition you mentioned, but encrypts the entire system.
Somethign I haven’t seen mentioned yet is clevis and tang, basically if you have more than one server then they can unlock each other and if they’re spatially separated then it is very unlikely they get stolen at the same time.
Though you have to make sure it stops working when a server get stolen, using a mesh VPN works just as well after the server is stolen so either use public IPS and a VPN or use a hidden raspberry pi that is unlikely to be stolen or make the other server stop tang after the first one is stolen.
I wanted to encrypt against physical theft but then I realized that the server weighs 20kg and there’s more valuable and lighter stuff to steal. And a thief would never be able to mount the zfs array, they’ll just fence my $2000 server for $50 and someone else will sell it piece by piece on ebay. And between people who buys used drives online on ebay almost nobody is going to do a full surface scan of a 12tb drive in the hopes of finding valuable data.
If you want to encrypt only the data partition you can use an approach like https://michael.stapelberg.ch/posts/2023-10-25-my-all-flash-zfs-network-storage-build/#encrypted-zfs to ulock it at boot.
TL;DR: store half of the decryption key on the computer and another half online and write a script that at boot fetches the second half and decrypt the drive. There is a timewindow where a thief could decrypt your data before you remove the key if they connect your computer to the network, but depending on your thread model can be acceptable. you can also decrypt the root portion with a similar approach but you need to store the script in the initramfs and it is not trivial.
Another option I’ve seen suggested is storing the decryption key on a USB pendrive and connect it with a long extension cord to the server. The assumption is that a thief would unplug all the cables before stealing your server.
Good question.
Debian doesn’t often require a reboot, but the longer you go, and if you need kernel modules (nvidia is the worst at this) you might need to reboot to keep everything in sync.
My suggestion: raspberry pi, like 1st edition, keep the key very secure, give it a usb serial console. When the server reboots, enter the password that way. It’s your emergency console.
I am looking for some recommendations on how to secure the data of my physical servers (against physical theft)…
Lock the case shut. Run a security wire (like for laptops) to the case and mount the other end to something big / immovable.
Truly paranoid: 24/7 video surveillance.
Super truly paranoid: same as above but in a locked room
