Root on ZFS with native encryption

Introduction

This documentation describes how to set up Alpine Linux using ZFS with a pool that uses ZFS' native encryption capabilities, which have been recently introduced in ZFS on Linux (ZoL) 0.8.0.

Note that you must install the /boot/ directory on an unencrypted partition (either an unencrypted ZFS pool or any other FS of your choosing, if it's compatible with your bootloader) to boot correctly.

Requirements

You'll need a medium to put a live image on. You can use any live medium that supports ZoL >=0.8.x, but as of writing this it's easiest to use Ubuntu 19.10, which comes with ZFS pre-installed.

Hard Disk Device Name

The following documentation uses the /dev/sda device as installation destination. If your environment uses a different device name for your hard disk, use the corresponding device names in the examples. It also uses rpool as name of the root pool, you can change this at will, but be sure to change it everywhere it's mentioned.

Setting up Alpine Linux Using ZFS with native encryption

To install Alpine Linux in a ZFS pool with encryption enable, you cannot use the official installation procedure, so follow along this guide.

Preparing the Installation Environment

This section assumes that you're using the previously mentioned Debian installation medium. If you're using a different medium feel free to skip this section.

After booting the Debian image you'll have to enable the experimental repos for the time being to be able to access ZFS 0.8. For this you'll have to edit /etc/apt/sources.list:

# sed 's/buster/experimental/' -i /etc/apt/sources.list
# echo 'deb http://deb.debian.org/debian experimental contrib' >> /etc/apk/repositories

Now install ZFS 0.8:

# apt update
# apt install libnvpair1linux libuutil1linux libzfs2linux libzpool2linux zfs-dkms zfsutils-linux zfs-zed

And load the ZFS module:

# modprobe zfs

Creating the Partition Layout

Linux requires an unencrypted /boot/ partition to boot. You can assign the remaining space for the encrypted ZFS pool.

• Start the fdisk utility to set up partitions:

# fdisk /dev/sda

• Create the /boot/ partition:

• Enter n → p → 1 → 1 → 100m to create a new 100 MB primary partition.

• Set the /boot/ partition active:

• Enter a → 1.

• Create the ZFS partition:

• Enter n → p → 2 to start creating the next partition. Press Enter to select the default start cylinder. Enter the size of partition. For example, 512m for 512 MB or 5g for 5 GB. Alternatively press Enter to set the maximum available size.

• To verify the settings, press p. The output shows, for example:

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sda1  *      2048   206847   204800  100M 83 Linux
/dev/sda2       206848 41943039 41736192 19.9G 83 Linux

• Press w to save the changes.

Setting up the root pool

You can create your rootpool with the following command:

# zpool create -o ashift=12 \
     -O acltype=posixacl -O canmount=off -O compression=lz4 \
     -O dnodesize=auto -O normalization=formD -O relatime=on -O xattr=sa \
     -O encryption=aes-256-gcm -O keylocation=prompt -O keyformat=passphrase \
     -O mountpoint=/ -R /mnt \
     rpool /dev/sda2

You will have to enter your passphrase at this point. Choose wisely, as your passphrase is most likely the weakest link in this setup.

A few notes on the options supplied to zpool:

• ashift=12 is recommended here because many drives today have 4KiB (or larger) physical sectors, even though they present 512B logical sectors

• acltype=posixacl enables POSIX ACLs globally

• normalization=formD eliminates some corner cases relating to UTF-8 filename normalization. It also enables utf8only=on, meaning that only files with valid UTF-8 filenames will be accepted.

• xattr=sa vastly improves the performance of extended attributes, but is Linux-only. If you care about using this pool on other OpenZFS implementation don't specify this option.

After completing this, confirm that the pool has been created:

# zpool status

Should return something like:

  pool: rpool
 state: ONLINE
  scan: none requested
config:

	NAME         STATE     READ WRITE CKSUM
	rpool       ONLINE       0     0     0
	  sda2  ONLINE       0     0     0

errors: No known data errors

Creating the required datasets

# zfs create -o mountpoint=none -o canmount=off rpool/ROOT
# zfs create -o mountpoint=legacy rpool/ROOT/alpine
# mount -t zfs rpool/ROOT/alpine /mnt/

Creating optional datasets (feel free to add your own)

# zfs create -o mountpoint=/home rpool/HOME
# zfs create -o mountpoint=/var/log rpool/LOG

Creating the /boot filesystem

# mkfs.ext4 /dev/sda1

Mounting the /boot filesystem

• Create the /mnt/boot/ directory and mount the /dev/sda1 partition in this directory:

# mkdir /mnt/boot/
# mount -t ext4 /dev/sda1 /mnt/boot/

Installing Alpine Linux

Please follow Installing Alpine Linux in a chroot to setup a base install of Alpine Linux.

After you've followed that guide, you still have to do some additional setup for ZFS:

• As of the time of writing this ZFS 0.8.x is only available in Edge, so you'll have to enable it in /etc/apk/repositories. Check pkgs.alpinelinux.org to see the status of this.

• Install the ZoL and linux-vanilla package: apk add linux-vanilla zfs

• Enable ZFS' services:

# rc-update add zfs-import sysinit
# rc-update add zfs-mount sysinit

• Edit the /etc/mkinitfs/mkinitfs.conf file and append zfs module to the features parameter:

features="ata base ide scsi usb virtio ext4 lvm zfs"

Be mindful to also include other modules which may be required for your setup, such as the nvme module.

• Rebuild the initial RAM disk:

# mkinitfs $(ls /lib/modules/)

• Edit the /etc/update-extlinux.conf file, set the root ZFS dataset and append the following kernel options to the default_kernel_opts parameter:

root=rpool/ROOT/alpine
default_kernel_opts="... rootfstype=zfs"

• Update extlinux's config (if you're not using a different bootloader)

# update-extlinux
# exit

Ignore the errors the update-extlinux utility displays.

• Write the MBR to the /dev/sda device:

# dd bs=440 count=1 conv=notrunc if=/mnt/usr/share/syslinux/mbr.bin of=/dev/sda

Unmounting the filesystems

• Unmount /mnt/boot/:

# umount /mnt/boot/

• Unmount all zfs filesystems:

# zfs unmount -a

• Reboot the system:

# reboot

Booting the system

Right now mkinitfs doesn't support ZFS asking for passwords during boot, so it'll throw you into a rescue shell for you to enter the password during boot. You have to do the following things after pressing enter:

# zfs load-key -a
# mount -t zfs rpool/ROOT/alpine /sysroot
# exit

And your system should continue booting! :)

Troubleshooting

General Procedure

In case your system fails to boot, you can verify the settings and fix incorrect configurations:

• Preparing the Installation Environment

• Load the ZFS kernel module:

# modprobe zfs

• Mount the file systems

# zpool import -R /mnt rpool
# mount -t ext4 /dev/sda1 /mnt/boot

• Verify that you run the steps described in the Installing Alpine Linux section correctly. Update the configuration if necessary.