Setting up ZFS on LUKS

Introduction

This documentation describes how to set up Alpine Linux using ZFS with a pool that is located in an encrypted partition. To encrypt the partition the Device Mapper crypt (dm-crypt) module and Linux Unified Key Setup (LUKS) is used.

Note that you must install the /boot/ directory on an unecrypted partition to boot correctly.

We'll be using the syslinux bootloader and traditional BIOS booting.

Requirements

• An instance Alpine on a medium other than the one you'll boot from, see official installation guide.

Note: We can't use the live environment from the installation isos, because we'll install the ZFS kernel module and that isn't possible with a read-only /boot (as provided by the iso).

Hard Disk Device Name

The following documentation uses the /dev/sda device as the installation destination. If your environment uses a different device name for your hard disk, use the corresponding device name in the examples.

Setting up Alpine Linux Using ZFS on Top of a LUKS Partition

To install Alpine Linux in a ZFS pool on top of a LUKS encrypted partition, you cannot use the official installation procedure. The installation requires several manual steps you must run in the Alpine Linux Live CD environment.

Preparing the Installation Environment

Before you begin to install Alpine Linux on the medium you intend to boot from, prepare the installation you already have:

• Update the apk cache:

# apk update

• Install the following packages required to set up ZFS and LUKS:

# apk add haveged cryptsetup e2fsprogs syslinux zfs zfs-$(uname -r | rev | cut -d'-' -f1 | rev)
# modprobe zfs

• Optionally, start the haveged service for unpredictable random numbers used for encryption:

# rc-service haveged start

Creating the Partition Layout

Linux requires an unencrypted /boot/ partition to boot. You can assign the remaining space for the encrypted ZFS pool.

• Start the fdisk utility to set up partitions:

# fdisk /dev/sda

• Create the /boot/ partition:

• Enter n → p → 1 → 1 → 100m to create a new 100 MB primary partition.

• Set the /boot/ partition active:

• Enter a → 1.

• Create the LUKS partition:

• Enter n → p → 2 to start creating the next partition. Press Enter to select the default start cylinder. Enter the size of partition. For example, 512m for 512 MB or 5g for 5 GB. Alternatively press Enter to set the maximum available size.

• To verify the settings, press p. The output shows, for example:

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sda1  *      2048   206847   204800  100M 83 Linux
/dev/sda2       206848 41943039 41736192 19.9G 83 Linux

• Press w to save the changes.

• Optionally, wipe the LUKS partition with random values:

# haveged -n 0 | dd of=/dev/sda2

Note: Depending on the size of the partition, this process can take several minutes to hours.

Encrypting the ZFS Partition

• To encrypt the partition which will later contain the LVM PV:

# cryptsetup luksFormat /dev/sda2

Note: Alpine Linux uses the en-us keyboard mapping when prompting for the password to encrypt the partition at boot time. If you changed the keyboard map in the temporary environment, the password you enter during encrypting the partition in this step, may not match the password you will enter during the system boots.

If you prefer setting an individual hashing algorithm and hashing schema:

• To run a benchmark:

# cryptsetup benchmark

• To encrypt the partition using individual settings, enter, for example:

# cryptsetup -v -c serpent-xts-plain64 -s 512 --hash whirlpool --iter-time 5000 --use-random luksFormat /dev/sda2

Creating the filesystems

• Open the LUKS partition:

# cryptsetup open --type luks /dev/sda2 crypt

Creating the ZFS pool

# zpool create -o ashift=12 -O normalization=formD -O atime=off -m none -R /mnt -O compression=lz4 tank /dev/mapper/crypt

Meaning of the zpool create options:

Option	Meaning
zpool create	Creating the zpool
-o ashift=12	4K blocks
-O normalization=formD	Set the default Unicode (UTF-8) normalization to 'formD'
-O atime=off	Disabling updates to file access time. This reduces writes to disk, but might cause issues with mailers, like mutt.
-m none	No mountpoint, as we'll handle this later.
-R /mnt	Set the altroot to /mnt. It's like a temporary mountpoint for the pool.
-O compression=lz4	Use lz4 compression for the pool. Is generally recommended.
tank	The pool name. tank will be used in throughout this guide.
/dev/mapper/crypt	The path to the block device ZFS will use.

After completing this, confirm that the pool has been created:

# zpool status

Should return something like:

  pool: tank
 state: ONLINE
  scan: none requested
config:

	NAME         STATE     READ WRITE CKSUM
	tank         ONLINE       0     0     0
	  crypt      ONLINE       0     0     0

errors: No known data errors

Creating the required datasets

# zfs create -o mountpoint=none -o canmount=off tank/ROOT
# zfs create -o mountpoint=/ tank/ROOT/alpine

Creating optional datasets (feel free to add your own)

# zfs create -o mountpoint=/home tank/HOME
# zfs create -o mountpoint=/var/log tank/LOG

Creating the /boot filesystem

# mkfs.ext4 /dev/sda1

Mounting the /boot filesystem

• Create /mnt/boot/ directory and mount the /dev/sda1 partition in this directory:

# mkdir /mnt/boot/
# mount -t ext4 /dev/sda1 /mnt/boot/

Installing Alpine Linux

In this step you will install Alpine Linux in the /mnt/ directory, which contains the mounted file system structure.

• Install Alpine Linux:

# setup-disk -m sys /mnt/

The installer downloads the latest packages to install the base installation. Additionally, the installer automatically creates the entries for the mount points in the fstab file (but we'll have to edit it manually later), which are currently mounted in the /mnt/ directory.

Note: The automatic writing of the master boot record (MBR) fails in this step. You will write the MBR later manually to the disk.

• To enable the operating system to decrypt the LUKS partition at boot time, create the /mnt/etc/crypttab file. Enter the following line into the file to decrypt the /dev/sda2 partition using the luks module and map it to the lvmcrypt name:

crypt    /dev/sda2    none    luks

• Delete the zfs entries in /mnt/etc/fstab as ZFS mounts them automagically, your fstab should look like this:

 UUID=6b4f2c9c-0a0f-4a8c-a73b-d2b47920ad6f	/boot	ext4	rw,relatime,stripe=4,data=ordered	0 2

• Edit the /mnt/etc/mkinitfs/mkinitfs.conf file and append the cryptsetup and zfs module to the features parameter:

features="ata base ide scsi usb virtio ext4 lvm cryptsetup zfs"

• Rebuild the initial RAM disk:

# mkinitfs -c /mnt/etc/mkinitfs/mkinitfs.conf -b /mnt/ $(ls /mnt/lib/modules/)

The command uses the settings from the mkinitfs.conf file set in the -c parameter to generate the RAM disk. The command is executed in the /mnt/ directory and the RAM disk is generated using the modules for the installed kernel. Without setting the kernel version using the $(ls /mnt/lib/modules/) option, mkinitfs tries to generate the RAM disk using the kernel version installed in the temporary environment, which can differ from the latest one installed by the setup-disk utility.

• Edit the /mnt/etc/update-extlinux.conf file, set the root ZFS dataset and append the following kernel options to the default_kernel_opts parameter:

root=tank/ROOT/alpine
default_kernel_opts="... cryptroot=/dev/sda2 cryptdm=crypt rootfstype=zfs"

The cryptroot parameter sets the name of the device that contains the root file system. The cryptdm parameter sets the name of the mapping previously set in the crypttab file. The rootfstype option sets the root filesystem type to zfs.

• Because the update-extlinux utility operators only on the /boot/ directory, temporarily change the root to the /mnt/ directory and update the boot loader configuration:

# chroot /mnt/
# update-extlinux
# exit

Ignore the errors the update-extlinux utility displays.

• Write the MBR to the /dev/sda device:

# dd bs=440 count=1 conv=notrunc if=/mnt/usr/share/syslinux/mbr.bin of=/dev/sda

Unmounting the filesystems

• Unmount /mnt/boot/:

# umount /mnt/boot/

• Unmount all zfs filesystems:

# zfs unmount -a

• Export all zfs pools:

# zpool export -a

• Close the lvmcrypt device:

# cryptsetup luksClose crypt

• Reboot the system:

# reboot

Troubleshooting

General Procedure

In case your system fails to boot, you can verify the settings and fix incorrect configurations:

• Prepare the temporary installation environment

• Load the ZFS kernel module:

# modprobe zfs

• Mount the file systems

# zpool import -R /mnt tank
# mount -t ext4 /dev/sda1 /mnt/boot

• Verify that you run the steps described in the Installing Alpine Linux section correctly. Update the configuration if necessary.

• Unmount the volumes and partitions

Todo: Multiple Disk ZFS on LUKS?