Immutable root with atomic upgrades: Difference between revisions
mNo edit summary |
Prabuanand (talk | contribs) m (fixed category name) |
||
(18 intermediate revisions by 5 users not shown) | |||
Line 1: | Line 1: | ||
=== What? === | === What? === | ||
This article provides a basic guide to setting up a read-only-root-based Alpine Linux system with several boot environments and atomic upgrades using | This article provides a basic guide to setting up a read-only-root-based Alpine Linux system with several boot environments and atomic upgrades using a modern bootloader and [[BTRFS|btrfs]]. | ||
=== Why? === | === Why? === | ||
Read-only root and atomic upgrades with ability to easily rollback or boot previous configurations is a concept that | Read-only root and atomic upgrades with the ability to easily rollback or boot previous configurations is a concept that has been gaining popularity recently. Distributions providing and promoting such features, for example, are [https://silverblue.fedoraproject.org/ Fedora Silverblue], [https://microos.opensuse.org/ Opensuse MicroOS], [https://nixos.org NixOS] and [https://guix.gnu.org GNU Guix]. | ||
While Alpine Linux has | While Alpine Linux has its killer features, it lacks the ones mentioned above on default setup. This is a proof of concept that it's possible to implement them in a minimal way on a minimal system. | ||
{{Note|Alpine Linux can also boot from RAM in '''diskless mode''' (see [[Installation]]) which supports preserving changes between reboots using [[lbu]].}} | {{Note|Alpine Linux can also boot from RAM in '''diskless mode''' (see [[Installation]]) which supports preserving changes between reboots using [[lbu]].}} | ||
= Preparation = | = Preparation = | ||
You should have bootable Alpine media. The process to obtain it | You should have bootable Alpine media. The process to obtain it is described on the [[Installation|installation page.]] | ||
= Partitioning disks = | = Partitioning disks = | ||
In this guide it | In this guide, it's assumed that you have a fresh UEFI system without an OS and have just booted into a live Alpine system using a USB flash drive or CD. | ||
The first step is creating partition table on your HDD/SSD target device (<code>/dev/sda</code> here): | The first step is creating partition table on your HDD/SSD target device (<code>/dev/sda</code> here): | ||
<pre># apk add gptfdisk | <pre># apk add gptfdisk | ||
Line 42: | Line 42: | ||
<pre># mount -t btrfs /dev/sda2 /mnt</pre> | <pre># mount -t btrfs /dev/sda2 /mnt</pre> | ||
= File system structure = | = File system structure = | ||
Now we should create file structure that would provide reliable atomic system upgrades.<br> | Now we should create the file structure that would provide reliable atomic system upgrades.<br> | ||
Start with following directories:<br> | Start with following directories:<br> | ||
<pre># mkdir /mnt/next</pre> | <pre># mkdir /mnt/next</pre> | ||
Line 49: | Line 49: | ||
Stores common non-snapshotting subvolumes.<br> | Stores common non-snapshotting subvolumes.<br> | ||
We may populate it right away: | We may populate it right away: | ||
<pre># btrfs subvolume create /mnt/commons/@var | <pre># btrfs subvolume create /mnt/commons/@var | ||
# btrfs subvolume create /mnt/commons/@home</pre> | # btrfs subvolume create /mnt/commons/@home</pre> | ||
Next, most important directories: | Next, most important directories: | ||
<pre># mkdir /mnt/snapshots</pre> | <pre># mkdir /mnt/snapshots</pre> | ||
Line 78: | Line 72: | ||
This setup allows us to just have static rEFInd config that points to to <code>/current/0/@</code>, <code>/current/1/@</code>, etc. while the actual underlying boot environment will change with each upgrade.<br> | This setup allows us to just have static rEFInd config that points to to <code>/current/0/@</code>, <code>/current/1/@</code>, etc. while the actual underlying boot environment will change with each upgrade.<br> | ||
But how will fs mounting services know which snapshot generation is currently loaded?<br> | But how will fs mounting services know which snapshot generation is currently loaded?<br> | ||
The answer is common <code>fstab</code> in the btrfs root.<br> | The answer is common <code>fstab</code> in the [[BTRFS|btrfs]] root.<br> | ||
Get UUIDs of the partitions first: | Get UUIDs of the partitions first: | ||
<pre># blkid > /mnt/fstab</pre> | <pre># blkid > /mnt/fstab</pre> | ||
Line 85: | Line 79: | ||
Example: | Example: | ||
<pre>UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 / btrfs subvol=CURRENT_SNAPSHOTS_PATH/@,ro,noatime 0 0 | <pre>UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 / btrfs subvol=CURRENT_SNAPSHOTS_PATH/@,ro,noatime 0 0 | ||
UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 /var | UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 /var btrfs subvol=/commons/@var,rw,noatime 0 0 | ||
UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 /home btrfs subvol=/commons/@home,rw,noatime 0 0 | UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 /home btrfs subvol=/commons/@home,rw,noatime 0 0 | ||
# UUID=2FE6-837A /boot/efi vfat rw,noatime,discard 0 2 | # UUID=2FE6-837A /boot/efi vfat rw,noatime,discard 0 2 | ||
Line 95: | Line 85: | ||
UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 swap swap rw,noatime,discard 0 0</pre> | UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 swap swap rw,noatime,discard 0 0</pre> | ||
<code>CURRENT_SNAPSHOTS_PATH</code> will be replaced by scripts with, for example, <code>/snapshots/20210411212549sdBXyLxg</code>, and the result will be piped into <code>/etc/fstab</code> of a created <code>@</code> snapshot during new generation preparations.<br> | <code>CURRENT_SNAPSHOTS_PATH</code> will be replaced by scripts with, for example, <code>/snapshots/20210411212549sdBXyLxg</code>, and the result will be piped into <code>/etc/fstab</code> of a created <code>@</code> snapshot during new generation preparations.<br> | ||
Root btrfs volume structure mounted on <code>/mnt</code>: | Root [[BTRFS|btrfs]] volume structure mounted on <code>/mnt</code>: | ||
<pre>|--mnt | <pre>|--mnt | ||
| |--commons | | |--commons | ||
| | |--@var | | | |--@var | ||
| | |--@home | | | |--@home | ||
| |--current | | |--current | ||
Line 118: | Line 104: | ||
= Base system install = | = Base system install = | ||
With the directory | With the directory structure prepared, we can begin installing a basic Alpine Linux system.<br> | ||
Considering that installation is done from Alpine system, we only need following parts of [[Alpine_Linux_in_a_chroot|the process]]: | Considering that installation is done from Alpine system, we only need following parts of [[Alpine_Linux_in_a_chroot|the process]]: | ||
<pre># apk -X https://dl-cdn.alpinelinux.org/alpine/latest-stable/main -U --allow-untrusted -p /mnt/snapshots/20210411212549sdBXyLxg/@ --initdb add alpine-base</pre> | <pre># apk -X https://dl-cdn.alpinelinux.org/alpine/latest-stable/main -U --allow-untrusted -p /mnt/snapshots/20210411212549sdBXyLxg/@ --initdb add alpine-base</pre> | ||
Line 138: | Line 124: | ||
# ln -s /tmp/resolv.conf /etc/resolv.conf</pre> | # ln -s /tmp/resolv.conf /etc/resolv.conf</pre> | ||
As soon as you in chroot, define repositories: | As soon as you're in chroot, define repositories: | ||
<pre># echo "https://dl-cdn.alpinelinux.org/alpine/latest-stable/main" > /etc/apk/repositories</pre> | <pre># echo "https://dl-cdn.alpinelinux.org/alpine/latest-stable/main" > /etc/apk/repositories</pre> | ||
This example shows only <code>main</code>, but you should also add <code>testing</code> and <code>community</code> if you need any packages in those.<br> | |||
Now it's time for firmware, kernel and btrfs packages: | Now it's time for the firmware, kernel, and btrfs packages: | ||
<pre># apk add -U linux-firmware linux-lts btrfs-progs</pre> | <pre># apk add -U linux-firmware linux-lts btrfs-progs</pre> | ||
You may want to change <code>linux-firmware</code> to a custom set of firmware packages suitable for | You may want to change <code>linux-firmware</code> to a custom set of firmware packages suitable for your system, for example, <code>linux-firmware-amd linux-firmware-amd-ucode linux-firmware-amdgpu linux-firmware-ath10k linux-firmware-qca</code> for a typical AMD laptop.<br> | ||
It | It's also important to add the <code>btrfs</code> feature to <code>mkinitfs.conf</code> and run <code>mkinitfs</code> manually: | ||
<pre># vi /etc/mkinitfs/mkinitfs.conf | <pre># vi /etc/mkinitfs/mkinitfs.conf | ||
# mkinitfs</pre> | # mkinitfs</pre> | ||
These steps prepare kernel and generate <code>initramfs</code> which | These steps prepare the kernel and generate the <code>initramfs</code>, which will be used later to boot from our first snapshot.<br> | ||
After that you should install any package you may need on first boot. | After that, you should install any package you may need on first boot. | ||
{{Warning|In case your PC only has wireless connection you should also install any suitable networking software, like <code>iwd</code> in this example, so you will not end up severed from network on your first boot.}} | {{Warning|In case your PC only has wireless connection you should also install any suitable networking software, like <code>iwd</code> in this example, so you will not end up severed from network on your first boot.}} | ||
{{Note|Due to root being immutable during operation, it's recommended to install | {{Note|Due to root being immutable during operation, it's recommended to install the <code>openresolv</code> package to support changing the network connection. In this case, <code>/etc/resolvconf.conf</code> should have <code>resolv_conf{{=}}/tmp/resolv.conf</code>, <code>/etc/resolv.conf</code> should be moved to <code>/tmp/resolv.conf</code>, and a link to the new resolv.conf location should be created: <code>ln -sfn /tmp/resolv.conf /etc/resolv.conf</code>. | ||
You may also use static DNS, but this would make your network activity directly identifiable to the DNS server provider, therefore it's not recommended. | You may also use static DNS, but this would make your network activity directly identifiable to the DNS server provider, therefore it's not recommended. | ||
}} | }} | ||
Line 159: | Line 145: | ||
<pre># passwd root</pre> | <pre># passwd root</pre> | ||
With the snapshot prepared and configured we can chroot out of it and unmount everything: | Don't forget to add essential services to their respective runlevels: | ||
<pre>rc-update add devfs sysinit | |||
rc-update add dmesg sysinit | |||
rc-update add mdev sysinit | |||
rc-update add hwdrivers sysinit | |||
rc-update add hwclock boot | |||
rc-update add modules boot | |||
rc-update add sysctl boot | |||
rc-update add hostname boot | |||
rc-update add bootmisc boot | |||
rc-update add syslog boot | |||
rc-update add mount-ro shutdown | |||
rc-update add killprocs shutdown | |||
rc-update add savecache shutdown</pre> | |||
With the snapshot prepared and configured, we can chroot out of it and unmount everything: | |||
<pre># umount -a | <pre># umount -a | ||
# exit</pre> | # exit</pre> | ||
Finish editing snapshot by setting <code>ro</code> flag and unmounting root volume: | Finish editing the snapshot by setting the <code>ro</code> flag and unmounting the root volume: | ||
<pre># btrfs property set -ts "/mnt/snapshots/20210411212549sdBXyLxg/@" ro true | <pre># btrfs property set -ts "/mnt/snapshots/20210411212549sdBXyLxg/@" ro true | ||
# umount /mnt</pre> | # umount /mnt</pre> | ||
= Bootloader installation = | = Bootloader installation = | ||
Mount the EFI partition: | Mount the EFI partition: | ||
<pre># mount -t vfat /dev/sda1 /mnt | <pre># mount -t vfat /dev/sda1 /mnt | ||
# mkdir /mnt/EFI</pre> | # mkdir /mnt/EFI</pre> | ||
Unpack prepared rEFInd archive and copy relevant files to <code>/mnt/EFI/</code> | == Bootloader configuration == | ||
<pre># | There are 2 options as examples of the bootloader installation: <code>rEFInd</code> and <code>GRUB</code>. | ||
# cp -r | Sometimes one of them will refuse to work on a system for no particular reason, in this case try the other one. | ||
=== rEFInd === | |||
Check the latest version number of the <code>refind</code> package: | |||
<pre># apk info -X https://dl-cdn.alpinelinux.org/alpine/edge/testing -U refind</pre> | |||
Download the latest version (replace 0.13.2-r3 in the example below) of the <code>refind</code> package: | |||
<pre># wget https://dl-cdn.alpinelinux.org/alpine/edge/testing/x86_64/refind-0.13.2-r3.apk</pre> | |||
Unpack the prepared rEFInd archive and copy relevant files to <code>/mnt/EFI/</code> | |||
<pre># tar -xzf refind-0.13.2-r3.apk | |||
# cp -r usr/share/refind /mnt/EFI/ | |||
# cd /mnt/EFI/refind</pre> | # cd /mnt/EFI/refind</pre> | ||
Rename config file and edit it: | Rename config file and edit it: | ||
<pre># mv refind.conf-sample refind.conf | <pre># mv refind.conf-sample refind.conf | ||
# vi refind.conf</pre> | # vi refind.conf</pre> | ||
And append following to the end of the file, remember to replace example UUIDs with your own for <code>root</code> (btrfs partition) and <code>resume</code> (swap partition) | And append following to the end of the file, remember to replace example UUIDs with your own for <code>root</code> ([[BTRFS|btrfs]] partition) and <code>resume</code> (swap partition). Keep in mind that if you named the btrfs volume other than <code>ROOT</code> during "Partitioning disks" stage, you have to change the <code>volume</code> field below accordingly. | ||
<pre> | <pre> | ||
menuentry "Alpine Linux" { | menuentry "Alpine Linux" { | ||
Line 208: | Line 216: | ||
} | } | ||
}</pre> | }</pre> | ||
{{Note|<code>"ROOT"</code> is the <code>PARTLABEL</code> of the btrfs partition. You may also use <code>PARTUUID</code> instead. To get both <code>blkid</code> from <code>blkid</code> package can be used. <code>blkid</code> included in busybox does not provide this information. }} | {{Note|<code>"ROOT"</code> is the <code>PARTLABEL</code> of the [[BTRFS|btrfs]] partition. You may also use <code>PARTUUID</code> instead. To get both <code>blkid</code> from <code>blkid</code> package can be used. <code>blkid</code> included in busybox does not provide this information. }} | ||
To add | === GRUB === | ||
<pre># apk add grub-efi</pre> | |||
GRUB requires two configuration files this time as we will use <code>grub-mkstandalone</code>. | |||
The first configuration file is internal and should only point to the second file, where we store the menu: | |||
<pre># cd /tmp | |||
# vi grub_internal.cfg</pre> | |||
Set the contents to the following, but make sure to replace <code>2FE6-837A</code> with your own EFI partition UUID: | |||
<pre>insmod part_gpt | |||
insmod fat | |||
search --set efi --fs-uuid 2FE6-837A | |||
configfile (${efi})/EFI/grub/grub.cfg</pre> | |||
The second config file is the main config where we describe the entire boot menu. | |||
<pre># vi grub.cfg</pre> | |||
Set to contain, but replace UUIDs: | |||
<pre>set timeout=3 | |||
menuentry "Alpine Linux Current" { | |||
search --set root --fs-uuid b9ff5e7b-e128-4e64-861a-2fdd794a9828 | |||
linux /current/0/@/boot/vmlinuz-edge root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/0/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash | |||
initrd /current/0/@/boot/initramfs-edge | |||
} | |||
menuentry "Alpine Linux Snapshot 1" { | |||
search --set root --fs-uuid b9ff5e7b-e128-4e64-861a-2fdd794a9828 | |||
linux /current/1/@/boot/vmlinuz-edge root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/1/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash | |||
initrd /current/1/@/boot/initramfs-edge | |||
} | |||
menuentry "Alpine Linux Snapshot 2" { | |||
search --set root --fs-uuid b9ff5e7b-e128-4e64-861a-2fdd794a9828 | |||
linux /current/2/@/boot/vmlinuz-edge root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/2/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash | |||
initrd /current/2/@/boot/initramfs-edge | |||
} | |||
menuentry "Alpine Linux Snapshot 3" { | |||
search --set root --fs-uuid b9ff5e7b-e128-4e64-861a-2fdd794a9828 | |||
linux /current/3/@/boot/vmlinuz-edge root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/3/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash | |||
initrd /current/3/@/boot/initramfs-edge | |||
}</pre> | |||
Generate the <code>grubx64.efi</code> binary: | |||
<pre># grub-mkstandalone -O x86_64-efi -o grubx64.efi "boot/grub/grub.cfg=/tmp/grub_internal.cfg" | |||
# mkdir /mnt/EFI/grub | |||
# mv grubx64.efi /mnt/EFI/grub/ | |||
# mv grub.cfg /mnt/EFI/grub/</pre> | |||
== Adding EFI boot entry == | |||
To add the chosen bootloader to UEFI, <code>efibootmgr</code> is a suitable tool. The following example is for rEFInd, but could be easily adjusted for GRUB: | |||
<pre># apk add efibootmgr | <pre># apk add efibootmgr | ||
# efibootmgr --create --disk /dev/sda --part 1 --loader /EFI/refind/refind_x64.efi --label "rEFInd" --verbose</pre> | # efibootmgr --create --disk /dev/sda --part 1 --loader /EFI/refind/refind_x64.efi --label "rEFInd" --verbose</pre> | ||
<code>/dev/sda</code> is our disk device and <code>1</code> is the number of partition containing | <code>/dev/sda</code> is our disk device and <code>1</code> is the number of the FAT32 partition containing the bootloader data. | ||
= Updating or altering the system = | = Updating or altering the system = | ||
<pre># touch /sbin/sysmut | {{Warning|Without the following step or an alternative you will have no easy way to mutate the installed system.}} | ||
# chmod +x /sbin/sysmut | {{Warning|These examples are implemented using <code>execline</code> and require the <code>execline</code> package in the system.}} | ||
# vi /sbin/sysmut</pre> | {{Note|These could surely be implemented in POSIX shell, however, <code>execline</code> provides a number of runtime advantages and the resulting script is much more readable.}} | ||
<pre># touch /usr/sbin/sysmut | |||
# chmod +x /usr/sbin/sysmut | |||
# vi /usr/sbin/sysmut</pre> | |||
Example script to mutate the the system: | Example script to mutate the the system: | ||
<pre>#!/bin/ | <pre>#!/bin/execlineb -W | ||
unshare --mount | |||
mkdir -p | importas -D 0 source 1 | ||
mount -t btrfs -o rw,noatime UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 | define mnt /media/root | ||
if { mkdir -p ${mnt} } | |||
mkdir -p | if { mount -t btrfs -o rw,noatime UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 ${mnt} } | ||
btrfs subvolume snapshot | foreground { | ||
sed | backtick -E dt { | ||
mount -t proc none | date -u +%Y%m%d%H%M%S | ||
mount -t sysfs sys | } | ||
mount -o bind /dev | backtick -E rnd { | ||
pipeline { cat /dev/urandom } | |||
btrfs property set -ts | pipeline { tr -dc a-zA-Z } | ||
pipeline { fold -w 8 } | |||
mkdir -p | head -n 1 | ||
ln -s | } | ||
cp -P | define newsnap ${dt}${rnd} | ||
cp -P | if { mkdir -p ${mnt}/snapshots/${newsnap} } | ||
cp -P | if { btrfs subvolume snapshot ${mnt}/current/${source}/@ ${mnt}/snapshots/${newsnap}/@ } | ||
mkdir -p | if { | ||
ln -sfn | redirfd -w 1 ${mnt}/snapshots/${newsnap}/@/etc/fstab | ||
mv | sed s#CURRENT_SNAPSHOTS_PATH#/snapshots/${newsnap}#g ${mnt}/fstab | ||
} | |||
if { mount -t proc none ${mnt}/snapshots/${newsnap}/@/proc } | |||
umount | if { mount -t sysfs sys ${mnt}/snapshots/${newsnap}/@/sys } | ||
if { mount -o bind,ro /dev ${mnt}/snapshots/${newsnap}/@/dev } | |||
foreground { | |||
foreground { mount -o bind,ro /etc/resolv.conf ${mnt}/snapshots/${newsnap}/@/etc/resolv.conf } | |||
foreground { | |||
chroot ${mnt}/snapshots/${newsnap}/@ | |||
foreground { mount -a } | |||
foreground { sh } | |||
importas apply ? | |||
foreground { umount -a } | |||
exit ${apply} | |||
} | |||
importas apply ? | |||
foreground { redirfd -w 2 /dev/null umount ${mnt}/snapshots/${newsnap}/@/etc/resolv.conf } | |||
ifelse { exit ${apply} } { | |||
if { btrfs property set -ts ${mnt}/snapshots/${newsnap}/@ ro true } | |||
define newlink ${dt}${rnd} | |||
if { mkdir -p ${mnt}/links/${newlink} } | |||
if { ln -s ../../snapshots/${newsnap} ${mnt}/links/${newlink}/0 } | |||
if { cp -P ${mnt}/current/0 ${mnt}/links/${newlink}/1 } | |||
if { cp -P ${mnt}/current/1 ${mnt}/links/${newlink}/2 } | |||
if { cp -P ${mnt}/current/2 ${mnt}/links/${newlink}/3 } | |||
if { mkdir -p ${mnt}/next } | |||
if { ln -sfn ./links/${newlink} ${mnt}/next/current } | |||
if { mv ${mnt}/next/current ${mnt}/ } | |||
echo "Changes applied" | |||
} | |||
echo "Changes discarded" | |||
} | |||
foreground { redirfd -w 2 /dev/null umount ${mnt}/snapshots/${newsnap}/@/proc } | |||
foreground { redirfd -w 2 /dev/null umount ${mnt}/snapshots/${newsnap}/@/sys } | |||
redirfd -w 2 /dev/null umount ${mnt}/snapshots/${newsnap}/@/dev | |||
} | |||
umount ${mnt} | |||
</pre> | |||
It will get you into the root shell chrooted into the new snapshot, where you can apply any change you like.<br> | It will get you into the root shell chrooted into the new snapshot, where you can apply any change you like. The origin of the new snapshot is defined by the first and only argument, in form of number. If no argument provided the <code>0</code> (current latest) is taken as origin.<br> | ||
If chroot shell exits with an error, there will be no switch to the new snapshots. This means you can manually discard changes while in the chroot by: | If chroot shell exits with an error, there will be no switch to the new snapshots. This means you can manually discard changes while in the chroot by: | ||
Line 255: | Line 342: | ||
Unused snapshots can be garbage-collected by: | Unused snapshots can be garbage-collected by: | ||
<pre># touch /sbin/syscln | <pre># touch /usr/sbin/syscln | ||
# chmod +x /sbin/syscln | # chmod +x /usr/sbin/syscln | ||
# vi /sbin/syscln</pre> | # vi /usr/sbin/syscln</pre> | ||
<pre>#!/bin/ | <pre>#!/bin/execlineb -W | ||
unshare --mount | |||
mkdir -p | define mnt /media/root | ||
mount -t btrfs -o rw,noatime UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 | if { mkdir -p ${mnt} } | ||
if { mount -t btrfs -o rw,noatime,compress=zstd:3 UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 ${mnt} } | |||
foreground { | |||
foreground { | |||
pipeline { | |||
foreground { | |||
} | pipeline { | ||
find -H ${mnt}/snapshots/ -maxdepth 1 -mindepth 1 -print0 | |||
} | |||
xargs -0 -r realpath | |||
} | |||
pipeline { | |||
find -H ${mnt}/current/ -maxdepth 1 -mindepth 1 -print0 | |||
} | |||
xargs -0 -r realpath | |||
} | |||
pipeline { tr \\n \\0 } | |||
pipeline { sort -z } | |||
pipeline { uniq -u -z } | |||
pipeline { xargs -0 -r -n 1 -I [] find -H [] -maxdepth 1 -mindepth 1 -print0 } | |||
xargs -0 -r btrfs subvolume delete | |||
umount | } | ||
foreground { find -H ${mnt}/snapshots/ -maxdepth 1 -mindepth 1 -empty -type d -delete } | |||
foreground { | |||
pipeline { | |||
foreground { | |||
pipeline { | |||
find -H ${mnt}/links/ -maxdepth 1 -mindepth 1 -print0 | |||
} | |||
xargs -0 -r realpath | |||
} | |||
realpath ${mnt}/current | |||
} | |||
pipeline { tr \\n \\0 } | |||
pipeline { sort -z } | |||
pipeline { uniq -u -z } | |||
pipeline { xargs -0 -r -n 1 -I [] find -H [] -maxdepth 1 -mindepth 1 -print0 } | |||
xargs -0 -r -n 1 unlink | |||
} | |||
find -H ${mnt}/links/ -maxdepth 1 -mindepth 1 -empty -type d -delete | |||
} | |||
umount ${mnt} | |||
</pre> | |||
= Allowing temporary runtime alterations = | = Allowing temporary runtime alterations = | ||
You can use <code>overlayfs</code> with <code>tmpfs</code> built into Alpine's init script to allow changes in the rootfs which will be automatically reverted upon reboot.<br> | You can use <code>overlayfs</code> with <code>tmpfs</code> built into Alpine's init script to allow changes in the rootfs which will be automatically reverted upon reboot.<br> | ||
To make use this | To make use of this, add <code>overlaytmpfs</code> to the kernel boot options in <code>refind.conf</code>, e.g.: | ||
<pre> | <pre> | ||
... | ... | ||
Line 296: | Line 404: | ||
... | ... | ||
</pre> | </pre> | ||
[[Category:Installation]][[Category:Filesystems]] |
Latest revision as of 03:59, 13 September 2024
What?
This article provides a basic guide to setting up a read-only-root-based Alpine Linux system with several boot environments and atomic upgrades using a modern bootloader and btrfs.
Why?
Read-only root and atomic upgrades with the ability to easily rollback or boot previous configurations is a concept that has been gaining popularity recently. Distributions providing and promoting such features, for example, are Fedora Silverblue, Opensuse MicroOS, NixOS and GNU Guix.
While Alpine Linux has its killer features, it lacks the ones mentioned above on default setup. This is a proof of concept that it's possible to implement them in a minimal way on a minimal system.
Preparation
You should have bootable Alpine media. The process to obtain it is described on the installation page.
Partitioning disks
In this guide, it's assumed that you have a fresh UEFI system without an OS and have just booted into a live Alpine system using a USB flash drive or CD.
The first step is creating partition table on your HDD/SSD target device (/dev/sda
here):
# apk add gptfdisk # gdisk /dev/sda > o ↵ > y ↵ > w ↵ > y ↵
Now we can define the partitions:
# cgdisk /dev/sda
Partition creation process consists of several steps:
- Start sector - you can safely use default value by pressing ↵
- Size
- Type (as hex code) - EFI is ef00, Linux filesystem is 8300, Swap is 8200.
Result table:
Part. # Size Partition Type Partition Name ---------------------------------------------------------------- 1 200.0 MiB EFI System EFI 2 200.0 GiB Linux filesystem ROOT 3 32.0 GiB Linux swap SWAP
ROOT
partition name will later be used in rEFInd configuration to identify boot volume.
Next step is creating filesystems:
# mkfs.vfat -F32 /dev/sda1 # mkfs.btrfs /dev/sda2 # mkswap /dev/sda3
Now we can mount our root volume:
# mount -t btrfs /dev/sda2 /mnt
File system structure
Now we should create the file structure that would provide reliable atomic system upgrades.
Start with following directories:
# mkdir /mnt/next
Stores next current
link, is necessary due to how busybox mv
does atomic link replacement.
# mkdir /mnt/commons
Stores common non-snapshotting subvolumes.
We may populate it right away:
# btrfs subvolume create /mnt/commons/@var # btrfs subvolume create /mnt/commons/@home
Next, most important directories:
# mkdir /mnt/snapshots
Stores directories containing snapshots belonging to one generation.
# mkdir /mnt/links
Stores generations of directories containing links to snapshot generations.
Let's create first generation and populate it with one OS root snapshot @
:
# NEWSNAPSHOTS="$(date -u +"%Y%m%d%H%M%S")$(cat /dev/urandom | tr -dc 'a-zA-Z' | fold -w 8 | head -n 1)" # mkdir "/mnt/snapshots/$NEWSNAPSHOTS" # btrfs subvolume create /mnt/snapshots/$NEWSNAPSHOTS/@
Populate links
:
# NEWLINKS="$(date -u +"%Y%m%d%H%M%S")$(cat /dev/urandom | tr -dc 'a-zA-Z' | fold -w 8 | head -n 1)" # mkdir "/mnt/links/$NEWLINKS" # ln -s "../../snapshots/$NEWSNAPSHOTS" "/mnt/links/$NEWLINKS/0" # ln -s "../../snapshots/$NEWSNAPSHOTS" "/mnt/links/$NEWLINKS/1" # ln -s "../../snapshots/$NEWSNAPSHOTS" "/mnt/links/$NEWLINKS/2" # ln -s "../../snapshots/$NEWSNAPSHOTS" "/mnt/links/$NEWLINKS/3"
You can have as many links as you like, just apply changes to rEFInd config and upgrade scripts described below accordingly.
Link that will point to latest links generation:
# ln -s "./links/$NEWLINKS" /mnt/current
This setup allows us to just have static rEFInd config that points to to /current/0/@
, /current/1/@
, etc. while the actual underlying boot environment will change with each upgrade.
But how will fs mounting services know which snapshot generation is currently loaded?
The answer is common fstab
in the btrfs root.
Get UUIDs of the partitions first:
# blkid > /mnt/fstab
Now edit fstab accordingly:
# vi /mnt/fstab
Example:
UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 / btrfs subvol=CURRENT_SNAPSHOTS_PATH/@,ro,noatime 0 0 UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 /var btrfs subvol=/commons/@var,rw,noatime 0 0 UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 /home btrfs subvol=/commons/@home,rw,noatime 0 0 # UUID=2FE6-837A /boot/efi vfat rw,noatime,discard 0 2 tmpfs /tmp tmpfs mode=1777,noatime,nosuid,nodev,size=2G 0 0 UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 swap swap rw,noatime,discard 0 0
CURRENT_SNAPSHOTS_PATH
will be replaced by scripts with, for example, /snapshots/20210411212549sdBXyLxg
, and the result will be piped into /etc/fstab
of a created @
snapshot during new generation preparations.
Root btrfs volume structure mounted on /mnt
:
|--mnt | |--commons | | |--@var | | |--@home | |--current | |--fstab | |--links | | |--20210411213742qwrXAJBz | | | |--0 | | | |--1 | | | |--2 | | | |--3 | |--next | |--snapshots | | |--20210411212549sdBXyLxg | | | |--@
Base system install
With the directory structure prepared, we can begin installing a basic Alpine Linux system.
Considering that installation is done from Alpine system, we only need following parts of the process:
# apk -X https://dl-cdn.alpinelinux.org/alpine/latest-stable/main -U --allow-untrusted -p /mnt/snapshots/20210411212549sdBXyLxg/@ --initdb add alpine-base
Now we can setup basic chroot to complete the installation process:
# export SNP="/mnt/snapshots/20210411212549sdBXyLxg/@" # mount -o bind /dev $SNP/dev # mount -t proc none $SNP/proc # mount -t sysfs sys $SNP/sys # sed "s#CURRENT_SNAPSHOTS_PATH#/snapshots/20210411212549sdBXyLxg#g" /mnt/fstab > "$SNP/etc/fstab" # cp -L /etc/resolv.conf "$SNP/etc/" # chroot "$SNP" /bin/sh # mount -a # mv /etc/resolv.conf /tmp/ # ln -s /tmp/resolv.conf /etc/resolv.conf
As soon as you're in chroot, define repositories:
# echo "https://dl-cdn.alpinelinux.org/alpine/latest-stable/main" > /etc/apk/repositories
This example shows only main
, but you should also add testing
and community
if you need any packages in those.
Now it's time for the firmware, kernel, and btrfs packages:
# apk add -U linux-firmware linux-lts btrfs-progs
You may want to change linux-firmware
to a custom set of firmware packages suitable for your system, for example, linux-firmware-amd linux-firmware-amd-ucode linux-firmware-amdgpu linux-firmware-ath10k linux-firmware-qca
for a typical AMD laptop.
It's also important to add the btrfs
feature to mkinitfs.conf
and run mkinitfs
manually:
# vi /etc/mkinitfs/mkinitfs.conf # mkinitfs
These steps prepare the kernel and generate the initramfs
, which will be used later to boot from our first snapshot.
After that, you should install any package you may need on first boot.
iwd
in this example, so you will not end up severed from network on your first boot.
openresolv
package to support changing the network connection. In this case, /etc/resolvconf.conf
should have resolv_conf=/tmp/resolv.conf
, /etc/resolv.conf
should be moved to /tmp/resolv.conf
, and a link to the new resolv.conf location should be created: ln -sfn /tmp/resolv.conf /etc/resolv.conf
.
You may also use static DNS, but this would make your network activity directly identifiable to the DNS server provider, therefore it's not recommended.
Now, configure the system, start with setting a password for the root:
# passwd root
Don't forget to add essential services to their respective runlevels:
rc-update add devfs sysinit rc-update add dmesg sysinit rc-update add mdev sysinit rc-update add hwdrivers sysinit rc-update add hwclock boot rc-update add modules boot rc-update add sysctl boot rc-update add hostname boot rc-update add bootmisc boot rc-update add syslog boot rc-update add mount-ro shutdown rc-update add killprocs shutdown rc-update add savecache shutdown
With the snapshot prepared and configured, we can chroot out of it and unmount everything:
# umount -a # exit
Finish editing the snapshot by setting the ro
flag and unmounting the root volume:
# btrfs property set -ts "/mnt/snapshots/20210411212549sdBXyLxg/@" ro true # umount /mnt
Bootloader installation
Mount the EFI partition:
# mount -t vfat /dev/sda1 /mnt # mkdir /mnt/EFI
Bootloader configuration
There are 2 options as examples of the bootloader installation: rEFInd
and GRUB
.
Sometimes one of them will refuse to work on a system for no particular reason, in this case try the other one.
rEFInd
Check the latest version number of the refind
package:
# apk info -X https://dl-cdn.alpinelinux.org/alpine/edge/testing -U refind
Download the latest version (replace 0.13.2-r3 in the example below) of the refind
package:
# wget https://dl-cdn.alpinelinux.org/alpine/edge/testing/x86_64/refind-0.13.2-r3.apk
Unpack the prepared rEFInd archive and copy relevant files to /mnt/EFI/
# tar -xzf refind-0.13.2-r3.apk # cp -r usr/share/refind /mnt/EFI/ # cd /mnt/EFI/refind
Rename config file and edit it:
# mv refind.conf-sample refind.conf # vi refind.conf
And append following to the end of the file, remember to replace example UUIDs with your own for root
(btrfs partition) and resume
(swap partition). Keep in mind that if you named the btrfs volume other than ROOT
during "Partitioning disks" stage, you have to change the volume
field below accordingly.
menuentry "Alpine Linux" { icon /EFI/refind/icons/os_linux.png volume "ROOT" loader /current/0/@/boot/vmlinuz-lts initrd /current/0/@/boot/initramfs-lts options "root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/0/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash" submenuentry "Boot fallback 1" { loader /current/1/@/boot/vmlinuz-lts initrd /current/1/@/boot/initramfs-lts options "root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/1/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash" } submenuentry "Boot fallback 2" { loader /current/2/@/boot/vmlinuz-lts initrd /current/2/@/boot/initramfs-lts options "root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/2/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash" } submenuentry "Boot fallback 3" { loader /current/3/@/boot/vmlinuz-lts initrd /current/3/@/boot/initramfs-lts options "root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/3/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash" } }
"ROOT"
is the PARTLABEL
of the btrfs partition. You may also use PARTUUID
instead. To get both blkid
from blkid
package can be used. blkid
included in busybox does not provide this information. GRUB
# apk add grub-efi
GRUB requires two configuration files this time as we will use grub-mkstandalone
.
The first configuration file is internal and should only point to the second file, where we store the menu:
# cd /tmp # vi grub_internal.cfg
Set the contents to the following, but make sure to replace 2FE6-837A
with your own EFI partition UUID:
insmod part_gpt insmod fat search --set efi --fs-uuid 2FE6-837A configfile (${efi})/EFI/grub/grub.cfg
The second config file is the main config where we describe the entire boot menu.
# vi grub.cfg
Set to contain, but replace UUIDs:
set timeout=3 menuentry "Alpine Linux Current" { search --set root --fs-uuid b9ff5e7b-e128-4e64-861a-2fdd794a9828 linux /current/0/@/boot/vmlinuz-edge root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/0/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash initrd /current/0/@/boot/initramfs-edge } menuentry "Alpine Linux Snapshot 1" { search --set root --fs-uuid b9ff5e7b-e128-4e64-861a-2fdd794a9828 linux /current/1/@/boot/vmlinuz-edge root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/1/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash initrd /current/1/@/boot/initramfs-edge } menuentry "Alpine Linux Snapshot 2" { search --set root --fs-uuid b9ff5e7b-e128-4e64-861a-2fdd794a9828 linux /current/2/@/boot/vmlinuz-edge root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/2/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash initrd /current/2/@/boot/initramfs-edge } menuentry "Alpine Linux Snapshot 3" { search --set root --fs-uuid b9ff5e7b-e128-4e64-861a-2fdd794a9828 linux /current/3/@/boot/vmlinuz-edge root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/3/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 quiet splash initrd /current/3/@/boot/initramfs-edge }
Generate the grubx64.efi
binary:
# grub-mkstandalone -O x86_64-efi -o grubx64.efi "boot/grub/grub.cfg=/tmp/grub_internal.cfg" # mkdir /mnt/EFI/grub # mv grubx64.efi /mnt/EFI/grub/ # mv grub.cfg /mnt/EFI/grub/
Adding EFI boot entry
To add the chosen bootloader to UEFI, efibootmgr
is a suitable tool. The following example is for rEFInd, but could be easily adjusted for GRUB:
# apk add efibootmgr # efibootmgr --create --disk /dev/sda --part 1 --loader /EFI/refind/refind_x64.efi --label "rEFInd" --verbose
/dev/sda
is our disk device and 1
is the number of the FAT32 partition containing the bootloader data.
Updating or altering the system
execline
and require the execline
package in the system.
execline
provides a number of runtime advantages and the resulting script is much more readable.# touch /usr/sbin/sysmut # chmod +x /usr/sbin/sysmut # vi /usr/sbin/sysmut
Example script to mutate the the system:
#!/bin/execlineb -W unshare --mount importas -D 0 source 1 define mnt /media/root if { mkdir -p ${mnt} } if { mount -t btrfs -o rw,noatime UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 ${mnt} } foreground { backtick -E dt { date -u +%Y%m%d%H%M%S } backtick -E rnd { pipeline { cat /dev/urandom } pipeline { tr -dc a-zA-Z } pipeline { fold -w 8 } head -n 1 } define newsnap ${dt}${rnd} if { mkdir -p ${mnt}/snapshots/${newsnap} } if { btrfs subvolume snapshot ${mnt}/current/${source}/@ ${mnt}/snapshots/${newsnap}/@ } if { redirfd -w 1 ${mnt}/snapshots/${newsnap}/@/etc/fstab sed s#CURRENT_SNAPSHOTS_PATH#/snapshots/${newsnap}#g ${mnt}/fstab } if { mount -t proc none ${mnt}/snapshots/${newsnap}/@/proc } if { mount -t sysfs sys ${mnt}/snapshots/${newsnap}/@/sys } if { mount -o bind,ro /dev ${mnt}/snapshots/${newsnap}/@/dev } foreground { foreground { mount -o bind,ro /etc/resolv.conf ${mnt}/snapshots/${newsnap}/@/etc/resolv.conf } foreground { chroot ${mnt}/snapshots/${newsnap}/@ foreground { mount -a } foreground { sh } importas apply ? foreground { umount -a } exit ${apply} } importas apply ? foreground { redirfd -w 2 /dev/null umount ${mnt}/snapshots/${newsnap}/@/etc/resolv.conf } ifelse { exit ${apply} } { if { btrfs property set -ts ${mnt}/snapshots/${newsnap}/@ ro true } define newlink ${dt}${rnd} if { mkdir -p ${mnt}/links/${newlink} } if { ln -s ../../snapshots/${newsnap} ${mnt}/links/${newlink}/0 } if { cp -P ${mnt}/current/0 ${mnt}/links/${newlink}/1 } if { cp -P ${mnt}/current/1 ${mnt}/links/${newlink}/2 } if { cp -P ${mnt}/current/2 ${mnt}/links/${newlink}/3 } if { mkdir -p ${mnt}/next } if { ln -sfn ./links/${newlink} ${mnt}/next/current } if { mv ${mnt}/next/current ${mnt}/ } echo "Changes applied" } echo "Changes discarded" } foreground { redirfd -w 2 /dev/null umount ${mnt}/snapshots/${newsnap}/@/proc } foreground { redirfd -w 2 /dev/null umount ${mnt}/snapshots/${newsnap}/@/sys } redirfd -w 2 /dev/null umount ${mnt}/snapshots/${newsnap}/@/dev } umount ${mnt}
It will get you into the root shell chrooted into the new snapshot, where you can apply any change you like. The origin of the new snapshot is defined by the first and only argument, in form of number. If no argument provided the 0
(current latest) is taken as origin.
If chroot shell exits with an error, there will be no switch to the new snapshots. This means you can manually discard changes while in the chroot by:
# exit 1
Deleting unused snapshots
Unused snapshots can be garbage-collected by:
# touch /usr/sbin/syscln # chmod +x /usr/sbin/syscln # vi /usr/sbin/syscln
#!/bin/execlineb -W unshare --mount define mnt /media/root if { mkdir -p ${mnt} } if { mount -t btrfs -o rw,noatime,compress=zstd:3 UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 ${mnt} } foreground { foreground { pipeline { foreground { pipeline { find -H ${mnt}/snapshots/ -maxdepth 1 -mindepth 1 -print0 } xargs -0 -r realpath } pipeline { find -H ${mnt}/current/ -maxdepth 1 -mindepth 1 -print0 } xargs -0 -r realpath } pipeline { tr \\n \\0 } pipeline { sort -z } pipeline { uniq -u -z } pipeline { xargs -0 -r -n 1 -I [] find -H [] -maxdepth 1 -mindepth 1 -print0 } xargs -0 -r btrfs subvolume delete } foreground { find -H ${mnt}/snapshots/ -maxdepth 1 -mindepth 1 -empty -type d -delete } foreground { pipeline { foreground { pipeline { find -H ${mnt}/links/ -maxdepth 1 -mindepth 1 -print0 } xargs -0 -r realpath } realpath ${mnt}/current } pipeline { tr \\n \\0 } pipeline { sort -z } pipeline { uniq -u -z } pipeline { xargs -0 -r -n 1 -I [] find -H [] -maxdepth 1 -mindepth 1 -print0 } xargs -0 -r -n 1 unlink } find -H ${mnt}/links/ -maxdepth 1 -mindepth 1 -empty -type d -delete } umount ${mnt}
Allowing temporary runtime alterations
You can use overlayfs
with tmpfs
built into Alpine's init script to allow changes in the rootfs which will be automatically reverted upon reboot.
To make use of this, add overlaytmpfs
to the kernel boot options in refind.conf
, e.g.:
... initrd /current/0/@/boot/initramfs-lts options "root=UUID=b9ff5e7b-e128-4e64-861a-2fdd794a9828 rootfstype=btrfs rootflags=subvol=/current/0/@,ro,noatime resume=UUID=f0239163-9d46-47c1-67a4-3ee1d63d0676 overlaytmpfs quiet splash" submenuentry "Boot fallback 1" { ...