OverlayFS

This page documents few use cases that makes use of OverlayFS in Alpine Linux installations. An OverlayFS also known as overlay-filesystem, tries to present a filesystem which is the result of overlaying one filesystem on top of the other.

An overlay filesystem combines two filesystems - an ‘upper’ filesystem and a ‘lower’ filesystem. When a name exists in both filesystems, the object in the ‘upper’ filesystem is visible while the object in the ‘lower’ filesystem is either hidden or, in the case of directories, merged with the ‘upper’ object.

Loopback image with overlayfs

In diskless mode, the entire system is loaded into memory at boot. Some usecases may however require to modify such loaded files, or map-in additional disk-based files (i.e: modify/complement modloop-provided firmware files or drivers, add additional packages which may not fit in RAM, etc).

This may be accomplished by overlaying disk-backed loopback filesystem into RAM-based diskless filesystem.

Note: such construct is a diversion from standard diskless mode and may compromise some of this mode core-benefits (potentially slower since disk access is involved, disk/SD wear if such loopback storage is made rw, etc...): user might want to reconsider (better) relevance of other supported install modes (system disk, data disk).

The example below illustrates a usecase on Pi (add extra packages that may not fit in RAM, with a loop-back disk-image), but can be derived for different purposes, on different platforms.

First, make the SD card writable again and change fstab to always do so:

mount /media/mmcblk0p1 -o rw,remount sed -i 's/vfat\ ro,/vfat\ rw,/' /etc/fstab

Create the loop-back file, this example is 1 GB:

dd if=/dev/zero of=/media/mmcblk0p1/persist.img bs=1024 count=0 seek=1048576

Install the ext4 utilities:

apk add e2fsprogs

Format the loop-back file:

mkfs.ext4 /media/mmcblk0p1/persist.img

Mount the storage:

echo "/media/mmcblk0p1/persist.img /media/persist ext4 rw,relatime,errors=remount-ro 0 0" >> /etc/fstab mkdir /media/persist mount -a

Note: Overlay workdir needs to be an empty directory on the same filesystem mount as the upper directory. So each overlay must use its own workdir.

Make the overlay folders, we are using the /usr directory here, but you can use /home or anything else.

mkdir /media/persist/usr mkdir /media/persist/.work_usr echo "overlay /usr overlay lowerdir=/usr,upperdir=/media/persist/usr,workdir=/media/persist/.work_usr 0 0" >> /etc/fstab mount -a

Your /etc/fstab file should look something like this:

Contents of /etc/fstab

/dev/cdrom      /media/cdrom    iso9660 noauto,ro 0 0
/dev/usbdisk    /media/usb      vfat    noauto,ro 0 0
/dev/mmcblk0p1 /media/mmcblk0p1 vfat rw,relatime,fmask=0022,dmask=0022,errors=remount-ro 0 0
/media/mmcblk0p1/persist.img /media/persist ext4 rw,relatime,errors=remount-ro 0 0
overlay /usr overlay lowerdir=/usr,upperdir=/media/persist/usr,workdir=/media/persist/.work_usr 0 0

Now commit the changes: (optionally remove the e2fsprogs, but it does contain repair tools)

lbu_commit -d

Remember, with this setup if you install things and you have done this overlay for /usr, you must not commit the 'apk add', otherwise, while it boots it will try and install it to memory, not to the persistent storage.

If you do want to install something small at boot, you can use apk add and lbu commit -d.

If it is something a bit bigger, then you can use apk add but then not commit it. It will be persistent (in /usr), but be sure to check everything you need is in that directory and not in folders you have not made persistent.

Immutable root with tmpfs overlay

Alpine initramfs features an option to run system disk install with tmpfs overlay on root filesystem: root filesystem is then mounted read-only (original system install is run unmodified across reboots), with all runtime files to be updated in RAM only (hence discarded at shutdown).

This option can provide an interesting alternative scenario to diskless install on systems with very limited RAM (full system is run from disk rather than copied into RAM), and prevents rw I/O stress & wear on media (dynamic files are stored in RAM).

Note: this option may come with some runtime limitations. For instance dockerd will not run without specific configuration under this option (incompatibility with docker's default overlay2 storage driver: vfs storage driver may be an option but with potential performance impacts).

root filesystem tmpfs overlay mode can be enabled by adding the overlaytmpfs=yes option to the Kernel command-line parameters, on system disk install configuration, on any platforms. Optional flags (like size= to limit allocated RAM) may also be added as comma-separated list in overlaytmpfsflags= parameter (defaults are: mode=0755,rw). More info here).

As an exemple below on Pi device, adding the parameter to /boot/cmdline.txt file saves constant writing to the sd-card. The /boot/cmdline.txt file appears as follows

Contents of /boot/cmdline.txt

root=UUID=xxxxxx modules=sd-mod,usb-storage,ext4 quiet rootfstype=ext4 overlaytmpfs=yes

The below output shows how the filesystem is mounted when OverlayFS for root is enabled:

# df -m

Filesystem           1M-blocks      Used Available Use% Mounted on
...
/dev/mmcblk0p2           59555       326     56172   1% /media/root-ro
root-tmpfs                 225        13       211   6% /media/root-rw
overlayfs                  225        13       211   6% /
...

# mount
...
/dev/mmcblk0p2 on /media/root-ro type ext4 (ro,relatime)
root-tmpfs on /media/root-rw type tmpfs (rw,relatime,mode=755)
overlayfs on / type overlay (rw,relatime,lowerdir=/media/root-ro,upperdir=/media/root-rw/root,workdir=/media/root-rw/work,uuid=on)
...

Removing and adding back the overlaytmpfs=yes option in the /boot/cmdline.txt file using a toggle script will allow easy maintenance on the pi and any software/configuration can be easily added/removed.

Another Immutable root with atomic upgrades advanced configuration is also possible.

Loopback image with overlayfs

Immutable root with tmpfs overlay

See also