Alpine local backup
When Alpine Linux boots in diskless mode, it initially only loads a few required packages from the boot device.
However, local adjustments in RAM are possible, e.g. by installing a package or adjusting the configuration files in /etc. And the modifications can also be saved to an overlay file (.apkovl) that can be automatically loaded when booting, to restore the saved state.
The .apkovl file that contains the custom configuration can be saved to writable storage with Alpine's local backup utility lbu. And by configuring a local package cache the additional packages, that the configuration depends on, can also be kept available on local, writable storage.
lbu to save state, because with a system disk all modifications are directly written to disk. Nevertheless, lbu can still be useful for some maintenance tasks.
By default, an lbu commit only stores modifications below /etc, with the exception of the /etc/init.d/ directory.
However, the lbu include allows to modify this set of included files, and can add additional files or folders.
Saving and loading ISO image customizations
Besides supporting a boot parameter to load customizations from a web server, Alpine's "diskless mode" ISO images also try to load an .apkovl volume from system partitions. It is therefore possible to save customized running states to a writable partition, and have these automatically loaded when booting the ISO image. A local package cache, that allows having additional and updated packages installed during boot, may also be stored on a partition.
To use partitions on internal disks, it's currently still needed to prepare a specific /etc/fstab entry manually as follows (read-only is possible, because the tools can temporariyl remount it writable while necessary):
1. Boot the diskless system from ISO
2. If necessary, create and format a partition as in Setting_up_disks_manually#For_.22diskless.22_and_.22data.22_disk_mode_installs_2. For the examples we will use /dev/sdXY
- Bug: Initramfs-init does not respect apkovl device fstab entry, so the partition can not be mounted to /boot, for example. The workaround given there does not seems to work either (
mkdir -p /boot/ovl_dev/cache ; touch /boot/ovl_dev/cache/.boot_repository)
=> Alternative: Configure the /etc/fstab to mount the writable partition to /media/sdXY instead /boot (i.e. conforming to the hot/cold-plug mountpoints):
3. mkdir /media/sdXY
4. echo "/dev/sdXY /media/sdXY ext4 noatime,ro 0 0" >> /etc/fstab
5. mount -a
6. Now, if setup-alpine has not run before, it may be used and should now allow to choose that partition for saving the local configs and package cache. (If asked, there is no need to first unmount the partition, that would only be needed to allow installing on its whole parent disk.)
If setup-alpine has already been used to configure the diskless system, just the storage settings may be modified directly with setup-lbu sdXY and mkdir /media/sdXY/cache ; setup-apkcache /media/sdXY/cache
7. If the partition is large enough, it can be useful to edit lbu.conf to uncomment and set BACKUP_LIMIT=3, for example, to allow reverting to a previous, working state if needed.
# apk add nano # nano /etc/lbu/lbu.conf
8. Finally, generate the first .apkovl file containing all the previous changes with Alpine_local_backup, by executing lbu commit, so the customizations that were just made will persist a reboot.
Then whenever new packages were installed or newly configured, and the changes should be kept, execute lbu commit again.
Notes:
- For a virtual machine, the Qemu example shows how an ISO image can automatically boot with .apkovl customizations.
- Alpine_local_backup explains how to include custom files outside of
/etcin the .apkovl file. - Alpine_Linux_package_management#Local_Cache covers managing a local package cache.
- All packages listed in /etc/apk/world of the lbu will be automatically installed when the live system boots.
- It's possible to load an APKOVL file from a webserver, by supplying a custom url with the
APKOVLkernel boot parameter. - If you don't have a web server you can run busybox's httpd temporarily to serve an .apkovl -
busybox httpd -p 127.0.0.1:80. - To customize the boot sequence further, one may refer to PXE_boot#Specifying_an_apkovl and How_to_make_a_custom_ISO_image_with_mkimage.
Committing changes
The lbu tool knows different subcommands to manage the .apkovl:
- lbu
- lbu commit (Same as 'lbu ci')
- lbu package (Same as 'lbu pkg')
- lbu status (Same as 'lbu st')
- lbu list (Same as 'lbu ls')
- lbu diff
- lbu include (Same as 'lbu inc' or 'lbu add')
- lbu exclude (Same as 'lbu ex' or 'lbu delete')
- lbu list-backup (Same as 'lbu lb')
- lbu revert
In the below examples you will find some characters with special meaning:
- | = or ('lbu commit|ci' means that you can type ether 'lbu commit' or 'lbu ci')
- [ ] = optional (In 'lbu commit|ci [-nv]' you can just skip the '-n', '-v' or '-nv' part if you don't want it)
Basic "commit" command
When you "commit" or save changes you've made to your system, lbu will generate a file named something like myboxname.apkovl.tar.gz ('myboxname' will be the same as the hostname).
This file (that contains your modifications) is called your "apkovl".
You will need to save your apkovl on some suitable media (floppy, usb, cf, other).
usage: lbu commit|ci [-nv] [<media>] Options: -d Remove old apk overlay files. -e Protect configuration with a password. -n Don't commit, just show what would have been committed. -p <password> Give encryption password on the command-line -v Verbose mode. The following values for <media> is supported: floppy usb If <media> is not specified, the environment variable LBU_MEDIA will be used. Password protection will use aes-256-cbc encryption. Other ciphers can be used by setting the DEFAULT_CIPHER or ENCRYPTION environment variables. For possible ciphers, try: openssl -v The password used to encrypt the file, can either be specified with the -p option or using the PASSWORD environment variable. The environment variables can also be set in /etc/lbu/lbu.conf
Creating an .apkovl "package" elsewhere than on the configured media
To "commit" changes, but overriding the destination of the generated apkovl file, use lbu package instead of lbu commit:
usage: lbu package|pkg -v [<dirname>|<filename>] Options: -v Verbose mode. If <dirname> is a directory, a package named <hostname>.apkovl.tar.gz will be created in the specified directory. If <filename> is specified, and is not a directory, a package with the specified name will be created. If neither <dirname> nor <filename> is specified, a package named <hostname>.apkovl.tar.gz will be created in current work directory.
Creating and saving an apkovl from a remote host
To create an apkovl from a client on a centralized server, it's possible to use lbu package with - as the apkovl name over SSH:
On the server:
ssh root@client "lbu package -" >client.apkovl.tar.gz
Checking what will be added to your apkovl
lbu status lists what will be saved the next time you run lbu commit. Its default output is incremental, that is, to only show what files have changed since the last commit; but this can be overridden with the -a flag:
usage: lbu status|st [-av] Options: -a Compare all files, not just since last commit. -v Also show include and exclude lists.
Another option is lbu list. This works like lbu status -a but the format of the output is a bit different. (It's strictly equivalent to lbu package -v /dev/null).
usage: lbu list|ls
A third option is lbu diff. This shows the same incremental changes that lbu status (without -a) does, but in a different format.
usage: lbu diff
Include special files/folders to the apkovl
Assume that you have some files that you want to permanently save, but they are located somewhere else than in /etc.
It could be /root/.ssh/authorized_keys (used by sshd to authenticate ssh-users). Such files/folders can be added to lbu's include list with the following command:
usage: lbu include|inc|add [-rv] <file> ...
lbu include|inc|add [-v] -l
Options:
-l List contents of include list.
-r Remove specified file(s) from include list instead of adding.
-v Verbose mode.
lbu include only modifies lbu's configuration; you will need to run lbu commit to actually create/modify your apkovl.
Exclude specific files/folders from the apkovl
Assume that you have some files located in /etc or one of its subfolders that you do not want to permanently save.
It could be some log file or status file that for some reason isn't in /var/log/ but in some location that would otherwise be tracked by lbu.
Such files/folders can be added to lbu's exclude list by manually editing that file or using the following command:
usage: lbu exclude|ex|delete [-rv] <file> ...
lbu exclude|ex|delete [-v] -l
Options:
-l List contents of exclude list.
-r Remove specified file(s) from exclude list instead of adding.
-v Verbose mode.
lbu exclude also only modifies lbu's configuration; you will need to run lbu commit to actually create/modify your apkovl.
Execute a script as part of a backup
Sometimes it is necessary to run a script before or after a backup. Scripts in two optional directories allow for this:
/etc/lbu/pre-package.d /etc/lbu/post-package.d
Files in those directories are run using run-script rules (meaning they must have the executable bit set, they are run in alphabetical order, and cannot contain an extension: runme works, but runme.sh does not.)
The scripts in pre-package.d are run before the apkovl is created; scripts in post-package.d are run after the apkovl is created.
Example
Rather than adding the raw database directories to /etc/lbu/include, you can do a "database dump". For purposes of example, we use postgresql:
- Create /etc/lbu/pre-package.d/sqldump with the following contents:
pg_dumpall -U postgres | gzip -c >/root/pgdatabases.gz
- Mark the file executable:
chmod +x /etc/lbu/pre-package.d/sqldump - Create /etc/lbu/post-package.d/sqldumpdelete with the following contents:
rm -f /root/pgdatabases.gz
- Mark the file executable:
chmod +x /etc/lbu/post-package.d/sqldumpdelete - Finally, add the database dump file to the list of files to back up:
lbu include root/pgdatabases.gz
Now whenever you do a lbu commit, the sql databases are dumped and gzipped to /root/pgdatabases.gz, and then the temporary file is deleted at the end of the lbu commit.
On a catastrophic restore, the databases are not automatically restored (that's not lbu's responsibility), but you will find a complete database dump in the /root directory, where it can be restored manually.
Multiple backup versions
Lbu can now keep multiple backups so you can revert back to older configs that are known to work. Set BACKUP_LIMIT in /etc/lbu/lbu.conf to the number of backups you want to keep.
If you've set BACKUP_LIMIT, then the previously active .apkovl will be renamed before creating the new file.
You can list the currently available backups with:
lbu list-backup [<media>]
and you can revert to an older one with:
lbu revert <filename> [<media>]
NOTE: Nothing is changed on your running system when "reverting", it only affects which apkovl is considered active for the next boot.