Diskless Mode: Difference between revisions

From Alpine Linux
(rephrased sentence and reorganised the content to make understanding easier.)
(Rephrased the sentence, added wiki tags, reduced repetitions)
Line 3: Line 3:
The mode is extremely fast and can save on unnecessary disk spin-ups, power, and wear. It is similar to what other linux distributions may call a "frugal" install or boot into with a "toram" option.  
The mode is extremely fast and can save on unnecessary disk spin-ups, power, and wear. It is similar to what other linux distributions may call a "frugal" install or boot into with a "toram" option.  


Custom configurations may be preserved or "persist" across reboots by using the Alpine local backup tool <code>[[Alpine_local_backup|lbu]]</code>. It enables committing and reverting local configuration system state by using '''.apkovl''' files that are saved to a "local backup" location and loaded when booting.  
Custom configurations may be preserved or "persist" across reboots by using the Alpine local backup tool <code>[[Alpine_local_backup|lbu]]</code>. It enables committing and reverting local configuration system state by using '''.apkovl''' files that are saved to a "local backup" location i.e a writable partition and loaded when booting.  


* If a writable partition is available, <code>setup-alpine</code> can be told to store the configs on that writable partition using <code>[[Alpine_local_backup|lbu]]</code> and the initial (and possibly read-only) installation media can remain the only boot device for the  "diskless" system.
* The initial and possibly read-only installation media can remain the only boot device for the  "diskless" system.  


* If additional or updated packages have been added to the system, these may also be made available for automatic (re)installation during the boot phase without any (re)downloading, by enabling a [[Alpine_Package_Keeper#Local_Cache|local package cache]] on the writable storage.
* Using the <code>[[Alpine_setup_scripts#setup-bootable|setup-bootable]]</code> script, it is also possible to [[Create_a_Bootable_Device|create a customizable boot device]] i.e a writable filesystem by copying the boot system from the read-only installation media to a writable partition (e.g. /dev/sdXY).
 
* Using the <code>[[Alpine_setup_scripts#setup-bootable|setup-bootable]]</code> script, it is also possible to [[Create_a_Bootable_Device|create a customizable boot device]] i.e a writable filesystem by copying the boot system i.e read-only installation media to a writable partition (e.g. /dev/sdXY).


== Installation  ==
== Installation  ==


Following the [[Installation#Installation_Step_Details|Installation steps]] to complete the [[Installation#Base_configuration|base configuration]] completes the pre-setup of [[#Diskless_Mode|"diskless"]] Alpine Linux system.
<code>Setup-alpine</code> script can store the configs on a writable partition using <code>[[Alpine_local_backup|lbu]]</code>.
 
If a writable partition is available, <code>setup-alpine</code> can be told to store the configs and the package cache on that writable partition. Use the <code>[[Alpine_setup_scripts#setup-lbu|setup-lbu]]</code> script to configure a "local backup" location for the [[#Diskless Mode|diskless]] system, and <code>[[Alpine_local_backup|lbu commit]]</code> to then save the local configuration state as an .apkovl file.  
If a writable partition is available, <code>setup-alpine</code> can be told to store the configs and the package cache on that writable partition. Use the <code>[[Alpine_setup_scripts#setup-lbu|setup-lbu]]</code> script to configure a "local backup" location for the [[#Diskless Mode|diskless]] system, and <code>[[Alpine_local_backup|lbu commit]]</code> to then save the local configuration state. Later, another directory on that same partition or another available partition may also be mounted as /home, or for example, for selected important applications to keep their run-time and user data on it. Refer [[#Saving_and_loading_ISO_image_customizations|Saving and loading ISO image customizations]] related to loading an .apkovl volume from system partitions.
 
As per Bug: [https://gitlab.alpinelinux.org/alpine/alpine-conf/-/issues/10473 #10473] Storing local configs and the package cache on '''internal disks requires''' [[Alpine_local_backup#Saving_and_loading_ISO_image_customizations|manual steps]] to have the partition listed, i.e. making a /etc/fstab entry, mountpoint, and mount, *before* running setup-alpine. The linked workaround also still requires to commit these configurations to disk manually before rebooting.
 
As per Bug: {{Issue|11589|The APKOVL loading of diskless setups doesn't work on btrfs and xfs filesystems, or nvme-based devices}} => So, for the moment, use only ext4 filesystem partitions on classic drives to store diskless mode states.


{{Note| mkfs.ext4 creates ext4 fs with 64bit feature enabled by default, but extlinux may not be able to boot with that, see Issue {{Issue|14895}}. You may need to add "-O ^has_journal,^64bit" to mkfs.ext4 to circumvent this.}}
Later, another directory on that same partition or another available partition may also be mounted as /home, or for example, for selected important applications to keep their run-time and user data on it. If additional or updated packages have been added to the system, these may also be made available for automatic (re)installation during the boot phase without any (re)downloading, by enabling a [[Alpine_Package_Keeper#Local_Cache|local package cache]] on the writable storage.


* The following creates an ext4 partition with disabled journaling, to reduce write operations and allow the disk to spin down after the .apkovl and the packages have been read from the partition during the boot.
<!--
 
Refer [[#Saving_and_loading_ISO_image_customizations|Saving and loading ISO image customizations]] related to loading an .apkovl volume from system partitions.
# mkfs.ext4 -O ^has_journal /dev/sdXY  # the "-O ^has_journal" DISABLES journaling ("^" means "not")
-->
As per Bug: [https://gitlab.alpinelinux.org/alpine/alpine-conf/-/issues/10473 #10473] Storing local configs and the package cache on '''internal disks requires''' [[Alpine_local_backup#Saving_and_loading_ISO_image_customizations|manual steps]]  '''before''' running <code>[[Alpine_setup_scripts#setup-alpine|setup-alpine]]</code> script. 
<!--
The linked workaround also still requires to commit these configurations to disk manually before rebooting.


In the above case, the initial read-only installation media may remain the boot device of the newly configured local "diskless" system.  It is also possible to copy the boot system to a partition (e.g. /dev/sdXY) with <code>[[Alpine_setup_scripts#setup-bootable|setup-bootable]]</code> script and [[Create_a_Bootable_Device|create a  customizable boot device]].
In the above case, the initial read-only installation media may remain the boot device of the newly configured local "diskless" system.  It is also possible to copy the boot system to a partition (e.g. /dev/sdXY) with <code>[[Alpine_setup_scripts#setup-bootable|setup-bootable]]</code> script and [[Create_a_Bootable_Device|create a  customizable boot device]].
 
-->
== Saving and loading ISO image customizations ==
== Saving and loading ISO image customizations ==
 
<!--
Besides supporting a boot parameter to load customizations from a web server, Alpine's "diskless mode" ISO images try to load an .apkovl volume from system partitions. It is therefore possible to save customized running states to a .apkovl file on a writable partition, and have these automatically loaded when booting the ISO image.
Besides supporting a boot parameter to load customizations from a web server, Alpine's "diskless mode" ISO images try to load an .apkovl volume from system partitions. It is therefore possible to save customized running states to a .apkovl file on a writable partition, and have these automatically loaded when booting the ISO image.


Line 36: Line 32:


The local package cache, which enables additional and updated packages to automatically be copied into RAM during boot, may be stored on the same partition as the .apkovl file.
The local package cache, which enables additional and updated packages to automatically be copied into RAM during boot, may be stored on the same partition as the .apkovl file.
-->


To use partitions on '''internal disks''', however, you need to prepare a specific /etc/fstab entry manually, and save the configuration, as follows (setup-alpine does not list them for saving configs or package cache):
To use partitions on '''internal disks''', however, you need to prepare a specific /etc/fstab entry manually, and save the configuration, as follows:
* Boot the diskless system from ISO and follow the [[Installation#Installation_Step_Details|Installation steps]] to complete the [[Installation#Base_configuration|base configuration]]. Now the pre-setup of [[#Diskless_Mode|"diskless"]] Alpine Linux system is considered complete.<br> 
* If necessary partition(s) are unavailable, manually [[#Creating_partitions|create]] a partition. For the examples we will use /dev/sdXY<br>
<!--
[[Setting_up_disks_manually#For_.22diskless.22_and_.22data.22_disk_mode_installs_2|"Setting up disks manually, for diskless and data disk mode installs"]].
-->
** Be aware of this Bug: {{Issue|11589|The APKOVL loading of diskless setups doesn't work on btrfs and xfs filesystems, or nvme-based devices}} => So, for the moment, use only ext4 filesystem partitions on classic drives to store diskless mode states.
**  mkfs.ext4 creates ext4 fs with 64bit feature enabled by default, but extlinux may not be able to boot with that due to Issue {{Issue|14895}}. You may need to add "-O ^64bit" to mkfs.ext4 to circumvent this. ("^" means "not").
{{Cmd|mkfs.ext4 -O ^has_journal,^64bit /dev/sdXY}}
* The above command creates an ext4 partition with disabled journaling, to reduce write operations and allow the disk to spin down after the .apkovl and the packages have been read from the partition during the boot.
<!--
** Bug: [https://gitlab.alpinelinux.org/alpine/mkinitfs/-/issues/5 Initramfs-init does not respect apkovl device fstab entry], so the partition can not be mounted to /boot. The workaround given there does not seem to work
(e.g. <code>mkdir -p /boot/ovl_dev/cache ; touch /boot/ovl_dev/cache/.boot_repository</code>)<br>
-->
* Due to a [https://gitlab.alpinelinux.org/alpine/mkinitfs/-/issues/5 bug], the partition can not be mounted to /boot. Configure the /etc/fstab to mount the writable partition to /media/sdXY instead of /boot i.e. conforming to the hot/cold-plug mountpoints. <br>
{{Cmd|mkdir /media/sdXY
echo "/dev/sdXY /media/sdXY ext4 noatime,ro 0 0" >> /etc/fstab}}
* Mounting read-only is possible, because the [[Alpine_local_backup|lbu tools]] can temporarily remount it writable for their operation. Use {{Codeline|<code>mount -a</code>}} to load the partitions listed in {{Path|/etc/fstab}}. Look at the output to verify that the changes have been applied correctly.<br>
* If setup-alpine has not run before, it may be used and should now enable you to choose the partition for saving the local configs and package cache. (If asked, there is no need to first unmount the partition, that would only be needed to allow installing on its parent disk.)<br>
* If setup-alpine has already been used to configure the diskless system, the storage settings may be modified directly with
{{cmd|setup-lbu sdXY}}
* [[Alpine_Package_Keeper#Local_Cache|Local package cache]] can be enabled as follows:
{{cmd|mkdir /media/sdXY/cache
setup-apkcache /media/sdXY/cache}}
* If the partition is large enough, it can be useful to edit {{Path|lbu.conf}} to uncomment and set {{Codeline|BACKUP_LIMIT{{=}}3}}. For example, to allow reverting to a previous, working state if needed.
{{cmd|apk add nano
nano /etc/lbu/lbu.conf }}
* Finally, generate the first .apkovl file containing all the previous changes by executing <code>lbu commit</code>, so the customizations that were just made will persist a reboot.


1. Boot the diskless system from ISO (as shown at [[Installation]])<br>
From then on, whenever packages are installed or newly configured, and the changes should be kept, execute <code>lbu commit</code> again.
2. If necessary, create and format a partition as explained in [[Setting_up_disks_manually#For_.22diskless.22_and_.22data.22_disk_mode_installs_2|"Setting up disks manually, for diskless and data disk mode installs"]]. For the examples we will use /dev/sdXY<br>
* Bug: [https://gitlab.alpinelinux.org/alpine/mkinitfs/-/issues/5 Initramfs-init does not respect apkovl device fstab entry], so the partition can not be mounted to /boot. The workaround given there does not seem to work (e.g. <code>mkdir -p /boot/ovl_dev/cache ; touch /boot/ovl_dev/cache/.boot_repository</code>)<br>
=> Alternative: Configure the /etc/fstab to mount the writable partition to /media/sdXY instead of /boot (i.e. conforming to the hot/cold-plug mountpoints):<br>
3. <code>mkdir /media/sdXY</code><br>
4. <code>echo "/dev/sdXY /media/sdXY ext4 noatime,ro 0 0" >> /etc/fstab</code><br> (Mounting read-only is possible, because the tools temporarily remount it writable for their operation.)<br>
5. Use <code>mount -a</code> to load the partitions listed in /etc/fstab. Look at the output of <code>mount</code> to verify that the /etc/fstab changes have been applied correctly.<br>
6. If setup-alpine has not run before, it may be used and should now enable you to choose the partition for saving the local configs and package cache. (If asked, there is no need to first unmount the partition, that would only be needed to allow installing on its parent disk.)<br>
If setup-alpine has already been used to configure the diskless system, the storage settings may be modified directly with <code>setup-lbu sdXY</code> and  <code>mkdir /media/sdXY/cache ; setup-apkcache /media/sdXY/cache</code><br>
7. If the partition is large enough, it can be useful to edit lbu.conf to uncomment and set <code>BACKUP_LIMIT=3</code>. For example, to allow reverting to a previous, working state if needed.
{{cmd| # apk add nano
# nano /etc/lbu/lbu.conf
}}
8. Finally, generate the first .apkovl file containing all the previous changes by executing <code>lbu commit</code>, so the customizations that were just made will persist a reboot.


From then on, whenever packages are installed or newly configured, and the changes should be kept, execute <code>lbu commit</code> again.
== Customizable boot device ==
{{Main|Create a Bootable Device}}
 
In the [[#Saving and loading ISO image customizations|above case]], the initial read-only installation media may remain the only boot device of the newly configured local "diskless" system.  It is also possible to copy the boot system to a partition (e.g. /dev/sdXY) with <code>[[Alpine_setup_scripts#setup-bootable|setup-bootable]]</code> script and [[Create_a_Bootable_Device|create a  customizable boot device]].


Notes:
== See Also ==
* For a virtual machine, the '''[[QEMU#Live_mode|QEMU example]]''' shows how an ISO image can automatically boot with .apkovl customizations.
* For a virtual machine, the '''[[QEMU#Live_mode|QEMU Diskless example]]''' shows how an ISO image can automatically boot with .apkovl customizations.
* The [[Alpine local backup#Include special files.2Ffolders to the apkovl|Include special files section]] explains how to include custom files outside of <code>/etc</code> in the .apkovl file.
* The [[Alpine local backup#Include special files.2Ffolders to the apkovl|Include special files section]] explains how to include custom files outside of <code>/etc</code> in the .apkovl file.
* [[Alpine Linux package management#Local Cache]] covers managing a local package cache.
* [[Alpine Linux package management#Local Cache]] covers managing a local package cache so that the packages listed in {{path|/etc/apk/world}} which gets saved with the lbu, gets automatically installed when the live system boots.
** The packages get listed in {{path|/etc/apk/world}} which gets saved with the lbu, for them to be automatically installed when the live system boots.
* It's possible to load an APKOVL file from a webserver, by supplying a custom url with the <code>APKOVL</code> kernel boot parameter.
** If you don't have a web server you can run busybox's httpd temporarily to serve an .apkovl - <code>busybox httpd -p 127.0.0.1:80</code>.
* To customize the boot sequence further, one may refer to [[PXE boot#Specifying an apkovl]] and [[How to make a custom ISO image with mkimage]].
* To customize the boot sequence further, one may refer to [[PXE boot#Specifying an apkovl]] and [[How to make a custom ISO image with mkimage]].
 
* It's possible to load an APKOVL file from a webserver, by supplying a custom url with the <code>APKOVL</code> kernel boot parameter. If you don't have a web server you can run busybox's httpd temporarily to serve an .apkovl - <code>busybox httpd -p 127.0.0.1:80</code>.


[[Category:Diskless]]
[[Category:Diskless]]

Revision as of 11:06, 1 November 2024

In Diskless mode the entire operating system with all applications are first loaded into RAM and then only run from there. This is the method already used to boot the Alpine Linux iso installation images. Alpine Linux can be installed and configured so that the system continue to boot like this if "disk=none" is specified while running the setup-alpine script.

The mode is extremely fast and can save on unnecessary disk spin-ups, power, and wear. It is similar to what other linux distributions may call a "frugal" install or boot into with a "toram" option.

Custom configurations may be preserved or "persist" across reboots by using the Alpine local backup tool lbu. It enables committing and reverting local configuration system state by using .apkovl files that are saved to a "local backup" location i.e a writable partition and loaded when booting.

  • The initial and possibly read-only installation media can remain the only boot device for the "diskless" system.

Installation

Setup-alpine script can store the configs on a writable partition using lbu. If a writable partition is available, setup-alpine can be told to store the configs and the package cache on that writable partition. Use the setup-lbu script to configure a "local backup" location for the diskless system, and lbu commit to then save the local configuration state as an .apkovl file.

Later, another directory on that same partition or another available partition may also be mounted as /home, or for example, for selected important applications to keep their run-time and user data on it. If additional or updated packages have been added to the system, these may also be made available for automatic (re)installation during the boot phase without any (re)downloading, by enabling a local package cache on the writable storage.

As per Bug: #10473 Storing local configs and the package cache on internal disks requires manual steps before running setup-alpine script.

Saving and loading ISO image customizations

To use partitions on internal disks, however, you need to prepare a specific /etc/fstab entry manually, and save the configuration, as follows:

mkfs.ext4 -O ^has_journal,^64bit /dev/sdXY

  • The above command creates an ext4 partition with disabled journaling, to reduce write operations and allow the disk to spin down after the .apkovl and the packages have been read from the partition during the boot.
  • Due to a bug, the partition can not be mounted to /boot. Configure the /etc/fstab to mount the writable partition to /media/sdXY instead of /boot i.e. conforming to the hot/cold-plug mountpoints.

mkdir /media/sdXY echo "/dev/sdXY /media/sdXY ext4 noatime,ro 0 0" >> /etc/fstab

  • Mounting read-only is possible, because the lbu tools can temporarily remount it writable for their operation. Use mount -a to load the partitions listed in /etc/fstab. Look at the output to verify that the changes have been applied correctly.
  • If setup-alpine has not run before, it may be used and should now enable you to choose the partition for saving the local configs and package cache. (If asked, there is no need to first unmount the partition, that would only be needed to allow installing on its parent disk.)
  • If setup-alpine has already been used to configure the diskless system, the storage settings may be modified directly with

setup-lbu sdXY

mkdir /media/sdXY/cache setup-apkcache /media/sdXY/cache

  • If the partition is large enough, it can be useful to edit lbu.conf to uncomment and set BACKUP_LIMIT=3. For example, to allow reverting to a previous, working state if needed.

apk add nano nano /etc/lbu/lbu.conf

  • Finally, generate the first .apkovl file containing all the previous changes by executing lbu commit, so the customizations that were just made will persist a reboot.

From then on, whenever packages are installed or newly configured, and the changes should be kept, execute lbu commit again.

Customizable boot device

In the above case, the initial read-only installation media may remain the only boot device of the newly configured local "diskless" system. It is also possible to copy the boot system to a partition (e.g. /dev/sdXY) with setup-bootable script and create a customizable boot device.

See Also

  • For a virtual machine, the QEMU Diskless example shows how an ISO image can automatically boot with .apkovl customizations.
  • The Include special files section explains how to include custom files outside of /etc in the .apkovl file.
  • Alpine Linux package management#Local Cache covers managing a local package cache so that the packages listed in /etc/apk/world which gets saved with the lbu, gets automatically installed when the live system boots.
  • To customize the boot sequence further, one may refer to PXE boot#Specifying an apkovl and How to make a custom ISO image with mkimage.
  • It's possible to load an APKOVL file from a webserver, by supplying a custom url with the APKOVL kernel boot parameter. If you don't have a web server you can run busybox's httpd temporarily to serve an .apkovl - busybox httpd -p 127.0.0.1:80.