Finding the fastest mirror

2022-11-10T18:47:02Z

Leonclx:

After you install Alpine, you may be wondering how do I figure out the fastest mirror again?

'''Don't use this script, it's broken'''
{{Cat|/home/user/fastestmirror|<nowiki>
data=""
for s in $(wget -qO- http://rsync.alpinelinux.org/alpine/MIRRORS.txt); do
t=$(time -f "%E" wget -q $s/MIRRORS.txt -O /dev/null 2>&1)
echo "$s was $t"
data="$data$t $s\n"
done

echo "===RESULTS==="

echo -e $data | sort
</nowiki>}}

This is more advanced implementation.

'''Don't use this script, it's broken'''
{{Cat|/etc/apk/fastest-mirror|<nowiki>
#!/bin/sh
get_hostname_url() {
local n=${1#*://}
echo ${n%%/*}
}

time_cmd() {
local proc=$(cut -d ' ' -f1 /proc/uptime)
local start="$(echo $proc | cut -d . -f1)$(echo $proc | cut -d . -f2)"
$@ >/dev/null 2>&1 || return
proc=$(cut -d ' ' -f1 /proc/uptime)
local end="$(echo $proc | cut -d . -f1)$(echo $proc | cut -d . -f2)"
echo $(( $end - $start ))
}

DATA=""
MIRRORS=$(wget -qO- "http://rsync.alpinelinux.org/alpine/MIRRORS.txt")
DST=/etc/apk/mirrors.txt

#find best
for URL in $MIRRORS; do
TIME=$(time_cmd wget -T 1 -t 1 -q ${URL%/} -O /dev/null)
if [ -n "$TIME" ]; then
echo "$(get_hostname_url $URL) was $TIME"
DATA="$DATA$TIME $URL\n"
fi
done

echo $DATA | sort -n | tail -n +2 > $DST
[ $? = 0 ] && echo file $DST created

BEST=$(head -n1 $DST | cut -d ' ' -f2)
echo "Best mirror is: $BEST"

sed -i -r 's#^http.+/(.+/main)#'${BEST%/}'/\1#' /etc/apk/repositories
sed -i -r 's#^http.+/(.+/community)#'${BEST%/}'/\1#' /etc/apk/repositories
sed -i -r 's#^http.+/(.+/testing)#'${BEST%/}'/\1#' /etc/apk/repositories

</nowiki>}}

[[Category:Package Manager]]

[[Category:Package Manager]]

Setting up disks manually

2022-10-02T08:34:06Z

Leonclx: added instructions for custom boot partition size

The menu choices that are provided by <code>setup-alpine</code> can not cover all needs, however, it's possible to:

* Call [[Alpine_setup_scripts#setup-disk|setup-disk]] directly, instead of through <code>setup-alpine</code>, and pass some specific partitioning parameters to it.
* Set up the partitions and filesystems manually (to the desired details) and [[Alpine_setup_scripts#Partitioning|tell setup-disk]] to install to the prepared target mountpoint.

(Beta refenence: [https://docs.alpinelinux.org/user-handbook/0.1a/Installing/manual.html#_finalizing docs.alpinelinux.org] )

= Using setup-alpine (setup-disk) features =

== For "diskless" and "data" disk mode installs ==

This requires some manual steps, as long as the setup-scripts auto-detect usable partitions only on *removable* devices (not internal harddisks). They do not support creating new, writable partitions to be used by Alpine.

How to prepare writable storage space is explained in the section [[Setting_up_disks_manually#Manual_partitioning| manual partitioning]], below.

== For "sys" disk mode installs ==

{{Draft|Some information on this page may be incomplete or outdate.}}

=== Disk layouts ===

It is possible to have one or more of RAID, encrypted, and/or Logical Volume on your {{Path|/}} (root) volume. However, the Alpine init script only knows how to handle them when they're layered in that order, and your initram and extlinux.conf files in the {{Path|/boot}} partition are configured properly.

Your {{Path|/boot}} directory cannot reside on an encrypted or LVM volume, at least not with Alpine's default bootloader (extlinux). (Grub2 can deal with {{Path|/boot}} being on an LVM volume.) The usual practice is to create a small partition for {{Path|/boot}}, then devote the rest of your disk to a separate partition on which you layer one or more of RAID, encryption, and/or Logical Volumes.

Sometimes {{Path|/boot}} is also set up as a mirrored (RAID1) volume. However, this is just for post-init access. That way, when you write a new kernel or bootloader config file to {{Path|/boot}}, it gets written to multiple physical partitions. During the pre-init, bootloader phase, only one of those partitions will be read from.

A typical setup might look like this:

<pre>
One-disk system
---------------
+------------------------------------------------+
| small partition (32--100M), holding |
| only /boot, filesystem needn't be journaled |
+------------------------------------------------+
| rest of disk in second partition |
| +------------------------------------------+ |
| | cryptsetup volume | |
| | +-------------------------------------+ | |
| | | LVM PV, containing single VG, | | |
| | | containing multiple LVs, holding | | |
| | | /, /home, swap, etc | | |
| | +-------------------------------------+ | |
| +------------------------------------------+ |
+------------------------------------------------+

Two-disk system
---------------
+------------------------------------------------+ +------------------------------------------------+
| small partition (32--100M), holding | | small partition (32--100M), holding | These 2 partitions might
| only /boot, filesystem needn't be journaled | | only /boot, filesystem needn't be journaled | form a mirrored (RAID1)
+------------------------------------------------+ +------------------------------------------------+ volume
| rest of disk in second partition | | rest of disk in second partition |
| T================================================================================================T | These 2 partitions form
| T +--------------------------------------------------------------------------------------------+ T | a second mirrored
| T | cryptsetup volume | T | (RAID1) volume
| T | +---------------------------------------------------------------------------------------+ | T |
| T | | LVM PV, containing single VG, | | T |
| T | | containing multiple LVs, holding | | T |
| T | | /, /home, swap, etc | | T |
| T | +---------------------------------------------------------------------------------------+ | T |
| T +--------------------------------------------------------------------------------------------+ T |
| T================================================================================================T |
| | | |
+------------------------------------------------+ +------------------------------------------------+

</pre>

In a three-disk system, the {{Path|/boot}} would still be RAID1, but the larger partition might, in that case, be RAID5.

=== RAID ===

<code>setup-disk</code> will automatically build a RAID array if you supply the '''-r''' switch, or if you specify more than one device.

If you want to build your RAID array manually, see [[Setting up a software RAID array]]. Then you can add additional layers of encryption and/or Logical Volumes, or just assemble the RAID array, and supply the {{Path|/dev/mdi}} device directly to [[setup-disk]]. When you're finished, be sure to disassemble the RAID array before rebooting.

If <code>setup-disk</code> sees that you're using RAID, either because you gave it the <code>-r</code> switch, or multiple devices, or a {{Path|/dev/mdi}} device, then it will set up your initramfs and extlinux.conf file properly. However, in other cases, such as when you're also using encryption, or you invoke <code>setup-disk</code> with a mounted directory argument, these might not be properly set up for RAID. In that case, you may need to manually edit/rebuild them. The following assumes that <code>$MNT</code> holds the root directory you're installing into:

{{Cmd|1=echo "/sbin/mdadm" > $MNT/etc/mkinitfs/files.d/raid
echo "/etc/mdadm.conf" >> $MNT/etc/mkinitfs/files.d/raid
# edit $MNT/etc/mkinitfs/mkinitfs.conf to make sure features="..."
# includes raid (this field is space-separated and quoted)
mkinitfs -c $MNT/etc/mkinitfs/mkinitfs.conf -b $MNT
# edit $MNT/etc/update-extlinux.conf to make sure modules=... contains
# raid1 or raid456 (whichever your / is on; this field is comma-separated)
# also check the root= setting
extlinux --raid --install $MNT/boot --update
}}

{{Todo|Does adding the <code>--update</code> option to <code>extlinux ...</code> suffice to make {{Path|/boot/extlinux.conf}} be regenerated? Or do we need to manually tweak that file, or run <code>update-extlinux</code>, as well?}}

You might also need to manually tweak {{Path|$MNT/etc/fstab}}. And you might need to copy {{Path|/usr/share/syslinux/mbr.bin}} to your disk's MBR.

=== Encryption ===

See [[Setting up encrypted volumes with LUKS]]. You can add an additional Logical Volume layer, or just unlock the volume you've created (using <code>cryptsetup luksOpen ...</code>), and supply the {{Path|/dev/mapper/something}} device directly to [[setup-disk]]. When you're finished, be sure to relock the volume (using <code>cryptsetup luksClose ...</code>) before rebooting.

If you install your {{Path|/}} (root) on an encrypted volume, you'll need to manually edit/rebuild your initram and your extlinux.conf file. The following assumes that <code>$MNT</code> holds the root directory you're installing into, that you've created the cryptvolume on the device {{Path|/dev/md2}}, and that you want to unlock the encrypted volume into a virtual volume named "crypt":

{{Cmd|1=# edit $MNT/etc/mkinitfs/mkinitfs.conf to make sure features="..."
# includes cryptsetup (this field is space-separated and quoted)
mkinitfs -c $MNT/etc/mkinitfs/mkinitfs.conf -b $MNT
# edit $MNT/etc/update-extlinux.conf to make sure default_kernel_opts="..."
# contains cryptroot=/dev/md1 and cryptdm=crypt (this field is also space-separated and quoted)
# also check the root= setting
extlinux --install $MNT/boot --update
}}

{{Todo|Does adding the <code>--update</code> option to <code>extlinux ...</code> suffice to make {{Path|/boot/extlinux.conf}} be regenerated? Or do we need to manually tweak that file, or run <code>update-extlinux</code>, as well?}}

You might also need to manually tweak {{Path|$MNT/etc/fstab}}.

=== LVM ===

<code>setup-disk</code> will automatically build and use volumes in a LVM group if you supply the '''-L''' switch.

If you instead want to build your LVM system manually, see [[Setting up Logical Volumes with LVM]]. Then <code>vgchange -ay</code>, format and mount your volumes, and supply the root mountpoint to [[setup-disk]]. When you're finished, be sure to
{{Cmd|umount ...
vgchange -an}}
before rebooting.

If <code>setup-disk</code> sees that you're using LVM, perhaps because you gave it the <code>-L</code> switch, then it will set up your initram and extlinux.conf file properly. However, in other cases, these might not be properly set up. In that case, you may need to manually edit/rebuild them. The following assumes that <code>$MNT</code> holds the root directory you're installing into:

{{Cmd|1=# edit $MNT/etc/mkinitfs/mkinitfs.conf to make sure features="..."
# includes lvm (this field is space-separated and quoted)
mkinitfs -c $MNT/etc/mkinitfs/mkinitfs.conf -b $MNT
# edit $MNT/etc/update-extlinux.conf to make sure root= is set correctly
extlinux --install $MNT/boot --update
}}

{{Todo|Does adding the <code>--update</code> option to <code>extlinux ...</code> suffice to make {{Path|/boot/extlinux.conf}} be regenerated? Or do we need to manually tweak that file, or run <code>update-extlinux</code>, as well?}}

You might also need to manually tweak {{Path|$MNT/etc/fstab}}.

= Manual partitioning =

== For "diskless" and "data" disk mode installs ==

The installation image enviroment may have only the very basic busybox built-in <code>fdisk</code> command available. However, alpine allows you to install more complete partitioning tool packages like <code>sfdisk</code> (scriptable fdisk), <code>gptfdisk</code>, <code>parted</code> (CLI), <code>cfdisk</code> (text menus), or even <code>gparted</code> (after [[Installation#Post-Install|setting up a graphical environment]]).

The following example installs <code>cfdisk</code>, looks for existing partitions, and runs cfdisk against <code>/dev/sdX</code>:
# apk add cfdisk
# blkid
/dev/sdX1: LABEL="some" UUID="..." TYPE="vfat"
/dev/sdX2: LABEL="other" UUID="..." TYPE="ext4"
# cfdisk /dev/sdX

* Be aware of this Bug: [https://gitlab.alpinelinux.org/alpine/aports/-/issues/11589 The APKOVL loading of diskless setups doesn't work on btrfs and xfs filesystems, or nvme-based devices] => So, for the moment, use only ext4 filesystem partitions on classic drives to store diskless mode states.

The following creates an ext4 partition with disabled journaling, to reduce write operations and allow the disk to spin down after the .apkovl and the packages have been read from the partition during the boot.
# apk add e2fsprogs
# mkfs.ext4 -O ^has_journal /dev/sdXY # the "-O ^has_journal" DISABLES journaling ("^" means "not")

{{Note|IMPORTANT! There may be additional steps necessary to make proper use of the partition, see
[[Alpine_local_backup#Saving_and_loading_ISO_image_customizations|Saving_and_loading_ISO_image_customizations]].
}}

== For "sys" disk mode installs ==

<code>setup-disk</code> will by default set up a root partition, a separate /boot partition and a swap partition. If you want a different layout, you can manually create the partitions, filesystems and mount them on {{Path|/mnt}} (or any other mount point) then run:

{{Cmd|setup-disk /mnt}}

<code>setup-disk</code> will install your running system on the mounted root, detect your file system layout and generate an fstab.

If you're using the MBR, you are responsible for making the proper partition bootable and make sure the MBR is OK for extlinux. If you're going to use EFI, make sure you mount <code>esp</code> partition on <code>/boot/efi</code> and set <code>BOOTLOADER=grub</code> and <code>USE_EFI=1</code>, and that {{Pkg|grub-efi}} and {{Pkg|efibootmgr}} are installed before running <code>setup-disk</code>. If your system still does not boot, you can try to manually create a boot entry with {{Pkg|efibootmgr}}, for which instructions can be found at [https://wiki.alpinelinux.org/wiki/Bootloaders#efibootmgr EFI Boot Stub].

See also [https://github.com/itoffshore/alpine-linux-scripts setup-partitions]

=== Dual-booting ===
See [[Installing Alpine on HDD dualbooting|Install to HDD with dual-boot]]

=== Other needs ===
* [[Installing Alpine Linux in a chroot]]
* [[Replacing non-Alpine Linux with Alpine remotely]]





== Setting up swap ==

# create a partition with the type set to "linux swap" (82) (If you're going to use a logical volume for swap, skip this step and <code>lvcreate</code> that instead.)
# <code>mkswap /dev/sda2</code>
# <code>echo -e "/dev/sda2 none swap sw 0 0" >> /mnt/etc/fstab</code>
# <code>swapon /dev/sda2</code> (or <code>rc-service swap start</code>)

{{Cmd|free -m}} will show how much swap space is available (in MB).

If you prefer maximum speed, you don't need configure any raid devices for swap. Just add 2 swap partitions on different disks and linux will stripe them automatically. The downside is that the moment one disk fails, the system will go down. For better reliability, put swap on RAID1.

{{Todo|Instructions for cryptswap?}}

== Custom boot partition size ==

The default boot partition size is under 40MB. This can cause errors if package updates require additional space on /boot.
Executing <code>export BOOT_SIZE=512</code> before running <code>setup-disk</code> will result in a 512MB boot partition.

[[Category:Installation]]
[[Category:Storage]]

Alpine Linux - User contributions [en]

Finding the fastest mirror

Setting up disks manually