https://wiki.alpinelinux.org/w/api.php?action=feedcontributions&user=Geek+at&feedformat=atomAlpine Linux - User contributions [en]2024-03-29T10:27:55ZUser contributionsMediaWiki 1.40.0https://wiki.alpinelinux.org/w/index.php?title=LXC&diff=22630LXC2022-10-19T10:05:37Z<p>Geek at: added info on static IPs in Debian since 11.3 Taken from https://www.claudiokuenzler.com/blog/1206/debian-bullseye-11.3-lxc-container-static-ip-config-not-working</p>
<hr />
<div>[https://linuxcontainers.org/ Linux Containers (LXC)] provides containers similar to BSD Jails, Linux VServers and Solaris Zones. It gives the impression of virtualization, but shares the kernel and resources with the "host". You can use lxc directly or through [[LXD]].<br />
<br />
== Installation ==<br />
Install the required packages:<br />
{{Cmd|apk add lxc bridge lxcfs lxc-download xz}}<br />
<br />
If you want to create containers other than Alpine, you'll need lxc-templates:<br />
<br />
{{Cmd|apk add lxc-templates}}<br />
<br />
== Upgrading from 2.x ==<br />
<br />
Starting with Alpine 3.9, we ship LXC version 3.1.<br />
LXC 3.x has major changes which can and will break your current setup.<br />
LXC 3.x will NOT ship with legacy container templates. Check your current container configs to see if you have any includes pointing to files that don't exist (shipped by legacy templates).<br />
For example if you use Alpine containers created with the Alpine template, you'll need to install:<br />
<br />
apk add lxc-templates-legacy-alpine<br />
<br />
Also make sure you convert your LXC config files to the new 2.x format (this is now required).<br />
<br />
lxc-update-config -c /var/lib/lxc/container-name/config<br />
<br />
Make sure you have removed '''cgroup_enable''' from your cmdline as this will fail to mount cgroups and fail LXC service.<br />
<br />
== Prepare network on host ==<br />
Set up a [[bridge]] on the host. Example ''/etc/network/interfaces'':<br />
<pre><br />
auto br0<br />
iface br0 inet dhcp<br />
bridge-ports eth0<br />
</pre><br />
<br />
Create a network configuration template for the guests, ''/etc/lxc/default.conf'':<br />
<pre><br />
lxc.net.0.type = veth<br />
lxc.net.0.link = br0<br />
lxc.net.0.flags = up<br />
lxc.net.0.hwaddr = fe:xx:xx:xx:xx:xx<br />
</pre><br />
<br />
== Create a guest ==<br />
<br />
=== Picking from the list ===<br />
<br />
{{Cmd|lxc-create -n guest1 -f /etc/lxc/default.conf -t download}}<br />
<br />
And just pick from the list. lxc-download and xz can be uninstalled after you are done.<br />
<br />
=== Alpine Template ===<br />
<br />
{{Cmd|lxc-create -n guest1 -f /etc/lxc/default.conf -t alpine}}<br />
<br />
This will create a ''/var/lib/lxc/guest1'' directory with a ''config'' file and a ''rootfs'' directory.<br />
<br />
Note: by default, the alpine template '''does not have networking service on''', you will need to add it using lxc-console<br />
<br />
<br />
If running on x64 compatible hardware, it is possible to create a 32bit guest:<br />
<br />
{{Cmd|lxc-create -n guest1 -f /etc/lxc/default.conf -t alpine -- --arch x86}}<br />
<br />
=== Debian template ===<br />
<br />
In order to create a debian template container you'll need to install some packages:<br />
<br />
{{Cmd|apk add debootstrap rsync}}<br />
<br />
You'll need to turn off some grsecurity chroot options otherwise the debootstrap will fail:<br />
<br />
{{Cmd|echo 0 > /proc/sys/kernel/grsecurity/chroot_caps<br />
echo 0 > /proc/sys/kernel/grsecurity/chroot_deny_chroot<br />
echo 0 > /proc/sys/kernel/grsecurity/chroot_deny_mount<br />
echo 0 > /proc/sys/kernel/grsecurity/chroot_deny_mknod<br />
echo 0 > /proc/sys/kernel/grsecurity/chroot_deny_chmod<br />
}}<br />
<br />
<br />
Remember to turn them back on, or simply reboot.<br />
<br />
Now you can run:<br />
{{Cmd|SUITE{{=}}wheezy lxc-create -n guest1 -f /etc/lxc/default.conf -t debian}}<br />
<br />
==== Setting a static IP ====<br />
Since Debian Bullseye 11.3 you can't assign a static IP address using the lxc config file of the container [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1009351 because of a systemd change].<br />
To make it work with a configuration like the following<br />
<br />
# grep net /var/lib/lxc/bullseye/config <br />
lxc.net.0.type = veth<br />
lxc.net.0.flags = up<br />
lxc.net.0.link = virbr1<br />
lxc.net.0.ipv4.address = 192.168.1.111/24<br />
lxc.net.0.ipv4.gateway = 192.168.1.1<br />
<br />
<br />
You have to attach to the container and run <br />
<br />
{{Cmd|lxc-attach -n bullseye<br />
systemctl stop systemd-networkd<br />
systemctl disable systemd-networkd<br />
reboot<br />
}}<br />
<br />
<br />
After the reboot the IP address should be set correctly. This can be confirmed using the lxc-ls command<br />
<br />
{{Cmd|# lxc-ls -f<br />
NAME STATE AUTOSTART GROUPS IPV4 IPV6 UNPRIVILEGED <br />
bullseye RUNNING 1 - 192.168.1.111 - false<br />
}}<br />
<br />
=== Ubuntu template ===<br />
<br />
In order to create an ubuntu template container, you'll need to turn off some grsecurity chroot options:<br />
<br />
{{Cmd|echo 0 > /proc/sys/kernel/grsecurity/chroot_caps<br />
echo 0 > /proc/sys/kernel/grsecurity/chroot_deny_chroot<br />
echo 0 > /proc/sys/kernel/grsecurity/chroot_deny_mount<br />
echo 0 > /proc/sys/kernel/grsecurity/chroot_deny_mknod<br />
echo 0 > /proc/sys/kernel/grsecurity/chroot_deny_chmod<br />
}}<br />
<br />
Remember to turn them back on, or simply reboot.<br />
<br />
Now you can run (replace %MIRROR% with the actual hostname, for example: http://us.archive.ubuntu.com/ubuntu/)<br />
<br />
{{Cmd|lxc-create -n guest2 -f /etc/lxc/default.conf -t ubuntu -- -r xenial -a amd64 -u user --password secretpassword --mirror $MIRROR }}<br />
<br />
{{Warning|Be sure to set systemd_container to yes in /etc/conf.d/lxc.CONTAINER. Otherwise, most functionality will be broken}}<br />
<br />
=== Unprivileged LXC images (Alpine / Debian / Ubuntu / Centos etc..) ===<br />
<br />
To enable unprivileged containers, one must create a uidgid map:<br />
<br />
echo root:1000000:65536 | tee -a /etc/subuid <br />
echo root:1000000:65536 | tee -a /etc/subgid<br />
<br />
This creates a uid and gid map for the root user starting at 1000000 with a size of 65536.<br />
<br />
To configure containers to use this mapping, add the following lines to the configuration:<br />
<br />
lxc.idmap = u 0 1000000 65536<br />
lxc.idmap = g 0 1000000 65536<br />
<br />
This can be in the global or container-specific configuration.<br />
<br />
To create an unprivileged lxc container, you need to use the download template. The download template must be installed:<br />
<br />
{{Cmd|apk add gnupg xz lxc-download<br />
lxc-create -n container-name -t download}}<br />
choose the Distribution | Release | Architecture.<br />
<br />
To be able to log in to a Debian container, you currently need to:<br />
{{Cmd|rm /lib/systemd/system/container-getty\@.service}}<br />
<br />
You can also [http://without-systemd.org/wiki/index.php/How_to_remove_systemd_from_a_Debian_jessie/sid_installationers remove Systemd from the container]{{dead link}}.<br />
<br />
== Starting/Stopping the guest ==<br />
<br />
First, you should enable the cgroup script:<br />
<br />
{{Cmd|rc-update add cgroups}}<br />
<br />
If you don't want to reboot, you can start the service by running<br />
<br />
{{Cmd|rc-service cgroups start}}<br />
<br />
Create a symlink to the ''/etc/init.d/lxc'' script for your guest.<br />
{{Cmd|ln -s lxc /etc/init.d/lxc.guest1}}<br />
<br />
You can start your guest with:<br />
{{Cmd|/etc/init.d/lxc.guest1 start}}<br />
<br />
Stop it with:<br />
{{Cmd|/etc/init.d/lxc.guest1 stop}}<br />
<br />
Make it autostart at boot-up with:<br />
{{Cmd| rc-update add lxc.guest1}}<br />
<br />
You can add to the container config: <code>lxc.start.auto = 1</code><br />
<br />
{{Cmd|rc-update add lxc}}<br />
<br />
to autostart containers with the lxc service only.<br />
<br />
== Connecting to the guest ==<br />
By default, sshd is not installed. You'll have to attach to the container or connect to the virtual console. This is done with:<br />
<br />
{{Cmd|lxc-attach -n guest1}}<br />
<br />
Type exit to detach from the container again (please check the grsec notes above)<br />
<br />
== Connect to virtual console ==<br />
<br />
{{Cmd|lxc-console -n guest1}}<br />
<br />
To disconnect, press {{key|Ctrl}}+{{key|a}} {{key|q}}<br />
<br />
== Deleting a guest ==<br />
Make sure the guest is stopped, then run:<br />
{{Cmd|lxc-destroy -n guest1}}<br />
This will erase everything, without asking any questions. It is equivalent to: {{Cmd|rm -r /var/lib/lxc/guest1}}<br />
<br />
== Advanced ==<br />
<br />
=== Creating a LXC container without modifying your network interfaces ===<br />
<br />
The problem with bridging is that the interface you bridge gets replaced with your new bridge interface.<br />
Let's say you have interface eth0 that you want to bridge. Your eth0 interface gets replaced with the br0 interface that you create. It also means that the interface you use needs to be placed into promiscuous mode to catch all the traffic that could de destined to the other side of the bridge, which may not be what you want.<br />
<br />
The solution is to create a dummy network interface, bridge that, and set up NAT so that traffic out of your bridge interface gets pushed through the interface of your choice.<br />
<br />
Let's create that dummy interface (thanks to ncopa for talking me out of macvlan and pointing out the dummy interface kernel module)<br />
<br />
{{Cmd|modprobe dummy}}<br />
<br />
This will create a dummy interface called dummy0 on your host. To create this interface on every boot, append "dummy" to /etc/modules:<br />
<br />
Now we will create a bridge called br0<br />
<br />
{{Cmd |brctl addbr br0<br />
brctl setfd br0 0 }}<br />
<br />
and then make that dummy interface one end of the bridge<br />
<br />
{{Cmd | brctl addif br0 dummy0 }}<br />
<br />
Next, let's give that bridged interface a reason to exist:<br />
<br />
{{ Cmd | ifconfig br0 192.168.1.1 netmask 255.255.255.0 up}}<br />
<br />
Create a file for your container. Let's say /etc/lxc/bridgenat.conf, with the following settings.<br />
<br />
<pre><br />
lxc.net.0.type = veth<br />
lxc.net.0.flags = up<br />
lxc.net.0.link = br0<br />
lxc.net.0.name = eth1<br />
lxc.net.0.ipv4.address = 192.168.1.2/24 192.168.1.255<br />
lxc.net.0.ipv4.gateway = 192.168.1.1<br />
lxc.net.0.veth.pair = veth-if-0<br />
</pre><br />
<br />
and build your container with that file:<br />
<br />
{{ Cmd | lxc-create -n alpine -f /etc/lxc/bridgenat.conf -t alpine }}<br />
<br />
You should now be able to ping your container from your host, and your host from your container.<br />
<br />
Your container needs to know where to push traffic that isn't within it's subnet. To do so, we tell the container to route through the bridge interface, br0<br />
From inside the container run<br />
<br />
{{ Cmd | route add default gw 192.168.1.1 }}<br />
<br />
The next step is to push the traffic coming from your private subnet over br0 out through your internet facing interface, or any interface you chose<br />
<br />
We are messing with your IP tables here, so make sure these settings don't conflict with anything you may have already set up.<br />
<br />
Say eth0 was your internet facing network interface, and br0 is the name of the bridge you made earlier. We'd do this:<br />
<br />
{{ Cmd | echo 1 > /proc/sys/net/ipv4/ip_forward<br />
iptables --table nat --append POSTROUTING --out-interface eth0 -j MASQUERADE<br />
iptables --append FORWARD --in-interface br0 -j ACCEPT<br />
}}<br />
<br />
Now you should be able to route through your bridge interface to the internet facing interface of your host from your container, just like at home!<br />
<br />
You could also have a dhcp server running on your host, and set it up to give IP addresses from your private subnet to any container that requests it, and then have one template for multiple alpine LXC containers, perfect for alpine development :)<br />
<br />
=== Using static IP ===<br />
<br />
If you're using static IP, you need to configure this properly on the guest /etc/network/interfaces. To stay in line with the above example, modify ''/var/lib/lxc/guest1/rootfs/etc/network/interfaces'' <br />
<br />
from<br />
<br />
#auto lo<br />
iface lo inet loopback<br />
auto eth0<br />
iface eth0 inet '''dhcp'''<br />
<br />
to <br />
<br />
#auto lo<br />
iface lo inet loopback<br />
auto eth0<br />
iface eth0 inet '''static'''<br />
address <lxc-container-ip> # IP which the lxc container should use<br />
gateway <gateway-ip> # IP of gateway to use, mostly same as on lxc-host<br />
netmask <netmask><br />
<br />
=== mem and swap ===<br />
<br />
{{Cmd|vim /boot/extlinux.conf}}<br />
<br />
{{Cmd|<br />
APPEND initrd{{=}}initramfs-3.10.13-1-grsec root{{=}}UUID{{=}}7cd8789f-5659-40f8-9548-ae8f89c918ab modules{{=}}sd-mod,usb-storage,ext4 quiet cgroup_enable{{=}}memory swapaccount{{=}}1<br />
}}<br />
<br />
=== checkconfig ===<br />
{{Cmd|lxc-checkconfig}}<br />
<br />
{{Cmd|<br />
Kernel configuration not found at /proc/config.gz; searching...<br />
Kernel configuration found at /boot/config-3.10.13-1-grsec<br />
--- Namespaces ---<br />
Namespaces: enabled<br />
Utsname namespace: enabled<br />
Ipc namespace: enabled<br />
Pid namespace: enabled<br />
User namespace: missing<br />
Network namespace: enabled<br />
Multiple /dev/pts instances: enabled<br />
<br />
--- Control groups ---<br />
Cgroup: enabled<br />
Cgroup clone_children flag: enabled<br />
Cgroup device: enabled<br />
Cgroup sched: enabled<br />
Cgroup cpu account: enabled<br />
Cgroup memory controller: missing<br />
Cgroup cpuset: enabled<br />
<br />
--- Misc ---<br />
Veth pair device: enabled<br />
Macvlan: enabled<br />
Vlan: enabled<br />
File capabilities: enabled<br />
<br />
Note : Before booting a new kernel, you can check its configuration<br />
usage : CONFIG{{=}}/path/to/config /usr/bin/lxc-checkconfig<br />
<br />
}}<br />
<br />
=== VirtualBox ===<br />
<br />
In order for the network to work on containers, you need to set "Promiscuous Mode" to "Allow All" in VirtualBox settings for the network adapter.<br />
<br />
[[File:VirtualBoxNetworkAdapter.jpg]]<br />
<br />
[[Category:Virtualization]]<br />
<br />
=== postgreSQL ===<br />
<br />
Inside the container run: {{Cmd|chmod go+w /dev/null}} to fix {{Cmd|rc-service postgresql start}}<br />
<br />
=== openVPN ===<br />
<br />
see [[Setting_up_a_OpenVPN_server#openVPN_and_LXC]]<br />
<br />
== LXC 1.0 Additional information ==<br />
<br />
Some info regarding new features in LXC 1.0<br />
<br />
https://www.stgraber.org/2013/12/20/lxc-1-0-blog-post-series/<br />
<br />
== See also ==<br />
* [[Howto-lxc-simple]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=How_to_setup_a_Alpine_Linux_mirror&diff=21264How to setup a Alpine Linux mirror2022-01-04T17:45:03Z<p>Geek at: changed alpine cdn</p>
<hr />
<div>== Introduction ==<br />
This document describes how to set up an Alpine Linux mirror and make it available via http and rsync.<br />
<br />
We will:<br />
* create the dir where we have the mirror<br />
* set up a cron job to sync with master mirror every hour<br />
* set up lighttpd for http access<br />
* set up rsync so other mirrors can rsync from you<br />
<br />
Make sure that you have enough disk space.<br />
<br />
Current (2021-06-10) disk usage in GB:<br />
<br />
{|class="wikitable"<br />
!edge<br />
!v3.0<br />
!v3.1<br />
!v3.2<br />
!v3.3<br />
!v3.4<br />
!v3.5<br />
!v3.6<br />
!v3.7<br />
!v3.8<br />
!v3.9<br />
!v3.10<br />
!v3.11<br />
!v3.12<br />
!v3.13<br />
!v3.14<br />
<br />
!total<br />
|-<br />
|228.6<br />
|16.5<br />
|17.5<br />
|14.5<br />
|20.4<br />
|24.3<br />
|26.8<br />
|44.3<br />
|42.5<br />
|58.9<br />
|72.8<br />
|112.1<br />
|134.6<br />
|146.4<br />
|147.4<br />
|143.4<br />
<br />
|'''1251.3'''<br />
|}<br />
<br />
Script used to calculate the size:<br />
<br />
<pre><br />
#!/usr/bin/env bash<br />
<br />
total=0<br />
dest="$(mktemp -d)"<br />
<br />
for dir in edge v3.0 v3.1 v3.2 v3.3 v3.4 v3.5 v3.6 v3.7 v3.8 v3.9 v3.10 v3.11 v3.12 v3.13 v3.14; do<br />
old_total="$total"<br />
src="rsync://rsync.alpinelinux.org/alpine/$dir/"<br />
size=`rsync -a -n --stats "$src" "$dest" | grep '^Total file size' | tr -d ',' | awk '{ print $4 }'`<br />
total=$(("$old_total" + "$size"))<br />
echo "$dir: $size" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'<br />
done<br />
<br />
echo "total: $total" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'<br />
rm -r "$dest"<br />
</pre><br />
<br />
== Setting up the cron job ==<br />
Install rsync which will be used to sync from the master mirror.<br />
{{Cmd|apk add rsync}}<br />
<br />
Save the following file as ''/etc/periodic/hourly/alpine-mirror''<br />
<pre><br />
#!/usr/bin/env sh<br />
<br />
# make sure we never run 2 rsync at the same time<br />
lockfile="/tmp/alpine-mirror.lock"<br />
if [ -z "$flock" ] ; then<br />
exec env flock=1 flock -n $lockfile "$0" "$@"<br />
fi<br />
<br />
src=rsync://rsync.alpinelinux.org/alpine/ <br />
dest=/var/www/localhost/htdocs/alpine/<br />
<br />
# uncomment this to exclude old v2.x branches<br />
#exclude="--exclude v2.*"<br />
<br />
mkdir -p "$dest"<br />
/usr/bin/rsync \<br />
--archive \<br />
--update \<br />
--hard-links \<br />
--delete \<br />
--delete-after \<br />
--delay-updates \<br />
--timeout=600 \<br />
$exclude \<br />
"$src" "$dest"<br />
<br />
</pre><br />
<br />
(or use [https://gist.github.com/jirutka/288c6fff7c0b8a835d143686207316be this script])<br />
<br />
Make it executable:<br />
{{Cmd|<nowiki>chmod +x /etc/periodic/hourly/alpine-mirror</nowiki>}}<br />
<br />
Now it will sync every hour. (given cron runs)<br />
<br />
== Setting up HTTP access via lighttpd ==<br />
<br />
Install the lighttpd server<br />
{{Cmd|apk add lighttpd}}<br />
<br />
Enable dir listings by uncommenting the following line in ''/etc/lighttpd/lighttpd.conf'':<br />
dir-listing.activate = "enable"<br />
<br />
Also set cache-control to force cache revalidate every 30 mins. Uncomment mod_setenv in ''/etc/lighttpd/lighttpd.conf'':<br />
"mod_setenv",<br />
<br />
Add also the following lines to ''/etc/lighttpd/lighttpd.conf'':<br />
setenv.add-response-header += ( <br />
"Cache-Control" => "must-revalidate"<br />
)<br />
<br />
Start lighttpd and make it start at boot:<br />
{{Cmd|rc-service lighttpd start<br />
rc-update add lighttpd}}<br />
<br />
{{Note|You may wish to consider [[Darkhttpd]] as an alternative to [[Lighttpd]]<br />
<br />
If so, simply install, start and auto-start the webserver:<br />
<br />
{{Cmd|apk add darkhttpd && rc-service darkhttpd start && rc-update add darkhttpd}}<br />
<br />
Darkhttpd will, by default, offer directory listings and serve data from /var/www/localhost/htdocs/ <br />
<br />
See the main article on [[Darkhttpd]] for more configuration options}}<br />
<br />
== Setting up rsyncd ==<br />
Add the following lines to ''/etc/rsyncd.conf'':<br />
<pre><br />
[alpine]<br />
path = /var/www/localhost/htdocs/alpine<br />
comment = My Alpine Linux Mirror<br />
</pre><br />
<br />
Optionally set a bandwidth limit in ''/etc/conf.d/rsyncd''. In this example we limit to 500Kbytes/s (approx 5Mbit/s)<br />
<pre><br />
RSYNC_OPTS="--bwlimit=500"<br />
</pre><br />
<br />
== Mirror statistics ==<br />
<br />
Simple bandwidth statistics can be generated with vnstat.<br />
<br />
{{Cmd|apk add vnstat}}<br />
<br />
edit /etc/vnstat.conf and replace the interface name with the appropriate one.<br />
<br />
Start vnstatd<br />
<br />
{{Cmd|/etc/init.d/vnstatd start }}<br />
<br />
copy the following script to /etc/periodic/15min/stats and make sure your crond is running.<br />
please not that heredoc should be tab indented or the script will fail. A working copy can be found here: http://tpaste.us/RrMv<br />
<br />
<pre><br />
#!/usr/bin/env sh<br />
<br />
output="/var/www/localhost/htdocs/.stats"<br />
nic="eth0"<br />
<br />
generate_index() {<br />
cat <<-EOF<br />
<!doctype html><br />
<html lang="en"><br />
<head><br />
<meta charset="utf-8"><br />
<meta http-equiv="cache-control" content=no-cache"><br />
<meta http-equiv="refresh" content="3000"><br />
<title>Alpine Linux mirror statistics</title><br />
</head><br />
<body><br />
<table border="0"><br />
<tr><td><img src="summary.png" alt="summary"></td><td><img src="hours.png" alt="hours"></td></tr><br />
<tr><td rowspan="2"><img src="days.png" alt="days"></td><td><img src="top10.png" alt="top10"></td></tr><br />
<tr><td><img src="months.png" alt="months"></td></tr><br />
</table><br />
</body><br />
</html><br />
EOF<br />
}<br />
<br />
if [ ! -f "$output"/index.html ]; then<br />
mkdir -p $output<br />
generate_index > "$output"/index.html<br />
fi<br />
<br />
for type in hours days months top10 summary hsummary vsummary; do<br />
vnstati --${type} -i $nic -o $output/${type}.png<br />
done<br />
</pre><br />
<br />
== Update mirror from mqtt ==<br />
<br />
If you want your mirror to be really uptodate compared to our master mirror you can subscribe to Alpine Linux message server "msg.alpinelinux.org" and check for upload messages.<br />
Add mqtt-exec to be able to execute processes when specific topics are being send.<br />
<br />
{{Cmd| apk add mqtt-exec}}<br />
<br />
mqtt-exec supports running multiple time so we need to setup a specific config.<br />
<br />
{{Cmd| ln -s /etc/init.d/mqtt-exec /etc/init.d/mqtt-exec.sync-mirror}}<br />
<br />
{{Cmd| ln -s /etc/conf.d/mqtt-exec /etc/conf.d/mqtt-exec.sync-mirror}}<br />
<br />
edit /etc/conf.d/mqtt-exec.sync-mirror<br />
<br />
<pre><br />
mqtt_topics="rsync/rsync.alpinelinux.org/#"<br />
exec_user="buildozer"<br />
exec_command="/usr/local/bin/sync-mirror"<br />
</pre><br />
<br />
Copy the following file to /usr/local/bin/sync-mirror and make it executable (dont forget to update the variables).<br />
<br />
<pre><br />
#!/bin/sh<br />
<br />
src="rsync://rsync.alpinelinux.org/alpine/"<br />
dest="/var/www/localhost/htdocs/alpine/"<br />
lock="/tmp/sync-mirror.lock"<br />
topic="$1"<br />
dir="$2"<br />
<br />
[ -z "$flock" ] && exec env flock=1 flock $lock $0 "$@"<br />
<br />
if [ -n "$dir" ] && [ -d "$dest/${dir%/*}" ]; then<br />
logger "Syncing directory: $dir"<br />
src="${src}${dir%/}/"<br />
dest="${dest}${dir%/}/"<br />
else<br />
logger "Syncing all directories"<br />
fi<br />
<br />
/usr/bin/rsync \<br />
--archive \<br />
--update \<br />
--verbose \<br />
--progress \<br />
--timeout=600 \<br />
--delay-updates \<br />
--delete-after \<br />
"$src" \<br />
"$dest"<br />
</pre><br />
<br />
And finally start mqtt-exec and let it listen on msg.alpinelinux.org<br />
<br />
{{Cmd|/etc/init.d/mqtt-exec.sync-mirror start}}<br />
<br />
To make sure you are not missing any packages (in case something goes wrong with MQTT subscription) you can periodically sync all directories by adding the script to cron.<br />
<br />
{{Cmd|ln -s /usr/local/bin/sync-mirror /etc/periodic/hourly/sync-mirror}}<br />
<br />
Now watch your syslog as it should tell you when it will update directories in your local mirror.<br />
<br />
== Partial mirror using nginx ==<br />
<br />
For a private mirror it might make sense to sync only the newest versions of Alpine to save space, but if you ''do'' point an old Alpine version to your mirror they should still be able to install packages. We can achieve this by using nginx to serve the mirrored content and use regex location matching to redirect requests to a public mirror.<br />
<br />
Let's assume you chose to only mirror Alpine versions up from v3.13. If a client asks your mirror for v.3.10 it should redirect to another mirror.<br />
<br />
Your nginx config server block should look something like this:<br />
<br />
<pre><br />
server {<br />
listen 80;<br />
server_name alpine.mydomain.local;<br />
root /data/alpine; # point to where your alpine mirror is located. make sure nginx is allowed to read it<br />
autoindex on; # Enable indexing<br />
<br />
# the following location block will match for v3.0 to v3.12<br />
# and will forward it to dl-4.alpinelinux.org.<br />
location ~* /(v3\.([1-9]|1[012]))$<br />
{<br />
return 302 http://dl-cdn.alpinelinux.org/alpine$request_uri;<br />
}<br />
}<br />
</pre><br />
<br />
The corresponding sync script could look something like this:<br />
<br />
<pre><br />
#!/usr/bin/env sh<br />
<br />
# make sure we never run 2 rsync at the same time<br />
lockfile="/tmp/alpine-mirror.lock"<br />
if [ -z "$flock" ] ; then<br />
exec env flock=1 flock -n $lockfile "$0" "$@"<br />
fi<br />
<br />
src=rsync://rsync.alpinelinux.org/alpine/<br />
dest=/data/alpine/<br />
<br />
exclude="--exclude v2.* --exclude v3.0 --exclude v3.1 --exclude v3.2 --exclude v3.3 --exclude v3.4 --exclude v3.5 --exclude v3.6 --exclude v3.7 --exclude v3.8 --exclude v3.9 --exclude v3.10 --exclude v3.11 --exclude v3.12"<br />
<br />
mkdir -p "$dest"<br />
/usr/bin/rsync -vvv \<br />
--archive \<br />
--update \<br />
--hard-links \<br />
--delete \<br />
--delete-after \<br />
--delete-excluded \<br />
--delay-updates \<br />
--timeout=600 \<br />
$exclude \<br />
"$src" "$dest"<br />
</pre><br />
<br />
[[Category:Server]]<br />
[[Category:Package Manager]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=How_to_setup_a_Alpine_Linux_mirror&diff=21231How to setup a Alpine Linux mirror2022-01-02T22:36:04Z<p>Geek at: /* Partial mirror using nginx */</p>
<hr />
<div>== Introduction ==<br />
This document describes how to set up an Alpine Linux mirror and make it available via http and rsync.<br />
<br />
We will:<br />
* create the dir where we have the mirror<br />
* set up a cron job to sync with master mirror every hour<br />
* set up lighttpd for http access<br />
* set up rsync so other mirrors can rsync from you<br />
<br />
Make sure that you have enough disk space.<br />
<br />
Current (2021-06-10) disk usage in GB:<br />
<br />
{|class="wikitable"<br />
!edge<br />
!v3.0<br />
!v3.1<br />
!v3.2<br />
!v3.3<br />
!v3.4<br />
!v3.5<br />
!v3.6<br />
!v3.7<br />
!v3.8<br />
!v3.9<br />
!v3.10<br />
!v3.11<br />
!v3.12<br />
!v3.13<br />
!v3.14<br />
<br />
!total<br />
|-<br />
|228.6<br />
|16.5<br />
|17.5<br />
|14.5<br />
|20.4<br />
|24.3<br />
|26.8<br />
|44.3<br />
|42.5<br />
|58.9<br />
|72.8<br />
|112.1<br />
|134.6<br />
|146.4<br />
|147.4<br />
|143.4<br />
<br />
|'''1251.3'''<br />
|}<br />
<br />
Script used to calculate the size:<br />
<br />
<pre><br />
#!/usr/bin/env bash<br />
<br />
total=0<br />
dest="$(mktemp -d)"<br />
<br />
for dir in edge v3.0 v3.1 v3.2 v3.3 v3.4 v3.5 v3.6 v3.7 v3.8 v3.9 v3.10 v3.11 v3.12 v3.13 v3.14; do<br />
old_total="$total"<br />
src="rsync://rsync.alpinelinux.org/alpine/$dir/"<br />
size=`rsync -a -n --stats "$src" "$dest" | grep '^Total file size' | tr -d ',' | awk '{ print $4 }'`<br />
total=$(("$old_total" + "$size"))<br />
echo "$dir: $size" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'<br />
done<br />
<br />
echo "total: $total" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'<br />
rm -r "$dest"<br />
</pre><br />
<br />
== Setting up the cron job ==<br />
Install rsync which will be used to sync from the master mirror.<br />
{{Cmd|apk add rsync}}<br />
<br />
Save the following file as ''/etc/periodic/hourly/alpine-mirror''<br />
<pre><br />
#!/usr/bin/env sh<br />
<br />
# make sure we never run 2 rsync at the same time<br />
lockfile="/tmp/alpine-mirror.lock"<br />
if [ -z "$flock" ] ; then<br />
exec env flock=1 flock -n $lockfile "$0" "$@"<br />
fi<br />
<br />
src=rsync://rsync.alpinelinux.org/alpine/ <br />
dest=/var/www/localhost/htdocs/alpine/<br />
<br />
# uncomment this to exclude old v2.x branches<br />
#exclude="--exclude v2.*"<br />
<br />
mkdir -p "$dest"<br />
/usr/bin/rsync \<br />
--archive \<br />
--update \<br />
--hard-links \<br />
--delete \<br />
--delete-after \<br />
--delay-updates \<br />
--timeout=600 \<br />
$exclude \<br />
"$src" "$dest"<br />
<br />
</pre><br />
<br />
(or use [https://gist.github.com/jirutka/288c6fff7c0b8a835d143686207316be this script])<br />
<br />
Make it executable:<br />
{{Cmd|<nowiki>chmod +x /etc/periodic/hourly/alpine-mirror</nowiki>}}<br />
<br />
Now it will sync every hour. (given cron runs)<br />
<br />
== Setting up HTTP access via lighttpd ==<br />
<br />
Install the lighttpd server<br />
{{Cmd|apk add lighttpd}}<br />
<br />
Enable dir listings by uncommenting the following line in ''/etc/lighttpd/lighttpd.conf'':<br />
dir-listing.activate = "enable"<br />
<br />
Also set cache-control to force cache revalidate every 30 mins. Uncomment mod_setenv in ''/etc/lighttpd/lighttpd.conf'':<br />
"mod_setenv",<br />
<br />
Add also the following lines to ''/etc/lighttpd/lighttpd.conf'':<br />
setenv.add-response-header += ( <br />
"Cache-Control" => "must-revalidate"<br />
)<br />
<br />
Start lighttpd and make it start at boot:<br />
{{Cmd|rc-service lighttpd start<br />
rc-update add lighttpd}}<br />
<br />
{{Note|You may wish to consider [[Darkhttpd]] as an alternative to [[Lighttpd]]<br />
<br />
If so, simply install, start and auto-start the webserver:<br />
<br />
{{Cmd|apk add darkhttpd && rc-service darkhttpd start && rc-update add darkhttpd}}<br />
<br />
Darkhttpd will, by default, offer directory listings and serve data from /var/www/localhost/htdocs/ <br />
<br />
See the main article on [[Darkhttpd]] for more configuration options}}<br />
<br />
== Setting up rsyncd ==<br />
Add the following lines to ''/etc/rsyncd.conf'':<br />
<pre><br />
[alpine]<br />
path = /var/www/localhost/htdocs/alpine<br />
comment = My Alpine Linux Mirror<br />
</pre><br />
<br />
Optionally set a bandwidth limit in ''/etc/conf.d/rsyncd''. In this example we limit to 500Kbytes/s (approx 5Mbit/s)<br />
<pre><br />
RSYNC_OPTS="--bwlimit=500"<br />
</pre><br />
<br />
== Mirror statistics ==<br />
<br />
Simple bandwidth statistics can be generated with vnstat.<br />
<br />
{{Cmd|apk add vnstat}}<br />
<br />
edit /etc/vnstat.conf and replace the interface name with the appropriate one.<br />
<br />
Start vnstatd<br />
<br />
{{Cmd|/etc/init.d/vnstatd start }}<br />
<br />
copy the following script to /etc/periodic/15min/stats and make sure your crond is running.<br />
please not that heredoc should be tab indented or the script will fail. A working copy can be found here: http://tpaste.us/RrMv<br />
<br />
<pre><br />
#!/usr/bin/env sh<br />
<br />
output="/var/www/localhost/htdocs/.stats"<br />
nic="eth0"<br />
<br />
generate_index() {<br />
cat <<-EOF<br />
<!doctype html><br />
<html lang="en"><br />
<head><br />
<meta charset="utf-8"><br />
<meta http-equiv="cache-control" content=no-cache"><br />
<meta http-equiv="refresh" content="3000"><br />
<title>Alpine Linux mirror statistics</title><br />
</head><br />
<body><br />
<table border="0"><br />
<tr><td><img src="summary.png" alt="summary"></td><td><img src="hours.png" alt="hours"></td></tr><br />
<tr><td rowspan="2"><img src="days.png" alt="days"></td><td><img src="top10.png" alt="top10"></td></tr><br />
<tr><td><img src="months.png" alt="months"></td></tr><br />
</table><br />
</body><br />
</html><br />
EOF<br />
}<br />
<br />
if [ ! -f "$output"/index.html ]; then<br />
mkdir -p $output<br />
generate_index > "$output"/index.html<br />
fi<br />
<br />
for type in hours days months top10 summary hsummary vsummary; do<br />
vnstati --${type} -i $nic -o $output/${type}.png<br />
done<br />
</pre><br />
<br />
== Update mirror from mqtt ==<br />
<br />
If you want your mirror to be really uptodate compared to our master mirror you can subscribe to Alpine Linux message server "msg.alpinelinux.org" and check for upload messages.<br />
Add mqtt-exec to be able to execute processes when specific topics are being send.<br />
<br />
{{Cmd| apk add mqtt-exec}}<br />
<br />
mqtt-exec supports running multiple time so we need to setup a specific config.<br />
<br />
{{Cmd| ln -s /etc/init.d/mqtt-exec /etc/init.d/mqtt-exec.sync-mirror}}<br />
<br />
{{Cmd| ln -s /etc/conf.d/mqtt-exec /etc/conf.d/mqtt-exec.sync-mirror}}<br />
<br />
edit /etc/conf.d/mqtt-exec.sync-mirror<br />
<br />
<pre><br />
mqtt_topics="rsync/rsync.alpinelinux.org/#"<br />
exec_user="buildozer"<br />
exec_command="/usr/local/bin/sync-mirror"<br />
</pre><br />
<br />
Copy the following file to /usr/local/bin/sync-mirror and make it executable (dont forget to update the variables).<br />
<br />
<pre><br />
#!/bin/sh<br />
<br />
src="rsync://rsync.alpinelinux.org/alpine/"<br />
dest="/var/www/localhost/htdocs/alpine/"<br />
lock="/tmp/sync-mirror.lock"<br />
topic="$1"<br />
dir="$2"<br />
<br />
[ -z "$flock" ] && exec env flock=1 flock $lock $0 "$@"<br />
<br />
if [ -n "$dir" ] && [ -d "$dest/${dir%/*}" ]; then<br />
logger "Syncing directory: $dir"<br />
src="${src}${dir%/}/"<br />
dest="${dest}${dir%/}/"<br />
else<br />
logger "Syncing all directories"<br />
fi<br />
<br />
/usr/bin/rsync \<br />
--archive \<br />
--update \<br />
--verbose \<br />
--progress \<br />
--timeout=600 \<br />
--delay-updates \<br />
--delete-after \<br />
"$src" \<br />
"$dest"<br />
</pre><br />
<br />
And finally start mqtt-exec and let it listen on msg.alpinelinux.org<br />
<br />
{{Cmd|/etc/init.d/mqtt-exec.sync-mirror start}}<br />
<br />
To make sure you are not missing any packages (in case something goes wrong with MQTT subscription) you can periodically sync all directories by adding the script to cron.<br />
<br />
{{Cmd|ln -s /usr/local/bin/sync-mirror /etc/periodic/hourly/sync-mirror}}<br />
<br />
Now watch your syslog as it should tell you when it will update directories in your local mirror.<br />
<br />
== Partial mirror using nginx ==<br />
<br />
For a private mirror it might make sense to sync only the newest versions of Alpine to save space, but if you ''do'' point an old alpine version to your mirror they should still be able to install packages. We can achieve this by using nginx to serve the mirrored content and use regex location matching to redirect requests to a public mirror.<br />
<br />
Let's assume you chose to only mirror alpine versions up from v3.13. If a client asks your mirror for v.3.10 it should redirect to another mirror.<br />
<br />
Your nginx config server block should look something like this:<br />
<br />
<pre><br />
server {<br />
listen 80;<br />
server_name alpine.mydomain.local;<br />
root /data/alpine; # point to where your alpine mirror is located. make sure nginx is allowed to read it<br />
autoindex on; # Enable indexing<br />
<br />
# the following location block will match for v3.0 to v3.12<br />
# and will forward it to dl-4.alpinelinux.org.<br />
location ~* /(v3\.([1-9]|1[012]))$<br />
{<br />
return 302 http://dl-4.alpinelinux.org/alpine$request_uri;<br />
}<br />
}<br />
</pre><br />
<br />
The corresponding sync script could look something like this:<br />
<br />
<pre><br />
#!/usr/bin/env sh<br />
<br />
# make sure we never run 2 rsync at the same time<br />
lockfile="/tmp/alpine-mirror.lock"<br />
if [ -z "$flock" ] ; then<br />
exec env flock=1 flock -n $lockfile "$0" "$@"<br />
fi<br />
<br />
src=rsync://rsync.alpinelinux.org/alpine/<br />
dest=/data/alpine/<br />
<br />
exclude="--exclude v2.* --exclude v3.0 --exclude v3.1 --exclude v3.2 --exclude v3.3 --exclude v3.4 --exclude v3.5 --exclude v3.6 --exclude v3.7 --exclude v3.8 --exclude v3.9 --exclude v3.10 --exclude v3.11 --exclude v3.12"<br />
<br />
mkdir -p "$dest"<br />
/usr/bin/rsync -vvv \<br />
--archive \<br />
--update \<br />
--hard-links \<br />
--delete \<br />
--delete-after \<br />
--delete-excluded \<br />
--delay-updates \<br />
--timeout=600 \<br />
$exclude \<br />
"$src" "$dest"<br />
</pre><br />
<br />
[[Category:Server]]<br />
[[Category:Package Manager]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=How_to_setup_a_Alpine_Linux_mirror&diff=21230How to setup a Alpine Linux mirror2022-01-02T22:35:14Z<p>Geek at: added info on how to make a partial mirror using nginx</p>
<hr />
<div>== Introduction ==<br />
This document describes how to set up an Alpine Linux mirror and make it available via http and rsync.<br />
<br />
We will:<br />
* create the dir where we have the mirror<br />
* set up a cron job to sync with master mirror every hour<br />
* set up lighttpd for http access<br />
* set up rsync so other mirrors can rsync from you<br />
<br />
Make sure that you have enough disk space.<br />
<br />
Current (2021-06-10) disk usage in GB:<br />
<br />
{|class="wikitable"<br />
!edge<br />
!v3.0<br />
!v3.1<br />
!v3.2<br />
!v3.3<br />
!v3.4<br />
!v3.5<br />
!v3.6<br />
!v3.7<br />
!v3.8<br />
!v3.9<br />
!v3.10<br />
!v3.11<br />
!v3.12<br />
!v3.13<br />
!v3.14<br />
<br />
!total<br />
|-<br />
|228.6<br />
|16.5<br />
|17.5<br />
|14.5<br />
|20.4<br />
|24.3<br />
|26.8<br />
|44.3<br />
|42.5<br />
|58.9<br />
|72.8<br />
|112.1<br />
|134.6<br />
|146.4<br />
|147.4<br />
|143.4<br />
<br />
|'''1251.3'''<br />
|}<br />
<br />
Script used to calculate the size:<br />
<br />
<pre><br />
#!/usr/bin/env bash<br />
<br />
total=0<br />
dest="$(mktemp -d)"<br />
<br />
for dir in edge v3.0 v3.1 v3.2 v3.3 v3.4 v3.5 v3.6 v3.7 v3.8 v3.9 v3.10 v3.11 v3.12 v3.13 v3.14; do<br />
old_total="$total"<br />
src="rsync://rsync.alpinelinux.org/alpine/$dir/"<br />
size=`rsync -a -n --stats "$src" "$dest" | grep '^Total file size' | tr -d ',' | awk '{ print $4 }'`<br />
total=$(("$old_total" + "$size"))<br />
echo "$dir: $size" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'<br />
done<br />
<br />
echo "total: $total" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'<br />
rm -r "$dest"<br />
</pre><br />
<br />
== Setting up the cron job ==<br />
Install rsync which will be used to sync from the master mirror.<br />
{{Cmd|apk add rsync}}<br />
<br />
Save the following file as ''/etc/periodic/hourly/alpine-mirror''<br />
<pre><br />
#!/usr/bin/env sh<br />
<br />
# make sure we never run 2 rsync at the same time<br />
lockfile="/tmp/alpine-mirror.lock"<br />
if [ -z "$flock" ] ; then<br />
exec env flock=1 flock -n $lockfile "$0" "$@"<br />
fi<br />
<br />
src=rsync://rsync.alpinelinux.org/alpine/ <br />
dest=/var/www/localhost/htdocs/alpine/<br />
<br />
# uncomment this to exclude old v2.x branches<br />
#exclude="--exclude v2.*"<br />
<br />
mkdir -p "$dest"<br />
/usr/bin/rsync \<br />
--archive \<br />
--update \<br />
--hard-links \<br />
--delete \<br />
--delete-after \<br />
--delay-updates \<br />
--timeout=600 \<br />
$exclude \<br />
"$src" "$dest"<br />
<br />
</pre><br />
<br />
(or use [https://gist.github.com/jirutka/288c6fff7c0b8a835d143686207316be this script])<br />
<br />
Make it executable:<br />
{{Cmd|<nowiki>chmod +x /etc/periodic/hourly/alpine-mirror</nowiki>}}<br />
<br />
Now it will sync every hour. (given cron runs)<br />
<br />
== Setting up HTTP access via lighttpd ==<br />
<br />
Install the lighttpd server<br />
{{Cmd|apk add lighttpd}}<br />
<br />
Enable dir listings by uncommenting the following line in ''/etc/lighttpd/lighttpd.conf'':<br />
dir-listing.activate = "enable"<br />
<br />
Also set cache-control to force cache revalidate every 30 mins. Uncomment mod_setenv in ''/etc/lighttpd/lighttpd.conf'':<br />
"mod_setenv",<br />
<br />
Add also the following lines to ''/etc/lighttpd/lighttpd.conf'':<br />
setenv.add-response-header += ( <br />
"Cache-Control" => "must-revalidate"<br />
)<br />
<br />
Start lighttpd and make it start at boot:<br />
{{Cmd|rc-service lighttpd start<br />
rc-update add lighttpd}}<br />
<br />
{{Note|You may wish to consider [[Darkhttpd]] as an alternative to [[Lighttpd]]<br />
<br />
If so, simply install, start and auto-start the webserver:<br />
<br />
{{Cmd|apk add darkhttpd && rc-service darkhttpd start && rc-update add darkhttpd}}<br />
<br />
Darkhttpd will, by default, offer directory listings and serve data from /var/www/localhost/htdocs/ <br />
<br />
See the main article on [[Darkhttpd]] for more configuration options}}<br />
<br />
== Setting up rsyncd ==<br />
Add the following lines to ''/etc/rsyncd.conf'':<br />
<pre><br />
[alpine]<br />
path = /var/www/localhost/htdocs/alpine<br />
comment = My Alpine Linux Mirror<br />
</pre><br />
<br />
Optionally set a bandwidth limit in ''/etc/conf.d/rsyncd''. In this example we limit to 500Kbytes/s (approx 5Mbit/s)<br />
<pre><br />
RSYNC_OPTS="--bwlimit=500"<br />
</pre><br />
<br />
== Mirror statistics ==<br />
<br />
Simple bandwidth statistics can be generated with vnstat.<br />
<br />
{{Cmd|apk add vnstat}}<br />
<br />
edit /etc/vnstat.conf and replace the interface name with the appropriate one.<br />
<br />
Start vnstatd<br />
<br />
{{Cmd|/etc/init.d/vnstatd start }}<br />
<br />
copy the following script to /etc/periodic/15min/stats and make sure your crond is running.<br />
please not that heredoc should be tab indented or the script will fail. A working copy can be found here: http://tpaste.us/RrMv<br />
<br />
<pre><br />
#!/usr/bin/env sh<br />
<br />
output="/var/www/localhost/htdocs/.stats"<br />
nic="eth0"<br />
<br />
generate_index() {<br />
cat <<-EOF<br />
<!doctype html><br />
<html lang="en"><br />
<head><br />
<meta charset="utf-8"><br />
<meta http-equiv="cache-control" content=no-cache"><br />
<meta http-equiv="refresh" content="3000"><br />
<title>Alpine Linux mirror statistics</title><br />
</head><br />
<body><br />
<table border="0"><br />
<tr><td><img src="summary.png" alt="summary"></td><td><img src="hours.png" alt="hours"></td></tr><br />
<tr><td rowspan="2"><img src="days.png" alt="days"></td><td><img src="top10.png" alt="top10"></td></tr><br />
<tr><td><img src="months.png" alt="months"></td></tr><br />
</table><br />
</body><br />
</html><br />
EOF<br />
}<br />
<br />
if [ ! -f "$output"/index.html ]; then<br />
mkdir -p $output<br />
generate_index > "$output"/index.html<br />
fi<br />
<br />
for type in hours days months top10 summary hsummary vsummary; do<br />
vnstati --${type} -i $nic -o $output/${type}.png<br />
done<br />
</pre><br />
<br />
== Update mirror from mqtt ==<br />
<br />
If you want your mirror to be really uptodate compared to our master mirror you can subscribe to Alpine Linux message server "msg.alpinelinux.org" and check for upload messages.<br />
Add mqtt-exec to be able to execute processes when specific topics are being send.<br />
<br />
{{Cmd| apk add mqtt-exec}}<br />
<br />
mqtt-exec supports running multiple time so we need to setup a specific config.<br />
<br />
{{Cmd| ln -s /etc/init.d/mqtt-exec /etc/init.d/mqtt-exec.sync-mirror}}<br />
<br />
{{Cmd| ln -s /etc/conf.d/mqtt-exec /etc/conf.d/mqtt-exec.sync-mirror}}<br />
<br />
edit /etc/conf.d/mqtt-exec.sync-mirror<br />
<br />
<pre><br />
mqtt_topics="rsync/rsync.alpinelinux.org/#"<br />
exec_user="buildozer"<br />
exec_command="/usr/local/bin/sync-mirror"<br />
</pre><br />
<br />
Copy the following file to /usr/local/bin/sync-mirror and make it executable (dont forget to update the variables).<br />
<br />
<pre><br />
#!/bin/sh<br />
<br />
src="rsync://rsync.alpinelinux.org/alpine/"<br />
dest="/var/www/localhost/htdocs/alpine/"<br />
lock="/tmp/sync-mirror.lock"<br />
topic="$1"<br />
dir="$2"<br />
<br />
[ -z "$flock" ] && exec env flock=1 flock $lock $0 "$@"<br />
<br />
if [ -n "$dir" ] && [ -d "$dest/${dir%/*}" ]; then<br />
logger "Syncing directory: $dir"<br />
src="${src}${dir%/}/"<br />
dest="${dest}${dir%/}/"<br />
else<br />
logger "Syncing all directories"<br />
fi<br />
<br />
/usr/bin/rsync \<br />
--archive \<br />
--update \<br />
--verbose \<br />
--progress \<br />
--timeout=600 \<br />
--delay-updates \<br />
--delete-after \<br />
"$src" \<br />
"$dest"<br />
</pre><br />
<br />
And finally start mqtt-exec and let it listen on msg.alpinelinux.org<br />
<br />
{{Cmd|/etc/init.d/mqtt-exec.sync-mirror start}}<br />
<br />
To make sure you are not missing any packages (in case something goes wrong with MQTT subscription) you can periodically sync all directories by adding the script to cron.<br />
<br />
{{Cmd|ln -s /usr/local/bin/sync-mirror /etc/periodic/hourly/sync-mirror}}<br />
<br />
Now watch your syslog as it should tell you when it will update directories in your local mirror.<br />
<br />
== Partial mirror using nginx ==<br />
<br />
For a private mirror it might make sense to sync only the newest versions of nginx to save space, but if you do point an old alpine version to your mirror they should still be able to install packages. We can achieve this by using nginx to serve the mirrored content and use regex location matching to redirect requests to a public mirror.<br />
<br />
Let's assume you chose to only mirror alpine versions up from v3.13. If a client asks your mirror for v.3.10 it should redirect to another mirror.<br />
<br />
Your nginx config server block should look something like this:<br />
<br />
<pre><br />
server {<br />
listen 80;<br />
server_name alpine.mydomain.local;<br />
root /data/alpine; # point to where your alpine mirror is located. make sure nginx is allowed to read it<br />
autoindex on; # Enable indexing<br />
<br />
# the following location block will match for v3.0 to v3.12<br />
# and will forward it to dl-4.alpinelinux.org.<br />
location ~* /(v3\.([1-9]|1[012]))$<br />
{<br />
return 302 http://dl-4.alpinelinux.org/alpine$request_uri;<br />
}<br />
}<br />
</pre><br />
<br />
The corresponding sync script could look something like this:<br />
<br />
<pre><br />
#!/usr/bin/env sh<br />
<br />
# make sure we never run 2 rsync at the same time<br />
lockfile="/tmp/alpine-mirror.lock"<br />
if [ -z "$flock" ] ; then<br />
exec env flock=1 flock -n $lockfile "$0" "$@"<br />
fi<br />
<br />
src=rsync://rsync.alpinelinux.org/alpine/<br />
dest=/data/alpine/<br />
<br />
exclude="--exclude v2.* --exclude v3.0 --exclude v3.1 --exclude v3.2 --exclude v3.3 --exclude v3.4 --exclude v3.5 --exclude v3.6 --exclude v3.7 --exclude v3.8 --exclude v3.9 --exclude v3.10 --exclude v3.11 --exclude v3.12"<br />
<br />
mkdir -p "$dest"<br />
/usr/bin/rsync -vvv \<br />
--archive \<br />
--update \<br />
--hard-links \<br />
--delete \<br />
--delete-after \<br />
--delete-excluded \<br />
--delay-updates \<br />
--timeout=600 \<br />
$exclude \<br />
"$src" "$dest"<br />
</pre><br />
<br />
[[Category:Server]]<br />
[[Category:Package Manager]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=VLAN&diff=21138VLAN2021-12-28T22:40:58Z<p>Geek at: /* Example with two interfaces on the same adapter. One with vlan and one without */</p>
<hr />
<div>This article shows how to configure a network interface as an IEEE 802.1q VLAN trunk.<br />
<br />
__TOC__<br />
<br />
{{Note| Alpine Linux v2.4 or later is required}}<br />
==Installation==<br />
First, install the ''vlan'' package. This will give you support for vlans in the ''/etc/network/interfaces'' file.<br />
{{Cmd|apk add vlan}}<br />
<br />
==Configuration==<br />
Edit the ''/etc/network/interfaces'' file:<br />
<pre><br />
auto eth0.8<br />
iface eth0.8 inet static<br />
address 192.168.0.2<br />
netmask 255.255.255.0<br />
gateway 192.168.0.1<br />
</pre><br />
With the ''vlan'' package installed, ifup will find the trailing .8 in eth0.8 and will create a vlan interface with vid 8 over eth0.<br />
<br />
Alternativly with vlan8 over eth0:<br />
<pre><br />
auto vlan8<br />
iface vlan8 inet static<br />
address 192.168.0.2<br />
netmask 255.255.255.0<br />
gateway 192.168.0.1<br />
vlan-raw-device eth0<br />
</pre><br />
<br />
A static ip address was used in the examples shown above, but dhcp can be used as well.<br />
<br />
== Example with bridges associated with VLANs over bonding with differing MTUs on the various VLANs ==<br />
This serves as an example of some of the more complicated networking possible. Particularly, this would work well for a hypervisor attached to a dedicated storage VLAN. Less complicated implementations can be achieved by merely removing the non-applicable parts.<br />
<pre><br />
auto lo<br />
iface lo inet loopback<br />
<br />
auto bond0<br />
iface bond0 inet manual<br />
bond_slaves eth0 eth1<br />
bond_mode 802.3ad<br />
bond_miimon 100<br />
bond_xmit_hash_policy layer2+3<br />
post-up ip link set dev bond0 mtu 9000<br />
<br />
iface bond0.1 inet manual<br />
<br />
auto br1<br />
iface br1 inet static<br />
address 192.168.1.196<br />
netmask 255.255.255.0<br />
gateway 192.168.1.1<br />
bridge_ports bond0.1<br />
bridge_stp off<br />
bridge_fd 0.0<br />
post-up ip link set dev bond0.1 mtu 1500<br />
<br />
iface bond0.10 inet manual<br />
<br />
auto br10<br />
iface br10 inet static<br />
address 192.168.10.1<br />
netmask 255.255.255.0<br />
bridge_ports bond0.10<br />
bridge_stp off<br />
bridge_fd 0.0<br />
</pre><br />
<br />
== Example with two interfaces on the same adapter. One with vlan and one without ==<br />
<br />
Since linux doesn't allow multiple default gateways we need to use a second routing table using iproute2<br />
<br />
{{Cmd|apk add iproute2}}<br />
<br />
Then we'll add two new routig tables to the config file. One for each network<br />
<br />
{{Cmd|echo "1 rt1" >> /etc/iproute2/rt_tables;echo "2 rt2" >> /etc/iproute2/rt_tables;}}<br />
<br />
Now we need to edit /etc/network/interfaces<br />
<br />
<pre><br />
auto lo<br />
iface lo inet loopback<br />
<br />
# the native interface without a vlan (also called untagged)<br />
<br />
auto eth0<br />
iface eth0 inet static<br />
address 192.168.1.100<br />
netmask 255.255.255.0<br />
gateway 192.168.1.1<br />
post-up ip route add 192.168.1.0/24 dev eth0 src 192.168.1.100 table rt1<br />
post-up ip route add default via 192.168.1.1 dev eth0 table rt1 # the actual gateway for this interface<br />
post-up ip rule add from 192.168.1.100/32 table rt1<br />
post-up ip rule add to 192.168.1.100/32 table rt1<br />
<br />
# second interface with the vlan tag 5<br />
auto eth0.5<br />
iface eth0.5 inet static<br />
address 192.168.5.100<br />
netmask 255.255.255.0<br />
post-up ip route add 192.168.5.0/24 dev eth0.5 src 192.168.5.100 table rt2<br />
post-up ip route add default via 192.168.5.1 dev eth0.5 table rt2 # the actual gateway for this interface<br />
post-up ip rule add from 192.168.5.100/32 table rt2<br />
post-up ip rule add to 192.168.5.100/32 table rt2<br />
</pre><br />
<br />
Note that if you want to add a third interface this way, you'll have to add another routing table<br />
<br />
[[Category:Networking]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=VLAN&diff=21137VLAN2021-12-28T21:42:37Z<p>Geek at: /* Example with two interfaces on the same adapter. One with vlan and one without */</p>
<hr />
<div>This article shows how to configure a network interface as an IEEE 802.1q VLAN trunk.<br />
<br />
__TOC__<br />
<br />
{{Note| Alpine Linux v2.4 or later is required}}<br />
==Installation==<br />
First, install the ''vlan'' package. This will give you support for vlans in the ''/etc/network/interfaces'' file.<br />
{{Cmd|apk add vlan}}<br />
<br />
==Configuration==<br />
Edit the ''/etc/network/interfaces'' file:<br />
<pre><br />
auto eth0.8<br />
iface eth0.8 inet static<br />
address 192.168.0.2<br />
netmask 255.255.255.0<br />
gateway 192.168.0.1<br />
</pre><br />
With the ''vlan'' package installed, ifup will find the trailing .8 in eth0.8 and will create a vlan interface with vid 8 over eth0.<br />
<br />
Alternativly with vlan8 over eth0:<br />
<pre><br />
auto vlan8<br />
iface vlan8 inet static<br />
address 192.168.0.2<br />
netmask 255.255.255.0<br />
gateway 192.168.0.1<br />
vlan-raw-device eth0<br />
</pre><br />
<br />
A static ip address was used in the examples shown above, but dhcp can be used as well.<br />
<br />
== Example with bridges associated with VLANs over bonding with differing MTUs on the various VLANs ==<br />
This serves as an example of some of the more complicated networking possible. Particularly, this would work well for a hypervisor attached to a dedicated storage VLAN. Less complicated implementations can be achieved by merely removing the non-applicable parts.<br />
<pre><br />
auto lo<br />
iface lo inet loopback<br />
<br />
auto bond0<br />
iface bond0 inet manual<br />
bond_slaves eth0 eth1<br />
bond_mode 802.3ad<br />
bond_miimon 100<br />
bond_xmit_hash_policy layer2+3<br />
post-up ip link set dev bond0 mtu 9000<br />
<br />
iface bond0.1 inet manual<br />
<br />
auto br1<br />
iface br1 inet static<br />
address 192.168.1.196<br />
netmask 255.255.255.0<br />
gateway 192.168.1.1<br />
bridge_ports bond0.1<br />
bridge_stp off<br />
bridge_fd 0.0<br />
post-up ip link set dev bond0.1 mtu 1500<br />
<br />
iface bond0.10 inet manual<br />
<br />
auto br10<br />
iface br10 inet static<br />
address 192.168.10.1<br />
netmask 255.255.255.0<br />
bridge_ports bond0.10<br />
bridge_stp off<br />
bridge_fd 0.0<br />
</pre><br />
<br />
== Example with two interfaces on the same adapter. One with vlan and one without ==<br />
<br />
Since linux doesn't allow multiple default gateways we need to use a second routing table using iproute2<br />
<br />
{{Cmd|apk add iproute2}}<br />
<br />
Then we'll add two new routig tables to the config file. One for each network<br />
<br />
{{Cmd|echo "1 rt1" >> /etc/iproute2/rt_tables;echo "2 rt2" >> /etc/iproute2/rt_tables;}}<br />
<br />
Now we need to edit /etc/network/interfaces<br />
<br />
<pre><br />
auto lo<br />
iface lo inet loopback<br />
<br />
# the native interface without a vlan (also called untagged)<br />
<br />
auto eth0<br />
iface eth0 inet static<br />
address 192.168.1.100<br />
netmask 255.255.255.0<br />
gateway 192.168.1.1<br />
post-up ip route add 192.168.1.0/24 dev eth0 src 192.168.1.100 table rt1<br />
post-up ip route add default via 192.168.1.1 dev eth5 table rt1 # the actual gateway for this interface<br />
post-up ip rule add from 192.168.1.100/32 table rt1<br />
post-up ip rule add to 192.168.1.100/32 table rt1<br />
<br />
# second interface with the vlan tag 5<br />
auto eth0.5<br />
iface eth0.5 inet static<br />
address 192.168.5.100<br />
netmask 255.255.255.0<br />
post-up ip route add 192.168.5.0/24 dev eth0.5 src 192.168.5.100 table rt2<br />
post-up ip route add default via 192.168.5.1 dev eth0.5 table rt2 # the actual gateway for this interface<br />
post-up ip rule add from 192.168.5.100/32 table rt2<br />
post-up ip rule add to 192.168.5.100/32 table rt2<br />
</pre><br />
<br />
Note that if you want to add a third interface this way, you'll have to add another routing table<br />
<br />
[[Category:Networking]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=VLAN&diff=21119VLAN2021-12-28T18:27:28Z<p>Geek at: added info on using multiple gateways</p>
<hr />
<div>This article shows how to configure a network interface as an IEEE 802.1q VLAN trunk.<br />
<br />
__TOC__<br />
<br />
{{Note| Alpine Linux v2.4 or later is required}}<br />
==Installation==<br />
First, install the ''vlan'' package. This will give you support for vlans in the ''/etc/network/interfaces'' file.<br />
{{Cmd|apk add vlan}}<br />
<br />
==Configuration==<br />
Edit the ''/etc/network/interfaces'' file:<br />
<pre><br />
auto eth0.8<br />
iface eth0.8 inet static<br />
address 192.168.0.2<br />
netmask 255.255.255.0<br />
gateway 192.168.0.1<br />
</pre><br />
With the ''vlan'' package installed, ifup will find the trailing .8 in eth0.8 and will create a vlan interface with vid 8 over eth0.<br />
<br />
Alternativly with vlan8 over eth0:<br />
<pre><br />
auto vlan8<br />
iface vlan8 inet static<br />
address 192.168.0.2<br />
netmask 255.255.255.0<br />
gateway 192.168.0.1<br />
vlan-raw-device eth0<br />
</pre><br />
<br />
A static ip address was used in the examples shown above, but dhcp can be used as well.<br />
<br />
== Example with bridges associated with VLANs over bonding with differing MTUs on the various VLANs ==<br />
This serves as an example of some of the more complicated networking possible. Particularly, this would work well for a hypervisor attached to a dedicated storage VLAN. Less complicated implementations can be achieved by merely removing the non-applicable parts.<br />
<pre><br />
auto lo<br />
iface lo inet loopback<br />
<br />
auto bond0<br />
iface bond0 inet manual<br />
bond_slaves eth0 eth1<br />
bond_mode 802.3ad<br />
bond_miimon 100<br />
bond_xmit_hash_policy layer2+3<br />
post-up ip link set dev bond0 mtu 9000<br />
<br />
iface bond0.1 inet manual<br />
<br />
auto br1<br />
iface br1 inet static<br />
address 192.168.1.196<br />
netmask 255.255.255.0<br />
gateway 192.168.1.1<br />
bridge_ports bond0.1<br />
bridge_stp off<br />
bridge_fd 0.0<br />
post-up ip link set dev bond0.1 mtu 1500<br />
<br />
iface bond0.10 inet manual<br />
<br />
auto br10<br />
iface br10 inet static<br />
address 192.168.10.1<br />
netmask 255.255.255.0<br />
bridge_ports bond0.10<br />
bridge_stp off<br />
bridge_fd 0.0<br />
</pre><br />
<br />
== Example with two interfaces on the same adapter. One with vlan and one without ==<br />
<br />
Since linux doesn't allow multiple default gateways we need to use a second routing table using iproute2<br />
<br />
{{Cmd|apk add iproute2}}<br />
<br />
Then we'll add a new routig table to the config file<br />
<br />
{{Cmd|echo "1 rt2" >> /etc/iproute2/rt_tables}}<br />
<br />
Only thing that's left is to<br />
<br />
<pre><br />
auto lo<br />
iface lo inet loopback<br />
<br />
# the native interface without a vlan (also called untagged)<br />
<br />
auto eth0<br />
iface eth0 inet static<br />
address 192.168.1.100<br />
netmask 255.255.255.0<br />
gateway 192.168.1.1<br />
<br />
# second interface with the vlan tag 5<br />
# note that we're adding the gateway through the post-up commands<br />
auto eth0.5<br />
iface eth0.5 inet static<br />
address 192.168.5.100<br />
netmask 255.255.255.0<br />
post-up ip route add 192.168.5.0/24 dev eth0.5 src 192.168.5.100 table rt2<br />
post-up ip route add default via 192.168.5.1 dev eth0.5 table rt2 # the actual gateway for this interface<br />
post-up ip rule add from 192.168.5.100/32 table rt2<br />
post-up ip rule add to 192.168.5.100/32 table rt2<br />
</pre><br />
<br />
Note that if you want to add a third interface this way, you'll have to add another routing table<br />
<br />
[[Category:Networking]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=KVM&diff=19616KVM2021-06-21T18:43:52Z<p>Geek at: added important info on iommu groups</p>
<hr />
<div>[https://www.linux-kvm.org/page/Main_Page KVM] is an free and open source virtualization solution in a kernel module. Although it is often simply referred to as KVM, the actual hypervisor is [https://www.qemu.org QEMU]. QEMU runs from user-space, but can integrate with KVM, providing better performance by leveraging the hardware from kernel-space. QEMU can virtualize x86, PowerPC, and S390 guests, amongst others. [https://libvirt.org Libvirt] is a management framework that integrates with QEMU/KVM, [https://wiki.alpinelinux.org/wiki/LXC LXC], [https://wiki.alpinelinux.org/wiki/Xen_Dom0 Xen] and others.<br />
<br />
== Installation ==<br />
The following commands provide '''libvirt''' as well as '''QEMU with emulation for x86_64''' and '''qemu-img''', a necessary component for using various disk formats such as qcow2. Without qemu-img, only raw disks are available. It can also convert images between several formats like vhdx and vmdk. It also provides the metapackage '''qemu-modules''', which provides subpackages needed for special features. In versions of Alpine before 3.13.0 these features were covered by '''QEMU with emulation for x86_64'''.<br />
{{Cmd|<nowiki># apk add libvirt-daemon qemu-img qemu-system-x86_64 qemu-modules<br />
# rc-update add libvirtd</nowiki>}}<br />
<br />
== Networking ==<br />
By default, libvirt uses NAT for VM connectivity. If you want to use the default configuration, you need to load the tun module.<br />
{{Cmd|# modprobe tun}}<br />
<br />
If you prefer bridging a guest over your Ethernet interface, you need to make a [https://wiki.alpinelinux.org/wiki/Bridge#Configuration_file bridge].<br />
<br />
It's quite common to use bridges with KVM environments. But when IPv6 is used, Alpine will assign itself a link-local address as well as an SLAAC address in case there's a router sending Router Advertisements. You don't want this because you don't want to have the KVM host an IP address in every network it serves to guests. Unfortunately IPv6 can not just be disabled for the bridge via a sysctl configuration file, because the bridge might not be up when the sysctl config is applied during boot. What works is to put a post-up hook into the /etc/network/interfaces file like this:<br />
auto brlan<br />
iface brlan inet manual<br />
bridge-ports eth1.5<br />
bridge-stp 0<br />
post-up ip -6 a flush dev brlan; sysctl -w net.ipv6.conf.brlan.disable_ipv6=1<br />
<br />
== Management ==<br />
For (non-root) management, you will need to add your user to the libvirt group.<br />
{{Cmd|# addgroup user libvirt}}<br />
<br />
You can use libvirt's virsh on the CLI. It can execute commands as well as run as an interactive shell. Read its manual page and/or use the "help" command for more info. Some basic commands are:<br />
<br />
{{Cmd|<nowiki>virsh help<br />
virsh list --all<br />
virsh start $domain<br />
virsh shutdown $domain</nowiki><br />
}}<br />
<br />
The libvirt project provides a GUI for managing hosts, called virt-manager. It handles local systems as well as remote ones via SSH.<br />
{{Cmd|<nowiki># apk add dbus polkit virt-manager terminus-font<br />
# rc-update add dbus</nowiki>}}<br />
<br />
In order to use libvirtd to remotely control KVM over ssh PolicyKit needs a .pkla informing it that this is allowed.<br />
Write the following file to /etc/polkit-1/localauthority/50-local.d/50-libvirt-ssh-remote-access-policy.pkla<br />
{{Cmd|<nowiki>[Remote libvirt SSH access]<br />
Identity=unix-group:libvirt<br />
Action=org.libvirt.unix.manage<br />
ResultAny=yes<br />
ResultInactive=yes<br />
ResultActive=yes</nowiki><br />
}}<br />
<br />
== Guest lifecycle management ==<br />
The libvirt-guests service (available from Alpine 3.13.5) allows running guests to be automatically suspended or shut down when the host is shut down or rebooted.<br />
<br />
The service is configured in /etc/conf.d/libvirt-guests. Enable the service with {{Cmd|# rc-update add libvirt-guests}}<br />
<br />
== vfio ==<br />
<br />
VFIO is more flexible way to do PCI passthrough. Let's suppose you want to use following ethernet card as PCI device in a VM.<br />
<br />
# lspci | grep 02:00.0<br />
02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)<br />
# lspci -n -s 02:00.0<br />
02:00.0 0200: 8086:10c9 (rev 01)<br />
<br />
First, create ''/etc/mkinitfs/features.d/vfio.modules'' with the following content, so mkinitfs includes the VFIO modules in the initramfs.<br />
<br />
kernel/drivers/vfio/vfio.ko<br />
kernel/drivers/vfio/vfio_virqfd.ko<br />
kernel/drivers/vfio/vfio_iommu_type1.ko<br />
kernel/drivers/vfio/pci/vfio-pci.ko<br />
<br />
Add ''vfio'' the the list of features in ''/etc/mkinitfs/mkinitfs.conf''.<br />
<br />
Modify following file to instruct ''mkinitfs'' to load following module with the options and rebuild kernel ramdisk.<br />
<br />
# cat /etc/modprobe.d/vfio.conf <<EOF<br />
options vfio-pci ids=8086:10c9<br />
options vfio_iommu_type1 allow_unsafe_interrupts=1<br />
softdep igb pre: vfio-pci<br />
EOF<br />
# mkinitfs<br />
<br />
Now modify GRUB, include ''intel_iommu=o iommu=pt'' for Intel platform (AMD uses ''amd_iommu=on'') and add the VFIO modules.<br />
<br />
# grep ^GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub<br />
GRUB_CMDLINE_LINUX_DEFAULT="modules=sd-mod,usb-storage,ext4,raid1,vfio,vfio-pci,vfio_iommu_type1,vfio_virqfd nomodeset rootfstype=ext4 intel_iommu=on iommu=pt console=ttyS0,115200"<br />
# grub-mkconfig -o /boot/grub/grub.cfg<br />
<br />
Reboot and check dmesg.<br />
<br />
# grep -i -e DMAR -e IOMMU /var/log/dmesg<br />
[ 0.343795] DMAR: Host address width 36<br />
[ 0.343797] DMAR: DRHD base: 0x000000fed90000 flags: 0x1<br />
[ 0.343804] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c90780106f0462 ecap f020e3<br />
[ 0.343806] DMAR: RMRR base: 0x000000000ed000 end: 0x000000000effff<br />
[ 0.343807] DMAR: RMRR base: 0x000000bf7ed000 end: 0x000000bf7fffff<br />
[ 0.553830] iommu: Default domain type: Passthrough (set via kernel command line)<br />
[ 0.902477] DMAR: No ATSR found<br />
[ 0.902563] DMAR: dmar0: Using Queued invalidation<br />
...<br />
[ 0.903256] pci 0000:02:00.0: Adding to iommu group 12<br />
...<br />
[ 0.903768] DMAR: Intel(R) Virtualization Technology for Directed I/O<br />
<br />
If you do not run libvirt VMs under ''root'' (''egrep '^#*user' /etc/libvirt/qemu.conf''), then you must have correct permission on ''/dev/vfio/<iommu_group>'', eg. ''/dev/vfio/12''. You have to tune ''/etc/mdev.conf'' or UDEV rules. Also note that if there are multiple PCI devices in the same iommu group, you always have to add all of them to the VM otherwise you'll get an error message like "Please ensure all devices within the iommu_group are bound to their vfio bus driver"<br />
<br />
# virsh dumpxml vm01 | xmllint --xpath '//*/hostdev' -<br />
<hostdev mode="subsystem" type="pci" managed="yes"><br />
<driver name="vfio"/><br />
<source><br />
<address domain="0x0000" bus="0x02" slot="0x00" function="0x0"/><br />
</source><br />
<alias name="hostdev0"/><br />
<address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/><br />
</hostdev><br />
<hostdev mode="subsystem" type="pci" managed="yes"><br />
<driver name="vfio"/><br />
<source><br />
<address domain="0x0000" bus="0x02" slot="0x00" function="0x1"/><br />
</source><br />
<alias name="hostdev1"/><br />
<address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/><br />
</hostdev><br />
<br />
If you directly use QEMU without libvirt and are trying to pass a GPU to your VM, you may get a "VFIO_MAP_DMA failed: Out of memory" error, when starting the VM as a non-root user. One way to fix it is to install the ''shadow'' package, and increase the amount of memory the user can lock via the ''/etc/security/limits.conf'' file:<br />
{{Cmd|<nowiki># apk add shadow<br />
# echo "youruser soft memlock RAMamount \<br />
youruser hard memlock RAMamount" >> /etc/security/limits.conf<br />
# reboot</nowiki>}}<br />
<br />
Replace "youruser" with the user you wish to run the VM as, and "RAMamount" with how much RAM your VM will need (in KB). The exact amount may throw the same error in the end, so you probably want to increase this value by a few dozen MB (typically +40).<br />
<br />
A lot of info at [https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF].<br />
<br />
[[Category:Virtualization]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=KVM&diff=19615KVM2021-06-21T15:29:09Z<p>Geek at: Without terminus-font package, the virt-manager window has squares instead of text</p>
<hr />
<div>[https://www.linux-kvm.org/page/Main_Page KVM] is an free and open source virtualization solution in a kernel module. Although it is often simply referred to as KVM, the actual hypervisor is [https://www.qemu.org QEMU]. QEMU runs from user-space, but can integrate with KVM, providing better performance by leveraging the hardware from kernel-space. QEMU can virtualize x86, PowerPC, and S390 guests, amongst others. [https://libvirt.org Libvirt] is a management framework that integrates with QEMU/KVM, [https://wiki.alpinelinux.org/wiki/LXC LXC], [https://wiki.alpinelinux.org/wiki/Xen_Dom0 Xen] and others.<br />
<br />
== Installation ==<br />
The following commands provide '''libvirt''' as well as '''QEMU with emulation for x86_64''' and '''qemu-img''', a necessary component for using various disk formats such as qcow2. Without qemu-img, only raw disks are available. It can also convert images between several formats like vhdx and vmdk. It also provides the metapackage '''qemu-modules''', which provides subpackages needed for special features. In versions of Alpine before 3.13.0 these features were covered by '''QEMU with emulation for x86_64'''.<br />
{{Cmd|<nowiki># apk add libvirt-daemon qemu-img qemu-system-x86_64 qemu-modules<br />
# rc-update add libvirtd</nowiki>}}<br />
<br />
== Networking ==<br />
By default, libvirt uses NAT for VM connectivity. If you want to use the default configuration, you need to load the tun module.<br />
{{Cmd|# modprobe tun}}<br />
<br />
If you prefer bridging a guest over your Ethernet interface, you need to make a [https://wiki.alpinelinux.org/wiki/Bridge#Configuration_file bridge].<br />
<br />
It's quite common to use bridges with KVM environments. But when IPv6 is used, Alpine will assign itself a link-local address as well as an SLAAC address in case there's a router sending Router Advertisements. You don't want this because you don't want to have the KVM host an IP address in every network it serves to guests. Unfortunately IPv6 can not just be disabled for the bridge via a sysctl configuration file, because the bridge might not be up when the sysctl config is applied during boot. What works is to put a post-up hook into the /etc/network/interfaces file like this:<br />
auto brlan<br />
iface brlan inet manual<br />
bridge-ports eth1.5<br />
bridge-stp 0<br />
post-up ip -6 a flush dev brlan; sysctl -w net.ipv6.conf.brlan.disable_ipv6=1<br />
<br />
== Management ==<br />
For (non-root) management, you will need to add your user to the libvirt group.<br />
{{Cmd|# addgroup user libvirt}}<br />
<br />
You can use libvirt's virsh on the CLI. It can execute commands as well as run as an interactive shell. Read its manual page and/or use the "help" command for more info. Some basic commands are:<br />
<br />
{{Cmd|<nowiki>virsh help<br />
virsh list --all<br />
virsh start $domain<br />
virsh shutdown $domain</nowiki><br />
}}<br />
<br />
The libvirt project provides a GUI for managing hosts, called virt-manager. It handles local systems as well as remote ones via SSH.<br />
{{Cmd|<nowiki># apk add dbus polkit virt-manager terminus-font<br />
# rc-update add dbus</nowiki>}}<br />
<br />
In order to use libvirtd to remotely control KVM over ssh PolicyKit needs a .pkla informing it that this is allowed.<br />
Write the following file to /etc/polkit-1/localauthority/50-local.d/50-libvirt-ssh-remote-access-policy.pkla<br />
{{Cmd|<nowiki>[Remote libvirt SSH access]<br />
Identity=unix-group:libvirt<br />
Action=org.libvirt.unix.manage<br />
ResultAny=yes<br />
ResultInactive=yes<br />
ResultActive=yes</nowiki><br />
}}<br />
<br />
== Guest lifecycle management ==<br />
The libvirt-guests service (available from Alpine 3.13.5) allows running guests to be automatically suspended or shut down when the host is shut down or rebooted.<br />
<br />
The service is configured in /etc/conf.d/libvirt-guests. Enable the service with {{Cmd|# rc-update add libvirt-guests}}<br />
<br />
== vfio ==<br />
<br />
VFIO is more flexible way to do PCI passthrough. Let's suppose you want to use following ethernet card as PCI device in a VM.<br />
<br />
# lspci | grep 02:00.0<br />
02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)<br />
# lspci -n -s 02:00.0<br />
02:00.0 0200: 8086:10c9 (rev 01)<br />
<br />
First, create ''/etc/mkinitfs/features.d/vfio.modules'' with the following content, so mkinitfs includes the VFIO modules in the initramfs.<br />
<br />
kernel/drivers/vfio/vfio.ko<br />
kernel/drivers/vfio/vfio_virqfd.ko<br />
kernel/drivers/vfio/vfio_iommu_type1.ko<br />
kernel/drivers/vfio/pci/vfio-pci.ko<br />
<br />
Add ''vfio'' the the list of features in ''/etc/mkinitfs/mkinitfs.conf''.<br />
<br />
Modify following file to instruct ''mkinitfs'' to load following module with the options and rebuild kernel ramdisk.<br />
<br />
# cat /etc/modprobe.d/vfio.conf <<EOF<br />
options vfio-pci ids=8086:10c9<br />
options vfio_iommu_type1 allow_unsafe_interrupts=1<br />
softdep igb pre: vfio-pci<br />
EOF<br />
# mkinitfs<br />
<br />
Now modify GRUB, include ''intel_iommu=o iommu=pt'' for Intel platform (AMD uses ''amd_iommu=on'') and add the VFIO modules.<br />
<br />
# grep ^GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub<br />
GRUB_CMDLINE_LINUX_DEFAULT="modules=sd-mod,usb-storage,ext4,raid1,vfio,vfio-pci,vfio_iommu_type1,vfio_virqfd nomodeset rootfstype=ext4 intel_iommu=on iommu=pt console=ttyS0,115200"<br />
# grub-mkconfig -o /boot/grub/grub.cfg<br />
<br />
Reboot and check dmesg.<br />
<br />
# grep -i -e DMAR -e IOMMU /var/log/dmesg<br />
[ 0.343795] DMAR: Host address width 36<br />
[ 0.343797] DMAR: DRHD base: 0x000000fed90000 flags: 0x1<br />
[ 0.343804] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c90780106f0462 ecap f020e3<br />
[ 0.343806] DMAR: RMRR base: 0x000000000ed000 end: 0x000000000effff<br />
[ 0.343807] DMAR: RMRR base: 0x000000bf7ed000 end: 0x000000bf7fffff<br />
[ 0.553830] iommu: Default domain type: Passthrough (set via kernel command line)<br />
[ 0.902477] DMAR: No ATSR found<br />
[ 0.902563] DMAR: dmar0: Using Queued invalidation<br />
...<br />
[ 0.903256] pci 0000:02:00.0: Adding to iommu group 12<br />
...<br />
[ 0.903768] DMAR: Intel(R) Virtualization Technology for Directed I/O<br />
<br />
If you do not run libvirt VMs under ''root'' (''egrep '^#*user' /etc/libvirt/qemu.conf''), then you must have correct permission on ''/dev/vfio/<iommu_group>'', eg. ''/dev/vfio/12''. You have to tune ''/etc/mdev.conf'' or UDEV rules.<br />
<br />
# virsh dumpxml vm01 | xmllint --xpath '//*/hostdev' -<br />
<hostdev mode="subsystem" type="pci" managed="yes"><br />
<driver name="vfio"/><br />
<source><br />
<address domain="0x0000" bus="0x02" slot="0x00" function="0x0"/><br />
</source><br />
<alias name="hostdev0"/><br />
<address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/><br />
</hostdev><br />
<hostdev mode="subsystem" type="pci" managed="yes"><br />
<driver name="vfio"/><br />
<source><br />
<address domain="0x0000" bus="0x02" slot="0x00" function="0x1"/><br />
</source><br />
<alias name="hostdev1"/><br />
<address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/><br />
</hostdev><br />
<br />
If you directly use QEMU without libvirt and are trying to pass a GPU to your VM, you may get a "VFIO_MAP_DMA failed: Out of memory" error, when starting the VM as a non-root user. One way to fix it is to install the ''shadow'' package, and increase the amount of memory the user can lock via the ''/etc/security/limits.conf'' file:<br />
{{Cmd|<nowiki># apk add shadow<br />
# echo "youruser soft memlock RAMamount \<br />
youruser hard memlock RAMamount" >> /etc/security/limits.conf<br />
# reboot</nowiki>}}<br />
<br />
Replace "youruser" with the user you wish to run the VM as, and "RAMamount" with how much RAM your VM will need (in KB). The exact amount may throw the same error in the end, so you probably want to increase this value by a few dozen MB (typically +40).<br />
<br />
A lot of info at [https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF].<br />
<br />
[[Category:Virtualization]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=Configure_a_Wireguard_interface_(wg)&diff=16974Configure a Wireguard interface (wg)2020-03-02T08:38:35Z<p>Geek at: added wg-tools usage for wg-quick and basic setup</p>
<hr />
<div>Wireguard is a very promising VPN technology and available since Alpine 3.10 in the community repository.<br />
<br />
apk add wireguard-lts (or wireguard-virt)<br />
<br />
The official documents from wireguard will show examples of how to setup an interface with the use of wg-quick.<br />
In this howto we are not going to use this utility but are going to use plain wg command and busybox ifupdown.<br />
<br />
apk add wireguard-tools-wg<br />
<br />
Now that you have all the tools installed we can setup the interface.<br />
The setup of your interface config is out of the scope of this document, you should consult the [https://git.zx2c4.com/WireGuard/about/src/tools/man/wg.8 manual page of wg].<br />
<br />
After you have finished setting up your wgX interface config you can add it to your /etc/network/interfaces:<br />
<br />
auto wg0<br />
iface wg0 inet static<br />
address x.x.x.x<br />
netmask 255.255.255.0<br />
pre-up ip link add dev wg0 type wireguard<br />
pre-up wg setconf wg0 /etc/wireguard/wg0.conf<br />
post-up ip route add x.x.x.x/24 dev wg0<br />
post-down ip link delete dev wg0<br />
<br />
This config will do:<br />
<br />
* bring the wireguard interface up<br />
* assign a config to this interface (which you have previously created)<br />
* setup the interface address and netmask<br />
* add the route ones the interface is up<br />
* remove the interface when it goes down<br />
<br />
To start the interface and stop it you can execute:<br />
<br />
ifup wg0<br />
ifdown wg0<br />
<br />
If <code>ifup wg0</code> fails silently, verify that the <code>bash</code> package is installed.<br />
<br />
== Bringing up an interface using wg-tools ==<br />
First we need to create a private and public key<br />
<br />
wg genkey | tee privatekey | wg pubkey > publickey<br />
<br />
Then we create a new config file <code>/etc/wireguard/wg0.conf</code><br />
<br />
[Interface]<br />
Address = 10.123.0.1/24<br />
ListenPort = 45340<br />
PrivateKey = SG1nXk2+kAAKnMkL5aX3NSFPaGjf9SQI/wWwFj9l9U4= # the key from the previously generated privatekey file<br />
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE;iptables -A FORWARD -o %i -j ACCEPT<br />
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE;iptables -D FORWARD -o %i -j ACCEPT<br />
<br />
The PostUp and PostDown steps are there to ensure the interface wg0 will accept and forward traffic to eth0. The postrouting and forward to %i is not required but it will enable "VPN mode" where users can access the internet through this server if desired.<br />
<br />
To bring up the new interface we can just use <code>wg-quick up wg0</code> and to bring it down we can use <code>wg-quick down wg0</code> which will clean up the interface and remove the ip table rules<br />
<br />
<br />
== Running with modloop ==<br />
If you are running from a RAM disk you can't modify the modloop.<br />
<br />
You can get around it by unpacking the modloop, mount the unpacked modules folder and then installing wireguard.<br />
<br />
#!/bin/sh<br />
apk add squashfs-tools # install squashfs tools to unpack modloop<br />
unsquashfs -d /root/squash /lib/modloop-lts # unpack modloop to root dir<br />
umount /.modloop # unmount existing modloop<br />
mount /root/squash/ /.modloop/ # mount unpacked modloop<br />
apk del wireguard-lts # uninstall previous wireguard install<br />
apk add wireguard-lts<br />
apk add wireguard-tools<br />
<br />
Now you could repack the squash filesystem or put this script in the /etc/local.d/ path so it runs on boot.<br />
<br />
[[Category:Networking]]</div>Geek athttps://wiki.alpinelinux.org/w/index.php?title=Configure_a_Wireguard_interface_(wg)&diff=16970Configure a Wireguard interface (wg)2020-03-02T00:23:12Z<p>Geek at: Added info on how to make it work using a RAM disk</p>
<hr />
<div>Wireguard is a very promising VPN technology and available since Alpine 3.10 in the community repository.<br />
<br />
apk add wireguard-lts (or wireguard-virt)<br />
<br />
The official documents from wireguard will show examples of how to setup an interface with the use of wg-quick.<br />
In this howto we are not going to use this utility but are going to use plain wg command and busybox ifupdown.<br />
<br />
apk add wireguard-tools-wg<br />
<br />
Now that you have all the tools installed we can setup the interface.<br />
The setup of your interface config is out of the scope of this document, you should consult the [https://git.zx2c4.com/WireGuard/about/src/tools/man/wg.8 manual page of wg].<br />
<br />
After you have finished setting up your wgX interface config you can add it to your /etc/network/interfaces:<br />
<br />
auto wg0<br />
iface wg0 inet static<br />
address x.x.x.x<br />
netmask 255.255.255.0<br />
pre-up ip link add dev wg0 type wireguard<br />
pre-up wg setconf wg0 /etc/wireguard/wg0.conf<br />
post-up ip route add x.x.x.x/24 dev wg0<br />
post-down ip link delete dev wg0<br />
<br />
This config will do:<br />
<br />
* bring the wireguard interface up<br />
* assign a config to this interface (which you have previously created)<br />
* setup the interface address and netmask<br />
* add the route ones the interface is up<br />
* remove the interface when it goes down<br />
<br />
To start the interface and stop it you can execute:<br />
<br />
ifup wg0<br />
ifdown wg0<br />
<br />
If <code>ifup wg0</code> fails silently, verify that the <code>bash</code> package is installed.<br />
<br />
== Running with modloop ==<br />
If you are running from a RAM disk you can't modify the modloop.<br />
<br />
You can get around it by unpacking the modloop, mount the unpacked modules folder and then installing wireguard.<br />
<br />
#!/bin/sh<br />
apk add squashfs-tools # install squashfs tools to unpack modloop<br />
unsquashfs -d /root/squash /lib/modloop-lts # unpack modloop to root dir<br />
umount /.modloop # unmount existing modloop<br />
mount /root/squash/ /.modloop/ # mount unpacked modloop<br />
apk del wireguard-lts # uninstall previous wireguard install<br />
apk add wireguard-lts<br />
apk add wireguard-tools<br />
<br />
Now you could repack the squash filesystem or put this script in the /etc/local.d/ path so it runs on boot.<br />
<br />
[[Category:Networking]]</div>Geek at