Alpine Linux - User contributions [en]

Release Notes for Alpine 3.22.0

2025-05-12T15:20:34Z

Mcrute:

As always, make sure to read [[Upgrading Alpine to a new major release]] when upgrading to a new release.

If you experience any issues with the upgrade, please let us know and file an issue in our repositories.

== Important changes ==

== Significant changes ==

=== apk-tools ===

This release is the last release using apk-tools v2.14. The next Alpine v3.23 will include apk-tools v3. The packages and index format are still using the legacy v2 format.

There are a number of new changes in the new apk-tools release. To test apk-tools v3, [https://wiki.alpinelinux.org/wiki/Upgrading_Alpine_Linux_to_a_new_release_branch#Upgrading_to_Edge upgrade to Alpine edge, enable the testing repository] and install {{pkg|apk-tools3}}.

{{Note|TODO: if edge upgrades to apk-tools 3, remove the testing repo and apk-tools3 package mention}}

=== KDE Plasma ===

The X11 session for KDE Plasma has been removed. If you had {{pkg|plasma-workspace-x11|branch=v3.21|arch=}} installed make sure to remove it from {{path|/etc/apk/world}}. Wayland is the only available option now.

=== SDL ===

{{pkg|sdl3}} and {{pkg|sdl2-compat}} were moved to the community repository and are now the default SDL provider. To force the installation of {{pkg|sdl2}}, use <code>apk add sdl2</code>.

=== Adwaita Fonts ===

As a result of [https://release.gnome.org/48/#new-fonts GNOME's switch] to {{pkg|adwaita-fonts}}, the default font for GTK-based applications (e.g. {{pkg|vte3}} or {{pkg|libadwaita}} Applications) was changed. If you want to revert or override the fonts, you can use:

<pre>
$ apk add font-cantarell font-adobe-source-code-pro
$ gsettings set org.gnome.desktop.interface font-name 'Cantarell 11'
$ gsettings set org.gnome.desktop.interface monospace-font-name 'Source Code Pro 11'
</pre>

{{pkg|adwaita-fonts}} is pulled in as a dependency of some packages since it broke the default behaviour for various applications.

=== GeoClue ===

Since Mozilla Location Services (MLS), which were used as the geolocation service in geoclue, [https://discourse.mozilla.org/t/retiring-the-mozilla-location-service/128693 has been retired by Mozilla], Alpine switched to [https://beacondb.net/ beaconDB] as the default geolocation provider. {{MR|76764}}

=== BIRD Routing Daemon ===

The {{pkg|bird|repo=community}} routing daemon package has been upgrade to version 3 which introduces a new multi-threaded architecture. A side effect of this change is [https://trubka.network.cz/pipermail/bird-users/2024-December/017973.html significantly increased memory consumption] of the daemon. This update is configuration-compatible with BIRD v2 but there are some behavioral backwards incompatibilities. Users should read the [https://gitlab.nic.cz/labs/bird/-/blob/stable-v3.0/doc/migration-bird3.md migration guide]. Notably the format of <tt>show route all</tt> has changed as has the format of the logs, which may impact scripts and automation that consume those outputs. Users who require BIRD v2 should install the {{pkg|bird2|repo=community}} package instead.

== Note-worthy updates ==

As always, many packages were upgraded. Make sure to read the individual release notes of the projects you use.

* Linux TODO
* busybox TODO
* GCC TODO
* LLVM TODO
* Go TODO
* Rust TODO
* PHP TODO
* GNOME TODO
* KDE Plasma TODO
* LXQt TODO
* Qt TODO
* wlroots TODO
* ISC BIND 9.20.TODO
* BIRD 3.1.TODO

=== LLVM ===

LLVM 20 is available as {{pkg|llvm}}/{{pkg|clang}} (or {{pkg|llvm20}}/{{pkg|clang20}} explictly) additionally to LLVM 19, 18, 17, 16 and 15. ({{MR|80901}}, {{MR|82502}})

LLD is now also splitted per version and is available as {{pkg|lld20}} (default for {{pkg|lld}}), {{pkg|lld19}} and {{pkg|lld18}}. ({{MR|81774}})

== Significant removals ==

=== LXD ===

LXD was moved to the testing repository and is therefore not available on Alpine Linux 3.22. It was deprecated in favor of {{pkg|incus}} and {{pkg|incus-feature}} (feature branch).

Take a look at the [https://linuxcontainers.org/incus/docs/main/howto/server_migrate_lxd/ Migration Guide] on how to migrate from LXD to Incus using the {{pkg|incus-conversion}} or {{pkg|incus-feature-conversion}} package.

=== Qt 5 ===

On 26th of May 2025, Qt 5 will be unmaintained upstream. Therefore we're removing unused libraries from our repositories and are migrating the remaining packages to Qt 6.

Applications that still use Qt 5 libraries won't be removed, but libraries without consumers in our repository will.

For more information, see Issue {{Issue|17114}}.

Release Notes for Alpine 3.22.0

2025-05-12T15:19:03Z

Mcrute:

As always, make sure to read [[Upgrading Alpine to a new major release]] when upgrading to a new release.

If you experience any issues with the upgrade, please let us know and file an issue in our repositories.

== Important changes ==

== Significant changes ==

=== apk-tools ===

This release is the last release using apk-tools v2.14. The next Alpine v3.23 will include apk-tools v3. The packages and index format are still using the legacy v2 format.

There are a number of new changes in the new apk-tools release. To test apk-tools v3, [https://wiki.alpinelinux.org/wiki/Upgrading_Alpine_Linux_to_a_new_release_branch#Upgrading_to_Edge upgrade to Alpine edge, enable the testing repository] and install {{pkg|apk-tools3}}.

{{Note|TODO: if edge upgrades to apk-tools 3, remove the testing repo and apk-tools3 package mention}}

=== KDE Plasma ===

The X11 session for KDE Plasma has been removed. If you had {{pkg|plasma-workspace-x11|branch=v3.21|arch=}} installed make sure to remove it from {{path|/etc/apk/world}}. Wayland is the only available option now.

=== SDL ===

{{pkg|sdl3}} and {{pkg|sdl2-compat}} were moved to the community repository and are now the default SDL provider. To force the installation of {{pkg|sdl2}}, use <code>apk add sdl2</code>.

=== Adwaita Fonts ===

As a result of [https://release.gnome.org/48/#new-fonts GNOME's switch] to {{pkg|adwaita-fonts}}, the default font for GTK-based applications (e.g. {{pkg|vte3}} or {{pkg|libadwaita}} Applications) was changed. If you want to revert or override the fonts, you can use:

<pre>
$ apk add font-cantarell font-adobe-source-code-pro
$ gsettings set org.gnome.desktop.interface font-name 'Cantarell 11'
$ gsettings set org.gnome.desktop.interface monospace-font-name 'Source Code Pro 11'
</pre>

{{pkg|adwaita-fonts}} is pulled in as a dependency of some packages since it broke the default behaviour for various applications.

=== GeoClue ===

Since Mozilla Location Services (MLS), which were used as the geolocation service in geoclue, [https://discourse.mozilla.org/t/retiring-the-mozilla-location-service/128693 has been retired by Mozilla], Alpine switched to [https://beacondb.net/ beaconDB] as the default geolocation provider. {{MR|76764}}

=== BIRD Routing Daemon ===

The {{pkg|bird|repo=community}} routing daemon package has been upgrade to version 3 which introduces a new multi-threaded architecture. A side effect of this change is [https://trubka.network.cz/pipermail/bird-users/2024-December/017973.html significantly increased memory consumption] of the daemon. This update is configuration-compatible with BIRD v2 but there are some behavioral backwards incompatibilities. Users should read the [https://gitlab.nic.cz/labs/bird/-/blob/stable-v3.0/doc/migration-bird3.md migration guide]. Notably the format of <tt>show route all</tt> has changed as has the format of the logs, which may impact scripts and automation that consume those outputs. Users who require BIRD v2 should install the community/bird2 package instead.

== Note-worthy updates ==

As always, many packages were upgraded. Make sure to read the individual release notes of the projects you use.

* Linux TODO
* busybox TODO
* GCC TODO
* LLVM TODO
* Go TODO
* Rust TODO
* PHP TODO
* GNOME TODO
* KDE Plasma TODO
* LXQt TODO
* Qt TODO
* wlroots TODO
* ISC BIND 9.20.TODO
* BIRD 3.1.TODO

=== LLVM ===

LLVM 20 is available as {{pkg|llvm}}/{{pkg|clang}} (or {{pkg|llvm20}}/{{pkg|clang20}} explictly) additionally to LLVM 19, 18, 17, 16 and 15. ({{MR|80901}}, {{MR|82502}})

LLD is now also splitted per version and is available as {{pkg|lld20}} (default for {{pkg|lld}}), {{pkg|lld19}} and {{pkg|lld18}}. ({{MR|81774}})

== Significant removals ==

=== LXD ===

LXD was moved to the testing repository and is therefore not available on Alpine Linux 3.22. It was deprecated in favor of {{pkg|incus}} and {{pkg|incus-feature}} (feature branch).

Take a look at the [https://linuxcontainers.org/incus/docs/main/howto/server_migrate_lxd/ Migration Guide] on how to migrate from LXD to Incus using the {{pkg|incus-conversion}} or {{pkg|incus-feature-conversion}} package.

=== Qt 5 ===

On 26th of May 2025, Qt 5 will be unmaintained upstream. Therefore we're removing unused libraries from our repositories and are migrating the remaining packages to Qt 6.

Applications that still use Qt 5 libraries won't be removed, but libraries without consumers in our repository will.

For more information, see Issue {{Issue|17114}}.

Release Notes for Alpine 3.22.0

2025-05-12T15:15:06Z

Mcrute: /* Significant changes */

As always, make sure to read [[Upgrading Alpine to a new major release]] when upgrading to a new release.

If you experience any issues with the upgrade, please let us know and file an issue in our repositories.

== Important changes ==

== Significant changes ==

=== apk-tools ===

This release is the last release using apk-tools v2.14. The next Alpine v3.23 will include apk-tools v3. The packages and index format are still using the legacy v2 format.

There are a number of new changes in the new apk-tools release. To test apk-tools v3, [https://wiki.alpinelinux.org/wiki/Upgrading_Alpine_Linux_to_a_new_release_branch#Upgrading_to_Edge upgrade to Alpine edge, enable the testing repository] and install {{pkg|apk-tools3}}.

{{Note|TODO: if edge upgrades to apk-tools 3, remove the testing repo and apk-tools3 package mention}}

=== KDE Plasma ===

The X11 session for KDE Plasma has been removed. If you had {{pkg|plasma-workspace-x11|branch=v3.21|arch=}} installed make sure to remove it from {{path|/etc/apk/world}}. Wayland is the only available option now.

=== SDL ===

{{pkg|sdl3}} and {{pkg|sdl2-compat}} were moved to the community repository and are now the default SDL provider. To force the installation of {{pkg|sdl2}}, use <code>apk add sdl2</code>.

=== Adwaita Fonts ===

As a result of [https://release.gnome.org/48/#new-fonts GNOME's switch] to {{pkg|adwaita-fonts}}, the default font for GTK-based applications (e.g. {{pkg|vte3}} or {{pkg|libadwaita}} Applications) was changed. If you want to revert or override the fonts, you can use:

<pre>
$ apk add font-cantarell font-adobe-source-code-pro
$ gsettings set org.gnome.desktop.interface font-name 'Cantarell 11'
$ gsettings set org.gnome.desktop.interface monospace-font-name 'Source Code Pro 11'
</pre>

{{pkg|adwaita-fonts}} is pulled in as a dependency of some packages since it broke the default behaviour for various applications.

=== GeoClue ===

Since Mozilla Location Services (MLS), which were used as the geolocation service in geoclue, [https://discourse.mozilla.org/t/retiring-the-mozilla-location-service/128693 has been retired by Mozilla], Alpine switched to [https://beacondb.net/ beaconDB] as the default geolocation provider. {{MR|76764}}

=== BIRD Routing Daemon ===

The BIRD routing daemon has been upgrade to version 3 which introduces a new multi-threaded architecture. A side effect of this change is [https://trubka.network.cz/pipermail/bird-users/2024-December/017973.html significantly increased memory consumption] of the daemon. This update is configuration-compatible with BIRD v2 but there are some behavioral backwards incompatibilities. Users should read the [https://gitlab.nic.cz/labs/bird/-/blob/stable-v3.0/doc/migration-bird3.md migration guide]. Notably the format of `show route all` has changed as has the format of the logs. Users who require BIRD v2 should install the community/bird2 package instead.

== Note-worthy updates ==

As always, many packages were upgraded. Make sure to read the individual release notes of the projects you use.

* Linux TODO
* busybox TODO
* GCC TODO
* LLVM TODO
* Go TODO
* Rust TODO
* PHP TODO
* GNOME TODO
* KDE Plasma TODO
* LXQt TODO
* Qt TODO
* wlroots TODO
* ISC BIND 9.20.TODO

=== LLVM ===

LLVM 20 is available as {{pkg|llvm}}/{{pkg|clang}} (or {{pkg|llvm20}}/{{pkg|clang20}} explictly) additionally to LLVM 19, 18, 17, 16 and 15. ({{MR|80901}}, {{MR|82502}})

LLD is now also splitted per version and is available as {{pkg|lld20}} (default for {{pkg|lld}}), {{pkg|lld19}} and {{pkg|lld18}}. ({{MR|81774}})

== Significant removals ==

=== LXD ===

LXD was moved to the testing repository and is therefore not available on Alpine Linux 3.22. It was deprecated in favor of {{pkg|incus}} and {{pkg|incus-feature}} (feature branch).

Take a look at the [https://linuxcontainers.org/incus/docs/main/howto/server_migrate_lxd/ Migration Guide] on how to migrate from LXD to Incus using the {{pkg|incus-conversion}} or {{pkg|incus-feature-conversion}} package.

=== Qt 5 ===

On 26th of May 2025, Qt 5 will be unmaintained upstream. Therefore we're removing unused libraries from our repositories and are migrating the remaining packages to Qt 6.

Applications that still use Qt 5 libraries won't be removed, but libraries without consumers in our repository will.

For more information, see Issue {{Issue|17114}}.

How to setup a Alpine Linux mirror

2023-01-13T18:14:58Z

Mcrute:

== Introduction ==
This document describes how to set up an Alpine Linux mirror and make it available via http and rsync.

We will:
* create the dir where we have the mirror
* set up a cron job to sync with master mirror every hour
* set up lighttpd for http access
* set up rsync so other mirrors can rsync from you

Make sure that you have enough disk space.

Current (2023-01-13) disk usage in GB:

{|class="wikitable"
!edge
!v3.0
!v3.1
!v3.2
!v3.3
!v3.4
!v3.5
!v3.6
!v3.7
!v3.8
!v3.9
!v3.10
!v3.11
!v3.12
!v3.13
!v3.14
!v3.15
!v3.16
!v3.17

!total
|-
|349
|17
|18
|15
|21
|25
|27
|45
|43
|60
|73
|92
|126
|148
|156
|174
|176
|184
|211

|'''1960'''
|}

Script used to calculate the size:

<pre>
#!/bin/sh

total=0
dest="$(mktemp -d)"

for dir in edge v3.0 v3.1 v3.2 v3.3 v3.4 v3.5 v3.6 v3.7 v3.8 v3.9 v3.10 v3.11 v3.12 v3.13 v3.14 v3.15 v3.16; do
old_total="$total"
src="rsync://rsync.alpinelinux.org/alpine/$dir/"
size=`rsync -a -n --stats "$src" "$dest" | grep '^Total file size' | tr -d ',' | awk '{ print $4 }'`
total=$(( $old_total + $size ))
echo "$dir: $size" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'
done

echo "total: $total" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'
rm -r "$dest"
</pre>

== Setting up the cron job ==
Install rsync which will be used to sync from the master mirror.
{{Cmd|apk add rsync}}

Save the following file as ''/etc/periodic/hourly/alpine-mirror''
<pre>
#!/usr/bin/env sh

# make sure we never run 2 rsync at the same time
lockfile="/tmp/alpine-mirror.lock"
if [ -z "$flock" ] ; then
exec env flock=1 flock -n $lockfile "$0" "$@"
fi

src=rsync://rsync.alpinelinux.org/alpine/
dest=/var/www/localhost/htdocs/alpine/

# uncomment this to exclude old v2.x branches
#exclude="--exclude v2.*"

mkdir -p "$dest"
/usr/bin/rsync \
--archive \
--update \
--hard-links \
--delete \
--delete-after \
--delay-updates \
--timeout=600 \
$exclude \
"$src" "$dest"

</pre>

(or use [https://gist.github.com/jirutka/288c6fff7c0b8a835d143686207316be this script])

Make it executable:
{{Cmd|<nowiki>chmod +x /etc/periodic/hourly/alpine-mirror</nowiki>}}

Now it will sync every hour. (given cron runs)

== Setting up HTTP access via lighttpd ==

Install the lighttpd server
{{Cmd|apk add lighttpd}}

Enable dir listings by uncommenting the following line in ''/etc/lighttpd/lighttpd.conf'':
dir-listing.activate = "enable"

Also set cache-control to force cache revalidate every 30 mins. Uncomment mod_setenv in ''/etc/lighttpd/lighttpd.conf'':
"mod_setenv",

Add also the following lines to ''/etc/lighttpd/lighttpd.conf'':
setenv.add-response-header += (
"Cache-Control" => "must-revalidate"
)

Start lighttpd and make it start at boot:
{{Cmd|rc-service lighttpd start
rc-update add lighttpd}}

{{Note|You may wish to consider [[Darkhttpd]] as an alternative to [[Lighttpd]]

If so, simply install, start and auto-start the webserver:

{{Cmd|apk add darkhttpd && rc-service darkhttpd start && rc-update add darkhttpd}}

Darkhttpd will, by default, offer directory listings and serve data from /var/www/localhost/htdocs/

See the main article on [[Darkhttpd]] for more configuration options}}

== Setting up rsyncd ==
Add the following lines to ''/etc/rsyncd.conf'':
<pre>
[alpine]
path = /var/www/localhost/htdocs/alpine
comment = My Alpine Linux Mirror
</pre>

Optionally set a bandwidth limit in ''/etc/conf.d/rsyncd''. In this example we limit to 500Kbytes/s (approx 5Mbit/s)
<pre>
RSYNC_OPTS="--bwlimit=500"
</pre>

== Mirror statistics ==

Simple bandwidth statistics can be generated with vnstat.

{{Cmd|apk add vnstat}}

edit /etc/vnstat.conf and replace the interface name with the appropriate one.

Start vnstatd

{{Cmd|/etc/init.d/vnstatd start }}

copy the following script to /etc/periodic/15min/stats and make sure your crond is running.
please not that heredoc should be tab indented or the script will fail. A working copy can be found here: http://tpaste.us/RrMv

<pre>
#!/usr/bin/env sh

output="/var/www/localhost/htdocs/.stats"
nic="eth0"

generate_index() {
cat <<-EOF
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="cache-control" content=no-cache">
<meta http-equiv="refresh" content="3000">
<title>Alpine Linux mirror statistics</title>
</head>
<body>
<table border="0">
<tr><td><img src="summary.png" alt="summary"></td><td><img src="hours.png" alt="hours"></td></tr>
<tr><td rowspan="2"><img src="days.png" alt="days"></td><td><img src="top10.png" alt="top10"></td></tr>
<tr><td><img src="months.png" alt="months"></td></tr>
</table>
</body>
</html>
EOF
}

if [ ! -f "$output"/index.html ]; then
mkdir -p $output
generate_index > "$output"/index.html
fi

for type in hours days months top10 summary hsummary vsummary; do
vnstati --${type} -i $nic -o $output/${type}.png
done
</pre>

== Update mirror from mqtt ==

If you want your mirror to be really uptodate compared to our master mirror you can subscribe to Alpine Linux message server "msg.alpinelinux.org" and check for upload messages.
Add mqtt-exec to be able to execute processes when specific topics are being send.

{{Cmd| apk add mqtt-exec}}

mqtt-exec supports running multiple time so we need to setup a specific config.

{{Cmd| ln -s /etc/init.d/mqtt-exec /etc/init.d/mqtt-exec.sync-mirror}}

{{Cmd| ln -s /etc/conf.d/mqtt-exec /etc/conf.d/mqtt-exec.sync-mirror}}

edit /etc/conf.d/mqtt-exec.sync-mirror

<pre>
mqtt_topics="rsync/rsync.alpinelinux.org/#"
exec_user="buildozer"
exec_command="/usr/local/bin/sync-mirror"
</pre>

Copy the following file to /usr/local/bin/sync-mirror and make it executable (dont forget to update the variables).

<pre>
#!/bin/sh

src="rsync://rsync.alpinelinux.org/alpine/"
dest="/var/www/localhost/htdocs/alpine/"
lock="/tmp/sync-mirror.lock"
topic="$1"
dir="$2"

[ -z "$flock" ] && exec env flock=1 flock $lock $0 "$@"

if [ -n "$dir" ] && [ -d "$dest/${dir%/*}" ]; then
logger "Syncing directory: $dir"
src="${src}${dir%/}/"
dest="${dest}${dir%/}/"
else
logger "Syncing all directories"
fi

/usr/bin/rsync \
--archive \
--update \
--verbose \
--progress \
--timeout=600 \
--delay-updates \
--delete-after \
"$src" \
"$dest"
</pre>

And finally start mqtt-exec and let it listen on msg.alpinelinux.org

{{Cmd|/etc/init.d/mqtt-exec.sync-mirror start}}

To make sure you are not missing any packages (in case something goes wrong with MQTT subscription) you can periodically sync all directories by adding the script to cron.

{{Cmd|ln -s /usr/local/bin/sync-mirror /etc/periodic/hourly/sync-mirror}}

Now watch your syslog as it should tell you when it will update directories in your local mirror.

== Partial mirror using nginx ==

For a private mirror it might make sense to sync only the newest versions of Alpine to save space, but if you ''do'' point an old Alpine version to your mirror they should still be able to install packages. We can achieve this by using nginx to serve the mirrored content and use regex location matching to redirect requests to a public mirror.

Let's assume you chose to only mirror Alpine versions up from v3.13. If a client asks your mirror for v.3.10 it should redirect to another mirror.

Your nginx config server block should look something like this:

<pre>
server {
listen 80;
server_name alpine.mydomain.local;
root /data/alpine; # point to where your alpine mirror is located. make sure nginx is allowed to read it
autoindex on; # Enable indexing

# the following location block will match for v3.0 to v3.12
# and will forward it to dl-4.alpinelinux.org.
location ~* /(v3\.([1-9]|1[012]))$
{
return 302 http://dl-cdn.alpinelinux.org/alpine$request_uri;
}
}
</pre>

The corresponding sync script could look something like this:

<pre>
#!/usr/bin/env sh

# make sure we never run 2 rsync at the same time
lockfile="/tmp/alpine-mirror.lock"
if [ -z "$flock" ] ; then
exec env flock=1 flock -n $lockfile "$0" "$@"
fi

src=rsync://rsync.alpinelinux.org/alpine/
dest=/data/alpine/

exclude="--exclude v2.* --exclude v3.0 --exclude v3.1 --exclude v3.2 --exclude v3.3 --exclude v3.4 --exclude v3.5 --exclude v3.6 --exclude v3.7 --exclude v3.8 --exclude v3.9 --exclude v3.10 --exclude v3.11 --exclude v3.12"

mkdir -p "$dest"
/usr/bin/rsync -vvv \
--archive \
--update \
--hard-links \
--delete \
--delete-after \
--delete-excluded \
--delay-updates \
--timeout=600 \
$exclude \
"$src" "$dest"
</pre>

[[Category:Server]]
[[Category:Package Manager]]

How to setup a Alpine Linux mirror

2023-01-13T18:14:19Z

Mcrute: Update mirror sizes

== Introduction ==
This document describes how to set up an Alpine Linux mirror and make it available via http and rsync.

We will:
* create the dir where we have the mirror
* set up a cron job to sync with master mirror every hour
* set up lighttpd for http access
* set up rsync so other mirrors can rsync from you

Make sure that you have enough disk space.

Current (2023-01-13) disk usage in GB:

{|class="wikitable"
!edge
!v3.0
!v3.1
!v3.2
!v3.3
!v3.4
!v3.5
!v3.6
!v3.7
!v3.8
!v3.9
!v3.10
!v3.11
!v3.12
!v3.13
!v3.14
!v3.15
!v3.16
|v3.17

!total
|-
|349
|17
|18
|15
|21
|25
|27
|45
|43
|60
|73
|92
|126
|148
|156
|174
|176
|184
|211

|'''1960'''
|}

Script used to calculate the size:

<pre>
#!/bin/sh

total=0
dest="$(mktemp -d)"

for dir in edge v3.0 v3.1 v3.2 v3.3 v3.4 v3.5 v3.6 v3.7 v3.8 v3.9 v3.10 v3.11 v3.12 v3.13 v3.14 v3.15 v3.16; do
old_total="$total"
src="rsync://rsync.alpinelinux.org/alpine/$dir/"
size=`rsync -a -n --stats "$src" "$dest" | grep '^Total file size' | tr -d ',' | awk '{ print $4 }'`
total=$(( $old_total + $size ))
echo "$dir: $size" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'
done

echo "total: $total" | awk '{ print $1 sprintf("%.1f", $2/1073741824) }'
rm -r "$dest"
</pre>

== Setting up the cron job ==
Install rsync which will be used to sync from the master mirror.
{{Cmd|apk add rsync}}

Save the following file as ''/etc/periodic/hourly/alpine-mirror''
<pre>
#!/usr/bin/env sh

# make sure we never run 2 rsync at the same time
lockfile="/tmp/alpine-mirror.lock"
if [ -z "$flock" ] ; then
exec env flock=1 flock -n $lockfile "$0" "$@"
fi

src=rsync://rsync.alpinelinux.org/alpine/
dest=/var/www/localhost/htdocs/alpine/

# uncomment this to exclude old v2.x branches
#exclude="--exclude v2.*"

mkdir -p "$dest"
/usr/bin/rsync \
--archive \
--update \
--hard-links \
--delete \
--delete-after \
--delay-updates \
--timeout=600 \
$exclude \
"$src" "$dest"

</pre>

(or use [https://gist.github.com/jirutka/288c6fff7c0b8a835d143686207316be this script])

Make it executable:
{{Cmd|<nowiki>chmod +x /etc/periodic/hourly/alpine-mirror</nowiki>}}

Now it will sync every hour. (given cron runs)

== Setting up HTTP access via lighttpd ==

Install the lighttpd server
{{Cmd|apk add lighttpd}}

Enable dir listings by uncommenting the following line in ''/etc/lighttpd/lighttpd.conf'':
dir-listing.activate = "enable"

Also set cache-control to force cache revalidate every 30 mins. Uncomment mod_setenv in ''/etc/lighttpd/lighttpd.conf'':
"mod_setenv",

Add also the following lines to ''/etc/lighttpd/lighttpd.conf'':
setenv.add-response-header += (
"Cache-Control" => "must-revalidate"
)

Start lighttpd and make it start at boot:
{{Cmd|rc-service lighttpd start
rc-update add lighttpd}}

{{Note|You may wish to consider [[Darkhttpd]] as an alternative to [[Lighttpd]]

If so, simply install, start and auto-start the webserver:

{{Cmd|apk add darkhttpd && rc-service darkhttpd start && rc-update add darkhttpd}}

Darkhttpd will, by default, offer directory listings and serve data from /var/www/localhost/htdocs/

See the main article on [[Darkhttpd]] for more configuration options}}

== Setting up rsyncd ==
Add the following lines to ''/etc/rsyncd.conf'':
<pre>
[alpine]
path = /var/www/localhost/htdocs/alpine
comment = My Alpine Linux Mirror
</pre>

Optionally set a bandwidth limit in ''/etc/conf.d/rsyncd''. In this example we limit to 500Kbytes/s (approx 5Mbit/s)
<pre>
RSYNC_OPTS="--bwlimit=500"
</pre>

== Mirror statistics ==

Simple bandwidth statistics can be generated with vnstat.

{{Cmd|apk add vnstat}}

edit /etc/vnstat.conf and replace the interface name with the appropriate one.

Start vnstatd

{{Cmd|/etc/init.d/vnstatd start }}

copy the following script to /etc/periodic/15min/stats and make sure your crond is running.
please not that heredoc should be tab indented or the script will fail. A working copy can be found here: http://tpaste.us/RrMv

<pre>
#!/usr/bin/env sh

output="/var/www/localhost/htdocs/.stats"
nic="eth0"

generate_index() {
cat <<-EOF
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="cache-control" content=no-cache">
<meta http-equiv="refresh" content="3000">
<title>Alpine Linux mirror statistics</title>
</head>
<body>
<table border="0">
<tr><td><img src="summary.png" alt="summary"></td><td><img src="hours.png" alt="hours"></td></tr>
<tr><td rowspan="2"><img src="days.png" alt="days"></td><td><img src="top10.png" alt="top10"></td></tr>
<tr><td><img src="months.png" alt="months"></td></tr>
</table>
</body>
</html>
EOF
}

if [ ! -f "$output"/index.html ]; then
mkdir -p $output
generate_index > "$output"/index.html
fi

for type in hours days months top10 summary hsummary vsummary; do
vnstati --${type} -i $nic -o $output/${type}.png
done
</pre>

== Update mirror from mqtt ==

If you want your mirror to be really uptodate compared to our master mirror you can subscribe to Alpine Linux message server "msg.alpinelinux.org" and check for upload messages.
Add mqtt-exec to be able to execute processes when specific topics are being send.

{{Cmd| apk add mqtt-exec}}

mqtt-exec supports running multiple time so we need to setup a specific config.

{{Cmd| ln -s /etc/init.d/mqtt-exec /etc/init.d/mqtt-exec.sync-mirror}}

{{Cmd| ln -s /etc/conf.d/mqtt-exec /etc/conf.d/mqtt-exec.sync-mirror}}

edit /etc/conf.d/mqtt-exec.sync-mirror

<pre>
mqtt_topics="rsync/rsync.alpinelinux.org/#"
exec_user="buildozer"
exec_command="/usr/local/bin/sync-mirror"
</pre>

Copy the following file to /usr/local/bin/sync-mirror and make it executable (dont forget to update the variables).

<pre>
#!/bin/sh

src="rsync://rsync.alpinelinux.org/alpine/"
dest="/var/www/localhost/htdocs/alpine/"
lock="/tmp/sync-mirror.lock"
topic="$1"
dir="$2"

[ -z "$flock" ] && exec env flock=1 flock $lock $0 "$@"

if [ -n "$dir" ] && [ -d "$dest/${dir%/*}" ]; then
logger "Syncing directory: $dir"
src="${src}${dir%/}/"
dest="${dest}${dir%/}/"
else
logger "Syncing all directories"
fi

/usr/bin/rsync \
--archive \
--update \
--verbose \
--progress \
--timeout=600 \
--delay-updates \
--delete-after \
"$src" \
"$dest"
</pre>

And finally start mqtt-exec and let it listen on msg.alpinelinux.org

{{Cmd|/etc/init.d/mqtt-exec.sync-mirror start}}

To make sure you are not missing any packages (in case something goes wrong with MQTT subscription) you can periodically sync all directories by adding the script to cron.

{{Cmd|ln -s /usr/local/bin/sync-mirror /etc/periodic/hourly/sync-mirror}}

Now watch your syslog as it should tell you when it will update directories in your local mirror.

== Partial mirror using nginx ==

For a private mirror it might make sense to sync only the newest versions of Alpine to save space, but if you ''do'' point an old Alpine version to your mirror they should still be able to install packages. We can achieve this by using nginx to serve the mirrored content and use regex location matching to redirect requests to a public mirror.

Let's assume you chose to only mirror Alpine versions up from v3.13. If a client asks your mirror for v.3.10 it should redirect to another mirror.

Your nginx config server block should look something like this:

<pre>
server {
listen 80;
server_name alpine.mydomain.local;
root /data/alpine; # point to where your alpine mirror is located. make sure nginx is allowed to read it
autoindex on; # Enable indexing

# the following location block will match for v3.0 to v3.12
# and will forward it to dl-4.alpinelinux.org.
location ~* /(v3\.([1-9]|1[012]))$
{
return 302 http://dl-cdn.alpinelinux.org/alpine$request_uri;
}
}
</pre>

The corresponding sync script could look something like this:

<pre>
#!/usr/bin/env sh

# make sure we never run 2 rsync at the same time
lockfile="/tmp/alpine-mirror.lock"
if [ -z "$flock" ] ; then
exec env flock=1 flock -n $lockfile "$0" "$@"
fi

src=rsync://rsync.alpinelinux.org/alpine/
dest=/data/alpine/

exclude="--exclude v2.* --exclude v3.0 --exclude v3.1 --exclude v3.2 --exclude v3.3 --exclude v3.4 --exclude v3.5 --exclude v3.6 --exclude v3.7 --exclude v3.8 --exclude v3.9 --exclude v3.10 --exclude v3.11 --exclude v3.12"

mkdir -p "$dest"
/usr/bin/rsync -vvv \
--archive \
--update \
--hard-links \
--delete \
--delete-after \
--delete-excluded \
--delay-updates \
--timeout=600 \
$exclude \
"$src" "$dest"
</pre>

[[Category:Server]]
[[Category:Package Manager]]

Apk spec

2022-07-18T03:32:32Z

Mcrute:

{{Draft}}

For end-user facing documentation about apk, check out the [[Package_management]] page.

This page is an attempt to document the internal data structures of the apk package manager. The canonical implementation of the apk format is [https://gitlab.alpinelinux.org/alpine/apk-tools apk-tools] and much of this information is gleaned from reading the source code.

There are three generations of the APK data formats. Version 1 is deprecated and no longer used, version 2 is currently the main version in use by apk-tools, and version 3 is under development. This page mostly describes the data formats used in version 2.

= Background =
== Tar Segments ==
Tar segments are a set of tar records. Normal tar files contain two null records at the end of the tar file to signal the end of the tarball. Tar segments are lacking these two records and can thus be concatenated before other tar files and will behave as one continuous tar file. The APK v2 package format makes use of both tar segments and tarballs.

Tar segments can be compressed using gzip compression. Gzip is a stream-based file format and multiple streams can be concatenated together. Most tooling will treat multiple gzip streams within a file as if it were a single stream. APK v2 files are aware of gzip streams and use them for file segmentation.

= Package Format V2 =
== Binary Format ==
APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the concatenated control segment and data tarball.

The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the concatenated control and data gzip streams. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.

The control segment contains the package metadata in a <tt>.PKGINFO</tt> file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.

The data tarball is a standard gzipped tarball with extra PAX headers that contain the SHA1 hash of each file in the tar header for that file. The hash is contained in a header called <tt>APK-TOOLS.checksum.SHA1</tt>. Unlike the other tar streams this tarball does contain the two end-of-tar null records. It is always the final segment of an APK package. Hashes are added with the [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-tar.c abuild-tar] tool.

== PKGINFO Format ==
The PKGINFO file contains the package metadata. This is a plain-text file similar to INI files. Lines that begin with <tt>#</tt> are comments and ignored. Unlike INI files the parsing format of this file is very strict. Each key-value pair must be separated by exactly one space, one equal sign, and one more space (<tt> = </tt>). Keys may be repeated in this file and should be treated as a list of values if repetitions are found.

The specification for what fields are valid in PKGINFO is largely defined by [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild.in abuild]. As of July 2022 the following fields are supported:

* <tt>pkgname</tt> - package name
* <tt>pkgver</tt> - package version
* <tt>pkgdesc</tt> - package description
* <tt>url</tt> - package url
* <tt>builddate</tt> - unix timestamp of the package build date/time
* <tt>packager</tt> - name (and typically email) of person who built the package
* <tt>size</tt> - the installed-size of the package
* <tt>arch</tt> - the architecture of the package (ex: x86_64)
* <tt>origin</tt> - the origin name of the package
* <tt>commit</tt> - the commit hash from which the package was built
* <tt>maintainer</tt> - name (and typically email) of the package maintainer
* <tt>replaces_priority</tt> - replaces priority field for package (integer)
* <tt>provider_priority</tt> - provider priority for the package (integer)
* <tt>license</tt> - license string for the package
* <tt>depend</tt> - dependencies for the package (repeated)
* <tt>replaces</tt> - packages this package replaces (repeated)
* <tt>provides</tt> - what this package provides (repeated)
* <tt>triggers</tt> - what packages this package triggers on (repeated)
* <tt>install_if</tt> - install this package if these packages are present (repeated)
* <tt>datahash</tt> - hex-encoded sha256 checksum of the data tarball

== Example of PKGINFO ==
<pre>
# Generated by abuild 3.9.0-r2
# using fakeroot version 1.25.3
# Wed Jul 6 19:09:49 UTC 2022
pkgname = busybox
pkgver = 1.35.0-r18
pkgdesc = Size optimized toolbox of many common UNIX utilities
url = https://busybox.net/
builddate = 1657134589
packager = Buildozer <alpine-devel@lists.alpinelinux.org>
size = 958464
arch = x86_64
origin = busybox
commit = 332d2fff53cd4537d415e15e55e8ceb6fe6eaedb
maintainer = Sören Tempel <soeren+alpine@soeren-tempel.net>
provider_priority = 100
license = GPL-2.0-only
replaces = busybox-initscripts
provides = /bin/sh
triggers = /bin /usr/bin /sbin /usr/sbin /lib/modules/*
# automatically detected:
provides = cmd:busybox=1.35.0-r18
provides = cmd:sh=1.35.0-r18
depend = so:libc.musl-x86_64.so.1
datahash = 7d3351ac6c3ebaf18182efb5390061f50d077ce5ade60a15909d91278f70ada7
</pre>

== Package Building Example ==
This is a set of commands to partially build a package. '''DO NOT DO THIS''', it's mainly an example to see how this all fits together. Use the official build tools to build packages.

<pre>
tar -c .PKGNIFO .pre-install | abuild-tar --cut | gzip -9 > $controldir/control.tar.gz
cd $pkgdir; tar -c * | abuild-tar --hash | gzip -9 > $controldir/data.tar.gz
cat $controldir/control.tar.gz $controldir/data.tar.gz > mypackage-1.0-r0.apk
</pre>

= Index Format V2 =
== Binary Format ==
The index is served as [http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz APKINDEX.tar.gz] and is downloaded by apk to power the package database. The index is signed similarly to packages. The main difference between the index and packages is that the index file contains only two segments.

The signature segment is identical to a package segment and is concatenated, in its own gzip stream, to the beginning of the APKINDEX tarball.

The APKINDEX tarball contains two files: a DESCRIPTION file and an APKINDEX file. Each of these files is in their own tar record and the final record is followed by the standard end-of-tar null records. The DESCRIPTION file is a simple text file containing a description of the index (ex: <tt>community v20210212-7170-g5c9853dc69</tt>). The APKINDEX file is a text file containing records for each package in the repository in a text-based format. Each record is separated by a newline.

== APKINDEX Format ==
The APKINDEX file contains a set of records extracted from the PKGINFO file of each package in the repository. Each line is prefixed with a letter, colon, and is followed by the value of the field. Lines are newline (<tt>\n</tt>) terminated and there is one blank line between records for a package.

The <tt>apk_pkg_write_index_entry</tt> function of [https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/ff7c8f6ee9dfa2add57b88dc271f6711030e72a0/src/package.c#L905 package.c] defines the currently accepted fields. As of July 2022, these are:

* <tt>C:</tt> - file checksum, see below
* <tt>P:</tt> - package name (corresponds to <tt>pkgname</tt> in PKGINFO)
* <tt>V:</tt> - package version (corresponds to <tt>pkgver</tt> in PKGINFO)
* <tt>A:</tt> - architecture (corresponds to <tt>arch</tt> in PKGINFO), optional
* <tt>S:</tt> - size of entire package, integer
* <tt>I:</tt> - installed size, integer (corresponds to <tt>size</tt> in PKGINFO)
* <tt>T:</tt> - description (corresponds to <tt>pkgdesc</tt> in PKGINFO)
* <tt>U:</tt> - url (corresponds to <tt>url</tt> in PKGINFO)
* <tt>L:</tt> - license (corresponds to <tt>license</tt> in PKGINFO)
* <tt>o:</tt> - origin (corresponds to <tt>origin</tt> in PKGINFO), optional
* <tt>m:</tt> - maintainer (corresponds to <tt>maintainer</tt> in PKGINFO), optional
* <tt>t:</tt> - build time (corresponds to <tt>builddate</tt> in PKGINFO), optional
* <tt>c:</tt> - commit (corresponds to <tt>commit</tt> in PKGINFO), optional
* <tt>k:</tt> - provider priority, integer (corresponds to <tt>provider_priority</tt> in PKGINFO), optional
* <tt>D:</tt> - dependencies (corresponds to <tt>depend</tt> in PKGINFO, concatenated by spaces into a single line)
* <tt>p:</tt> - provides (corresponds to <tt>provides</tt> in PKGINFO, concatenated by spaces into a single line)
* <tt>i:</tt> - install if (corresponds to <tt>install_if</tt> in PKGINFO, concatenated by spaces into a single line)

== Package Checksum Field ==
The package checksum field is the SHA1 hash of the second gzip stream (control stream) in the package. The binary hash digest is base64 encoded. This is prefixed with <tt>Q1</tt> to differentiate it from the MD5 hashes used in older index formats. It is not possible to compute this checksum with standard command line tools but the apk-tools can compute it in their <tt>index</tt> operation.

== Example APKINDEX Record ==
<pre>
C:Q1P4IRU/u5yB4CSnUEBRD1WWwajrY=
P:jool-tools
V:4.1.5-r0
A:x86_64
S:140605
I:434176
T:Userspace control tools for SIIT / NAT64 Jool
U:https://www.jool.mx
L:GPL-2.0-only
o:jool-tools
m:Jakub Jirutka <jakub@jirutka.cz>
t:1620480809
c:771b3b0910ea9c7736db6ca4ff5c37ca9cf9af0d
D:so:libc.musl-x86_64.so.1 so:libnl-3.so.200 so:libnl-genl-3.so.200
p:cmd:jool=4.1.5-r0 cmd:jool_siit=4.1.5-r0 cmd:joold=4.1.5-r0
</pre>

= Installed Database V2 =
The installed database is used by apk to track which packages are installed and what modifications those packages have made to the system. This file is located at <tt>/lib/apk/db/installed</tt>. The installed file is a plaintext file of the same format as APKINDEX (contained in APKINDEX.tar.gz). It is neither compressed nor signed. Each record in the installed file starts with a package index record with the same fields as the APKINDEX file. The installed file adds some additional fields that are defined in [https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/ff7c8f6ee9dfa2add57b88dc271f6711030e72a0/src/database.c#L937 database.c]. As of July 2022 these additional fields are:

* <tt>r:</tt> - packages which this package replaces, space separated list
* <tt>q:</tt> - replaces priority, integer, optional
* <tt>s:</tt> - repository tag, optional, this will be set if the package is tagged to a repository in the world file (ex: linux@testing)
* <tt>f:</tt> - indicates broken items, space separated (f=files, s=scripts, x=xattrs, S=file hashes)

The following fields are repeated and in groups consist of a set of mutations made to the system to install the package.

ACL lines are specified as uid, colon, gid, colon, and mode.

* <tt>F:</tt> - directory name that was created by package, repeated
* <tt>M:</tt> - directory ACL, only if different than the default of 0:0:0755
* <tt>R:</tt> - file name, relative to preceding directory name
* <tt>a:</tt> - file ACL
* <tt>Z:</tt> - file checksum, if the checksum in the package is not none, a <tt>Q1</tt> prefix indicates this will be a SHA1 hash in base64 format

[[Category:Package Manager]] [[Category:Development]]

Apk spec

2022-07-18T01:41:09Z

Mcrute:

{{Draft}}

For end-user facing documentation about apk, check out the [[Package_management]] page.

This page is an attempt to document the internal data structures of the apk package manager. The canonical implementation of the apk format is [https://gitlab.alpinelinux.org/alpine/apk-tools apk-tools] and much of this information is gleaned from reading the source code.

There are three generations of the APK data formats. Version 1 is deprecated and no longer used, version 2 is currently the main version in use by apk-tools, and version 3 is under development. This page mostly describes the data formats used in version 2.

= Background =
== Tar Segments ==
Tar segments are a set of tar records. Normal tar files contain two null records at the end of the tar file to signal the end of the tarball. Tar segments are lacking these two records and can thus be concatenated before other tar files and will behave as one continuous tar file. The APK v2 package format makes use of both tar segments and tarballs.

Tar segments can be compressed using gzip compression. Gzip is a stream-based file format and multiple streams can be concatenated together. Most tooling will treat multiple gzip streams within a file as if it were a single stream. APK v2 files are aware of gzip streams and use them for file segmentation.

= Package Format V2 =
== Binary Format ==
APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the concatenated control segment and data tarball.

The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the concatenated control and data gzip streams. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.

The control segment contains the package metadata in a <tt>.PKGINFO</tt> file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.

The data tarball is a standard gzipped tarball with extra PAX headers that contain the SHA1 hash of each file in the tar header for that file. The hash is contained in a header called <tt>APK-TOOLS.checksum.SHA1</tt>. Unlike the other tar streams this tarball does contain the two end-of-tar null records. It is always the final segment of an APK package. Hashes are added with the [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-tar.c abuild-tar] tool.

== PKGINFO Format ==
The PKGINFO file contains the package metadata. This is a plain-text file similar to INI files. Lines that begin with <tt>#</tt> are comments and ignored. Unlike INI files the parsing format of this file is very strict. Each key-value pair must be separated by exactly one space, one equal sign, and one more space (<tt> = </tt>). Keys may be repeated in this file and should be treated as a list of values if repetitions are found.

The specification for what fields are valid in PKGINFO is largely defined by [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild.in abuild]. As of July 2022 the following fields are supported:

* <tt>pkgname</tt> - package name
* <tt>pkgver</tt> - package version
* <tt>pkgdesc</tt> - package description
* <tt>url</tt> - package url
* <tt>builddate</tt> - unix timestamp of the package build date/time
* <tt>packager</tt> - name (and typically email) of person who built the package
* <tt>size</tt> - the installed-size of the package
* <tt>arch</tt> - the architecture of the package (ex: x86_64)
* <tt>origin</tt> - the origin name of the package
* <tt>commit</tt> - the commit hash from which the package was built
* <tt>maintainer</tt> - name (and typically email) of the package maintainer
* <tt>replaces_priority</tt> - replaces priority field for package (integer)
* <tt>provider_priority</tt> - provider priority for the package (integer)
* <tt>license</tt> - license string for the package
* <tt>depend</tt> - dependencies for the package (repeated)
* <tt>replaces</tt> - packages this package replaces (repeated)
* <tt>provides</tt> - what this package provides (repeated)
* <tt>triggers</tt> - what packages this package triggers on (repeated)
* <tt>install_if</tt> - install this package if these packages are present (repeated)
* <tt>datahash</tt> - hex-encoded sha256 checksum of the data tarball

== Example of PKGINFO ==
<pre>
# Generated by abuild 3.9.0-r2
# using fakeroot version 1.25.3
# Wed Jul 6 19:09:49 UTC 2022
pkgname = busybox
pkgver = 1.35.0-r18
pkgdesc = Size optimized toolbox of many common UNIX utilities
url = https://busybox.net/
builddate = 1657134589
packager = Buildozer <alpine-devel@lists.alpinelinux.org>
size = 958464
arch = x86_64
origin = busybox
commit = 332d2fff53cd4537d415e15e55e8ceb6fe6eaedb
maintainer = Sören Tempel <soeren+alpine@soeren-tempel.net>
provider_priority = 100
license = GPL-2.0-only
replaces = busybox-initscripts
provides = /bin/sh
triggers = /bin /usr/bin /sbin /usr/sbin /lib/modules/*
# automatically detected:
provides = cmd:busybox=1.35.0-r18
provides = cmd:sh=1.35.0-r18
depend = so:libc.musl-x86_64.so.1
datahash = 7d3351ac6c3ebaf18182efb5390061f50d077ce5ade60a15909d91278f70ada7
</pre>

== Package Building Example ==
This is a set of commands to partially build a package. '''DO NOT DO THIS''', it's mainly an example to see how this all fits together. Use the official build tools to build packages.

<pre>
tar -c .PKGNIFO .pre-install | abuild-tar --cut | gzip -9 > $controldir/control.tar.gz
cd $pkgdir; tar -c * | abuild-tar --hash | gzip -9 > $controldir/data.tar.gz
cat $controldir/control.tar.gz $controldir/data.tar.gz > mypackage-1.0-r0.apk
</pre>

= Index Format V2 =
== Binary Format ==
The index is served as [http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz APKINDEX.tar.gz] and is downloaded by apk to power the package database. The index is signed similarly to packages. The main difference between the index and packages is that the index file contains only two segments.

The signature segment is identical to a package segment and is concatenated, in its own gzip stream, to the beginning of the APKINDEX tarball.

The APKINDEX tarball contains two files: a DESCRIPTION file and an APKINDEX file. Each of these files is in their own tar record and the final record is followed by the standard end-of-tar null records. The DESCRIPTION file is a simple text file containing a description of the index (ex: <tt>community v20210212-7170-g5c9853dc69</tt>). The APKINDEX file is a text file containing records for each package in the repository in a text-based format. Each record is separated by a newline.

== APKINDEX Format ==
The APKINDEX file contains a set of records extracted from the PKGINFO file of each package in the repository. Each line is prefixed with a letter, colon, and is followed by the value of the field. Lines are newline (<tt>\n</tt>) terminated and there is one blank line between records for a package.

The <tt>apk_pkg_write_index_entry</tt> function of [https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/ff7c8f6ee9dfa2add57b88dc271f6711030e72a0/src/package.c#L905 package.c] defines the currently accepted fields. As of July 2022, these are:

* <tt>C:</tt> - file checksum, see below
* <tt>P:</tt> - package name (corresponds to <tt>pkgname</tt> in PKGINFO)
* <tt>V:</tt> - package version (corresponds to <tt>pkgver</tt> in PKGINFO)
* <tt>A:</tt> - architecture (corresponds to <tt>arch</tt> in PKGINFO), optional
* <tt>S:</tt> - size of entire package, integer
* <tt>I:</tt> - installed size, integer (corresponds to <tt>size</tt> in PKGINFO)
* <tt>T:</tt> - description (corresponds to <tt>pkgdesc</tt> in PKGINFO)
* <tt>U:</tt> - url (corresponds to <tt>url</tt> in PKGINFO)
* <tt>L:</tt> - license (corresponds to <tt>license</tt> in PKGINFO)
* <tt>o:</tt> - origin (corresponds to <tt>origin</tt> in PKGINFO), optional
* <tt>m:</tt> - maintainer (corresponds to <tt>maintainer</tt> in PKGINFO), optional
* <tt>t:</tt> - build time (corresponds to <tt>builddate</tt> in PKGINFO), optional
* <tt>c:</tt> - commit (corresponds to <tt>commit</tt> in PKGINFO), optional
* <tt>k:</tt> - provider priority, integer (corresponds to <tt>provider_priority</tt> in PKGINFO), optional
* <tt>D:</tt> - dependencies (corresponds to <tt>depend</tt> in PKGINFO, concatenated by spaces into a single line)
* <tt>p:</tt> - provides (corresponds to <tt>provides</tt> in PKGINFO, concatenated by spaces into a single line)
* <tt>i:</tt> - install if (corresponds to <tt>install_if</tt> in PKGINFO, concatenated by spaces into a single line)

== Package Checksum Field ==
The package checksum field is the SHA1 hash of the second gzip stream (control stream) in the package. The binary hash digest is base64 encoded. This is prefixed with <tt>Q1</tt> to differentiate it from the MD5 hashes used in older index formats. It is not possible to compute this checksum with standard command line tools but the apk-tools can compute it in their <tt>index</tt> operation.

== Example APKINDEX Record ==
<pre>
C:Q1P4IRU/u5yB4CSnUEBRD1WWwajrY=
P:jool-tools
V:4.1.5-r0
A:x86_64
S:140605
I:434176
T:Userspace control tools for SIIT / NAT64 Jool
U:https://www.jool.mx
L:GPL-2.0-only
o:jool-tools
m:Jakub Jirutka <jakub@jirutka.cz>
t:1620480809
c:771b3b0910ea9c7736db6ca4ff5c37ca9cf9af0d
D:so:libc.musl-x86_64.so.1 so:libnl-3.so.200 so:libnl-genl-3.so.200
p:cmd:jool=4.1.5-r0 cmd:jool_siit=4.1.5-r0 cmd:joold=4.1.5-r0
</pre>

[[Category:Package Manager]] [[Category:Development]]

Apk spec

2022-07-18T01:39:26Z

Mcrute:

{{Draft}}

For end-user facing documentation about apk, check out the [[Package_management]] page.

This page is an attempt to document the internal data structures of the apk package manager. The canonical implementation of the apk format is [https://gitlab.alpinelinux.org/alpine/apk-tools apk-tools] and much of this information is gleaned from reading the source code.

There are three generations of the APK data formats. Version 1 is deprecated and no longer used, version 2 is currently the main version in use by apk-tools, and version 3 is under development. This page mostly describes the data formats used in version 2.

= Package Format V2 =
== Tar Segments ==
Tar segments are a set of tar records. Normal tar files contain two null records at the end of the tar file to signal the end of the tarball. Tar segments are lacking these two records and can thus be concatenated before other tar files and will behave as one continuous tar file. The APK v2 package format makes use of both tar segments and tarballs.

Tar segments can be compressed using gzip compression. Gzip is a stream-based file format and multiple streams can be concatenated together. Most tooling will treat multiple gzip streams within a file as if it were a single stream. APK v2 files are aware of gzip streams and use them for file segmentation.

== Binary Format ==
APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the concatenated control segment and data tarball.

The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the concatenated control and data gzip streams. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.

The control segment contains the package metadata in a <tt>.PKGINFO</tt> file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.

The data tarball is a standard gzipped tarball with extra PAX headers that contain the SHA1 hash of each file in the tar header for that file. The hash is contained in a header called <tt>APK-TOOLS.checksum.SHA1</tt>. Unlike the other tar streams this tarball does contain the two end-of-tar null records. It is always the final segment of an APK package. Hashes are added with the [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-tar.c abuild-tar] tool.

== PKGINFO Format ==
The PKGINFO file contains the package metadata. This is a plain-text file similar to INI files. Lines that begin with <tt>#</tt> are comments and ignored. Unlike INI files the parsing format of this file is very strict. Each key-value pair must be separated by exactly one space, one equal sign, and one more space (<tt> = </tt>). Keys may be repeated in this file and should be treated as a list of values if repetitions are found.

The specification for what fields are valid in PKGINFO is largely defined by [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild.in abuild]. As of July 2022 the following fields are supported:

* <tt>pkgname</tt> - package name
* <tt>pkgver</tt> - package version
* <tt>pkgdesc</tt> - package description
* <tt>url</tt> - package url
* <tt>builddate</tt> - unix timestamp of the package build date/time
* <tt>packager</tt> - name (and typically email) of person who built the package
* <tt>size</tt> - the installed-size of the package
* <tt>arch</tt> - the architecture of the package (ex: x86_64)
* <tt>origin</tt> - the origin name of the package
* <tt>commit</tt> - the commit hash from which the package was built
* <tt>maintainer</tt> - name (and typically email) of the package maintainer
* <tt>replaces_priority</tt> - replaces priority field for package (integer)
* <tt>provider_priority</tt> - provider priority for the package (integer)
* <tt>license</tt> - license string for the package
* <tt>depend</tt> - dependencies for the package (repeated)
* <tt>replaces</tt> - packages this package replaces (repeated)
* <tt>provides</tt> - what this package provides (repeated)
* <tt>triggers</tt> - what packages this package triggers on (repeated)
* <tt>install_if</tt> - install this package if these packages are present (repeated)
* <tt>datahash</tt> - hex-encoded sha256 checksum of the data tarball

== Example of PKGINFO ==
<pre>
# Generated by abuild 3.9.0-r2
# using fakeroot version 1.25.3
# Wed Jul 6 19:09:49 UTC 2022
pkgname = busybox
pkgver = 1.35.0-r18
pkgdesc = Size optimized toolbox of many common UNIX utilities
url = https://busybox.net/
builddate = 1657134589
packager = Buildozer <alpine-devel@lists.alpinelinux.org>
size = 958464
arch = x86_64
origin = busybox
commit = 332d2fff53cd4537d415e15e55e8ceb6fe6eaedb
maintainer = Sören Tempel <soeren+alpine@soeren-tempel.net>
provider_priority = 100
license = GPL-2.0-only
replaces = busybox-initscripts
provides = /bin/sh
triggers = /bin /usr/bin /sbin /usr/sbin /lib/modules/*
# automatically detected:
provides = cmd:busybox=1.35.0-r18
provides = cmd:sh=1.35.0-r18
depend = so:libc.musl-x86_64.so.1
datahash = 7d3351ac6c3ebaf18182efb5390061f50d077ce5ade60a15909d91278f70ada7
</pre>

== Package Building Example ==
This is a set of commands to partially build a package. '''DO NOT DO THIS''', it's mainly an example to see how this all fits together. Use the official build tools to build packages.

<pre>
tar -c .PKGNIFO .pre-install | abuild-tar --cut | gzip -9 > $controldir/control.tar.gz
cd $pkgdir; tar -c * | abuild-tar --hash | gzip -9 > $controldir/data.tar.gz
cat $controldir/control.tar.gz $controldir/data.tar.gz > mypackage-1.0-r0.apk
</pre>

= Index Format V2 =
== Binary Format ==
The index is served as [http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz APKINDEX.tar.gz] and is downloaded by apk to power the package database. The index is signed similarly to packages. The main difference between the index and packages is that the index file contains only two segments.

The signature segment is identical to a package segment and is concatenated, in its own gzip stream, to the beginning of the APKINDEX tarball.

The APKINDEX tarball contains two files: a DESCRIPTION file and an APKINDEX file. Each of these files is in their own tar record and the final record is followed by the standard end-of-tar null records. The DESCRIPTION file is a simple text file containing a description of the index (ex: <tt>community v20210212-7170-g5c9853dc69</tt>). The APKINDEX file is a text file containing records for each package in the repository in a text-based format. Each record is separated by a newline.

== APKINDEX Format ==
The APKINDEX file contains a set of records extracted from the PKGINFO file of each package in the repository. Each line is prefixed with a letter, colon, and is followed by the value of the field. Lines are newline (<tt>\n</tt>) terminated and there is one blank line between records for a package.

The <tt>apk_pkg_write_index_entry</tt> function of [https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/ff7c8f6ee9dfa2add57b88dc271f6711030e72a0/src/package.c#L905 package.c] defines the currently accepted fields. As of July 2022, these are:

* <tt>C:</tt> - file checksum, see below
* <tt>P:</tt> - package name (corresponds to <tt>pkgname</tt> in PKGINFO)
* <tt>V:</tt> - package version (corresponds to <tt>pkgver</tt> in PKGINFO)
* <tt>A:</tt> - architecture (corresponds to <tt>arch</tt> in PKGINFO), optional
* <tt>S:</tt> - size of entire package, integer
* <tt>I:</tt> - installed size, integer (corresponds to <tt>size</tt> in PKGINFO)
* <tt>T:</tt> - description (corresponds to <tt>pkgdesc</tt> in PKGINFO)
* <tt>U:</tt> - url (corresponds to <tt>url</tt> in PKGINFO)
* <tt>L:</tt> - license (corresponds to <tt>license</tt> in PKGINFO)
* <tt>o:</tt> - origin (corresponds to <tt>origin</tt> in PKGINFO), optional
* <tt>m:</tt> - maintainer (corresponds to <tt>maintainer</tt> in PKGINFO), optional
* <tt>t:</tt> - build time (corresponds to <tt>builddate</tt> in PKGINFO), optional
* <tt>c:</tt> - commit (corresponds to <tt>commit</tt> in PKGINFO), optional
* <tt>k:</tt> - provider priority, integer (corresponds to <tt>provider_priority</tt> in PKGINFO), optional
* <tt>D:</tt> - dependencies (corresponds to <tt>depend</tt> in PKGINFO, concatenated by spaces into a single line)
* <tt>p:</tt> - provides (corresponds to <tt>provides</tt> in PKGINFO, concatenated by spaces into a single line)
* <tt>i:</tt> - install if (corresponds to <tt>install_if</tt> in PKGINFO, concatenated by spaces into a single line)

== Package Checksum Field ==
The package checksum field is the SHA1 hash of the second gzip stream (control stream) in the package. The binary hash digest is base64 encoded. This is prefixed with <tt>Q1</tt> to differentiate it from the MD5 hashes used in older index formats. It is not possible to compute this checksum with standard command line tools but the apk-tools can compute it in their <tt>index</tt> operation.

== Example APKINDEX Record ==
<pre>
C:Q1P4IRU/u5yB4CSnUEBRD1WWwajrY=
P:jool-tools
V:4.1.5-r0
A:x86_64
S:140605
I:434176
T:Userspace control tools for SIIT / NAT64 Jool
U:https://www.jool.mx
L:GPL-2.0-only
o:jool-tools
m:Jakub Jirutka <jakub@jirutka.cz>
t:1620480809
c:771b3b0910ea9c7736db6ca4ff5c37ca9cf9af0d
D:so:libc.musl-x86_64.so.1 so:libnl-3.so.200 so:libnl-genl-3.so.200
p:cmd:jool=4.1.5-r0 cmd:jool_siit=4.1.5-r0 cmd:joold=4.1.5-r0
</pre>

[[Category:Package Manager]] [[Category:Development]]

Apk internals

2022-07-18T01:38:18Z

Mcrute: Redirected page to Apk spec

#REDIRECT [[Apk_spec]]

Apkindex format

2022-07-18T01:38:16Z

Mcrute: Redirected page to Apk spec

#REDIRECT [[Apk_spec]]

Alpine package format

2022-07-18T01:38:06Z

Mcrute: Redirected page to Apk spec

#REDIRECT [[Apk_spec]]

Apk spec

2022-07-18T01:37:23Z

Mcrute:

{{Draft}}

For end-user facing documentation about apk, check out the [[Package_management]] page.

This page is an attempt to document the internal data structures of the apk package manager. The canonical implementation of the apk format is [https://gitlab.alpinelinux.org/alpine/apk-tools apk-tools] and much of this information is gleaned from reading the source code.

There are three generations of the APK data formats. Version 1 is deprecated and no longer used, version 2 is currently the main version in use by apk-tools, and version 3 is under development. This page mostly describes the data formats used in version 2.

= Package Format V2 =
== Tar Segments ==
Tar segments are a set of tar records. Normal tar files contain two null records at the end of the tar file to signal the end of the tarball. Tar segments are lacking these two records and can thus be concatenated before other tar files and will behave as one continuous tar file. The APK v2 package format makes use of both tar segments and tarballs.

Tar segments can be compressed using gzip compression. Gzip is a stream-based file format and multiple streams can be concatenated together. Most tooling will treat multiple gzip streams within a file as if it were a single stream. APK v2 files are aware of gzip streams and use them for file segmentation.

== Binary Format ==
APK v2 packages contain two tar segments followed by a tarball each in their own gzip stream (3 streams total). These streams contain the package signature, control data, and package data. The package data is a tarball of the files contained in a package laid out in a way that allows it to be unpacked at the filesystem root such that all files are placed in the correct location on the system. The control tar segment contains the package metadata along with any install scripts. The signature tar segment contains a single file that is a binary signature over the concatenated control segment and data tarball.

The signature file is a DER encoded PKCS1v15 RSA signature of the SHA1 hash of the concatenated control and data gzip streams. The filename has the format <tt>.SIGN.RSA.<key_name>.rsa.pub</tt> (for example <tt>.SIGN.RSA.alpine-devel@lists.alpinelinux.org-5261cecb.rsa.pub</tt>). This file is placed inside of a tar record with permissions 0644, uid 0, and gid 0. This tar record (lacking end-of-tar records) is gzip compressed, forming a signature tar segment, and is concatenated onto the front of the combined control and data segments. [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-sign.in abuild-sign] is responsible for generating these signature segments.

The control segment contains the package metadata in a <tt>.PKGINFO</tt> file as well as all of the scripts (if any) that are used by apk during installation and removal of the package. For historical reasons all files in the control tar segment are prefixed with a dot (<tt>.</tt>). The control segment is constructed by placing each file for the package into a tar record, concatenating those tar records, gzipping the tar records, and concatenating them onto the front of the data tarball. The SHA1 hash of this gzip stream is used as the checksum <tt>C:</tt> field in the APKINDEX file.

The data tarball is a standard gzipped tarball with extra PAX headers that contain the SHA1 hash of each file in the tar header for that file. The hash is contained in a header called <tt>APK-TOOLS.checksum.SHA1</tt>. Unlike the other tar streams this tarball does contain the two end-of-tar null records. It is always the final segment of an APK package. Hashes are added with the [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild-tar.c abuild-tar] tool.

== PKGINFO Format ==
The PKGINFO file contains the package metadata. This is a plain-text file similar to INI files. Lines that begin with <tt>#</tt> are comments and ignored. Unlike INI files the parsing format of this file is very strict. Each key-value pair must be separated by exactly one space, one equal sign, and one more space (<tt> = </tt>). Keys may be repeated in this file and should be treated as a list of values if repetitions are found.

The specification for what fields are valid in PKGINFO is largely defined by [https://gitlab.alpinelinux.org/alpine/abuild/-/blob/master/abuild.in abuild]. As of July 2022 the following fields are supported:

* <tt>pkgname</tt> - package name
* <tt>pkgver</tt> - package version
* <tt>pkgdesc</tt> - package description
* <tt>url</tt> - package url
* <tt>builddate</tt> - unix timestamp of the package build date/time
* <tt>packager</tt> - name (and typically email) of person who built the package
* <tt>size</tt> - the installed-size of the package
* <tt>arch</tt> - the architecture of the package (ex: x86_64)
* <tt>origin</tt> - the origin name of the package
* <tt>commit</tt> - the commit hash from which the package was built
* <tt>maintainer</tt> - name (and typically email) of the package maintainer
* <tt>replaces_priority</tt> - replaces priority field for package (integer)
* <tt>provider_priority</tt> - provider priority for the package (integer)
* <tt>license</tt> - license string for the package
* <tt>depend</tt> - dependencies for the package (repeated)
* <tt>replaces</tt> - packages this package replaces (repeated)
* <tt>provides</tt> - what this package provides (repeated)
* <tt>triggers</tt> - what packages this package triggers on (repeated)
* <tt>install_if</tt> - install this package if these packages are present (repeated)
* <tt>datahash</tt> - hex-encoded sha256 checksum of the data tarball

== Example of PKGINFO ==
<pre>
# Generated by abuild 3.9.0-r2
# using fakeroot version 1.25.3
# Wed Jul 6 19:09:49 UTC 2022
pkgname = busybox
pkgver = 1.35.0-r18
pkgdesc = Size optimized toolbox of many common UNIX utilities
url = https://busybox.net/
builddate = 1657134589
packager = Buildozer <alpine-devel@lists.alpinelinux.org>
size = 958464
arch = x86_64
origin = busybox
commit = 332d2fff53cd4537d415e15e55e8ceb6fe6eaedb
maintainer = Sören Tempel <soeren+alpine@soeren-tempel.net>
provider_priority = 100
license = GPL-2.0-only
replaces = busybox-initscripts
provides = /bin/sh
triggers = /bin /usr/bin /sbin /usr/sbin /lib/modules/*
# automatically detected:
provides = cmd:busybox=1.35.0-r18
provides = cmd:sh=1.35.0-r18
depend = so:libc.musl-x86_64.so.1
datahash = 7d3351ac6c3ebaf18182efb5390061f50d077ce5ade60a15909d91278f70ada7
</pre>

== Package Building Example ==
This is a set of commands to partially build a package. '''DO NOT DO THIS''', it's mainly an example to see how this all fits together. Use the official build tools to build packages.

<pre>
tar -c .PKGNIFO .pre-install | abuild-tar --cut | gzip -9 > $controldir/control.tar.gz
cd $pkgdir; tar -c * | abuild-tar --hash | gzip -9 > $controldir/data.tar.gz
cat $controldir/control.tar.gz $controldir/data.tar.gz > mypackage-1.0-r0.apk
</pre>

= Index Format V2 =
== Binary Format ==
The index is served as [http://dl-cdn.alpinelinux.org/alpine/edge/main/x86_64/APKINDEX.tar.gz APKINDEX.tar.gz] and is downloaded by apk to power the package database. The index is signed similarly to packages. The main difference between the index and packages is that the index file contains only two segments.

The signature segment is identical to a package segment and is concatenated, in its own gzip stream, to the beginning of the APKINDEX tarball.

The APKINDEX tarball contains two files: a DESCRIPTION file and an APKINDEX file. Each of these files is in their own tar record and the final record is followed by the standard end-of-tar null records. The DESCRIPTION file is a simple text file containing a description of the index (ex: <tt>community v20210212-7170-g5c9853dc69</tt>). The APKINDEX file is a text file containing records for each package in the repository in a text-based format. Each record is separated by a newline.

== APKINDEX Format ==
The APKINDEX file contains a set of records extracted from the PKGINFO file of each package in the repository. Each line is prefixed with a letter, colon, and is followed by the value of the field. Lines are newline (<tt>\n</tt>) terminated and there is one blank line between records for a package.

The <tt>apk_pkg_write_index_entry</tt> function of [https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/ff7c8f6ee9dfa2add57b88dc271f6711030e72a0/src/package.c#L905 package.c] defines the currently accepted fields. As of July 2022, these are:

* <tt>C:</tt> - file checksum, the SHA1 hash of the second gzip stream of the package, base64-encoded and prefixed with <tt>Q1</tt> to indicate SHA1
* <tt>P:</tt> - package name (corresponds to <tt>pkgname</tt> in PKGINFO)
* <tt>V:</tt> - package version (corresponds to <tt>pkgver</tt> in PKGINFO)
* <tt>A:</tt> - architecture (corresponds to <tt>arch</tt> in PKGINFO), optional
* <tt>S:</tt> - size of entire package, integer
* <tt>I:</tt> - installed size, integer (corresponds to <tt>size</tt> in PKGINFO)
* <tt>T:</tt> - description (corresponds to <tt>pkgdesc</tt> in PKGINFO)
* <tt>U:</tt> - url (corresponds to <tt>url</tt> in PKGINFO)
* <tt>L:</tt> - license (corresponds to <tt>license</tt> in PKGINFO)
* <tt>o:</tt> - origin (corresponds to <tt>origin</tt> in PKGINFO), optional
* <tt>m:</tt> - maintainer (corresponds to <tt>maintainer</tt> in PKGINFO), optional
* <tt>t:</tt> - build time (corresponds to <tt>builddate</tt> in PKGINFO), optional
* <tt>c:</tt> - commit (corresponds to <tt>commit</tt> in PKGINFO), optional
* <tt>k:</tt> - provider priority, integer (corresponds to <tt>provider_priority</tt> in PKGINFO), optional
* <tt>D:</tt> - dependencies (corresponds to <tt>depend</tt> in PKGINFO, concatenated by spaces into a single line)
* <tt>p:</tt> - provides (corresponds to <tt>provides</tt> in PKGINFO, concatenated by spaces into a single line)
* <tt>i:</tt> - install if (corresponds to <tt>install_if</tt> in PKGINFO, concatenated by spaces into a single line)

== Package Checksum Field ==
The package checksum field is the SHA1 hash of the second gzip stream (control stream) in the package. The binary hash digest is base64 encoded. This is prefixed with <tt>Q1</tt> to differentiate it from the MD5 hashes used in older index formats. It is not possible to compute this checksum with standard command line tools but the apk-tools can compute it in their <tt>index</tt> operation.

== Example APKINDEX Record ==
<pre>
C:Q1P4IRU/u5yB4CSnUEBRD1WWwajrY=
P:jool-tools
V:4.1.5-r0
A:x86_64
S:140605
I:434176
T:Userspace control tools for SIIT / NAT64 Jool
U:https://www.jool.mx
L:GPL-2.0-only
o:jool-tools
m:Jakub Jirutka <jakub@jirutka.cz>
t:1620480809
c:771b3b0910ea9c7736db6ca4ff5c37ca9cf9af0d
D:so:libc.musl-x86_64.so.1 so:libnl-3.so.200 so:libnl-genl-3.so.200
p:cmd:jool=4.1.5-r0 cmd:jool_siit=4.1.5-r0 cmd:joold=4.1.5-r0
</pre>

[[Category:Package Manager]] [[Category:Development]]

Migrating data

2020-06-01T01:52:32Z

Mcrute: Remove tools that don't work on Alpine

We compare several methods for copying/migrating large amounts of data. The aim is to preserve all file permissions and metadata, which will require root access on both source and target filesystems.

We include information about some options that aren't available on BusyBox's implementation of these tools, because you may be copying data to/from systems with other Unix tools installed, which do provide those options.

{{Warning| None of the methods below will copy the [[Partitioning and Bootmanagers|bootmanager]] (ext/syslinux, grub, etc) to a new drive; you'll have to install it there explicitly.}}

== Bit-copying methods ==

If your source data occupies an entire partition (that is, nothing else resides on that partition), and your target partition is the same size or larger, and you're willing to have it use the same filesystem as the source, then one option is to bit-copy the whole source partition:

Boot from LiveCD and make sure that neither /dev/<var>sourcepart</var> nor /dev/<var>targetpart</var> are mounted.

Example:
{{Cmd| dd bs{{=}}10M if{{=}}/dev/<var>sourcepart</var> of{{=}}/dev/<var>targetpart</var>}}

This also copies unoccupied blocks on the <var>sourcepart</var> to the <var>targetpart</var>. A more optimal variant is to use [http://www.partimage.org/Main_Page partimage], which copies only the used portions of the sourcepart. One limitation is that currently (v0.6.9, released July 2010, current as of Mar 2012) partimage doesn't support ext4 format.

{{Todo| Generally the target partition will have same partition type (e.g., Linux 0x83) as the source; but is this strictly necessary?}}

Once you've finished, the filesystem on <var>targetpart</var> may occupy less than the total space available on <var>targetpart</var>. You may want to use [[Filesystems|a tool like resize2fs]] to grow the partition.

== Higher-level methods ==

The other options we'll discuss suppose that the target filesystem is already formatted and mounted as {{Path|/target}}.

If you're proposing to copy volatile data from a running system, you may want to copy from a lvm snapshot, or do the copy from single-user mode, or from a LiveCD or other boot medium so that your source data is no longer in use during the copy. If you're just proposing to copy from {{Path|/home}} or such, no such precautions are necessary.

If you're copying a complete filesystem, you'll probably only want to copy the {{Path|/sys}} and {{Path|/proc}} mountpoints, but not their current contents. Remember to copy {{Path|/boot}} and {{Path|/dev}} and {{Path|/tmp}}, the last two of which may require special attention. Pay attention to other mountpoints (such as {{Path|/mnt}}, {{Path|/media}}) and RAM-based filesystems (such as {{Path|/run}} and {{Path|/lib/rc/init.d}}).

{{Todo|The preceding comments assume you know what you are doing, and only serve as reminders. Would be good to link to a fuller explanation.}}

{{Note| /dev is tricky nowadays on some systems since it's hidden by udev. Here's a possible solution:
{{Cmd|mkdir /tmp/dev
mount --move /dev /tmp/dev
<i>copy /dev to /mnt/sdb5 using one of the methods elsewhere on this page</i>
mount --move /tmp/dev /dev
rmdir /tmp/dev}}
}}

=== cp ===
Depending on what system you're working with, and what kind of data you're copying, you ''may'' be able to do a satisfactory ''local'' copy using just <code>cp</code>. But there are a number of limits here that prevent this from being a general solution.

The Gnu implementation of <code>cp</code> provides the following options (among others):

* <code>-a</code>: same as <code>-RP -p/--preserve=mode,ownership,timestamps,links,context,xattr</code>
* <code>-v</code>: verbose
* <code>-l</code>: make hard links instead of copying
* <code>-s</code>: make symlinks instead of copying (source file names must be absolute)
* <code>--sparse=always</code>
* <code>-u</code>, <code>--update</code>: don't copy files or symlinks over an existing file unless its mtime is older
* <code>-x</code>, <code>--one-file-system</code>: skip subdirectories on different volumes

The BusyBox version provides all of the Gnu options except <code>--sparse</code> and <code>-u</code> and <code>-x</code>, and may lack the <code>context</code> and <code>xattr</code> options to <code>--preserve</code>. It will preserve hard links: if {{Path|/source/A}} and {{Path|/source/B}} are hard links, {{Path|/target/A}} and {{Path|/target/B}} will also be hard links with each other (though not with the originals). BusyBox's <code>cp</code> seems to silently ignore the <code>-v</code> option.

The FreeBSD implementation provides all of these except <code>-s</code> and <code>--sparse</code> and <code>-u</code>, and the <code>--preserve</code> options from <code>links</code> on: in particular, it always copies hard links as separate files. The Mac version shares the limitations of the FreeBSD version, and additionally lacks <code>-x</code>. Also, on the Mac version the <code>-a</code> shortcut isn't available, you must explicitly say <code>-RPp</code>.

{{Note| As just explained, some of these implementations will break hard links.}}

{{Warning| Avoid the lower-case <code>-r</code> option. Its behavior varies between implementations, and may diverge from what you want for special files or symlinks.}}

=== tar ===

Options recognized by BusyBox and other implementations:

* <code>-c</code>: create a new archive, records existing perms and mtimes, symbolic owner/group (can specify <code>--numeric-owner</code>), hard links among archive elements
* <code>-x</code>: extract files from an archive, uses current owner/group unless root, subtracts current umask unless root or <code>-p</code>, when extracting hard links must include first
* <code>-t</code>: list the contents of an archive

* <code>-f <var>archive</var></code>: use <code>-f -</code> for stdin/stdout
: {{Note| On some but not all implementations, the archive defaults to stdin/stdout when <code>-f</code> is not supplied. It's most portable to always declare this explicitly.}}
* <code>-O</code>: extract files to standard output
* <code>-C <var>dir</var></code>: change to directory <var>dir</var>, when creating this is order-sensitive
* <code>-v</code>: verbose

* <code>-z</code>: compress archive with gzip
* <code>-j</code>: compress archive with bzip2
* <code>-Z</code>: compress archive with compress

* <code>-h</code>: follow symlinks, archive and extract the files they point to (like <code>-L</code> in other programs)
* <code>-m</code>: don't extract file's original mtime, leave touched with the extraction time
* <code>-T <var>file</var></code>: get names to extract or create from <var>file</var>, which can also include "-C/etc" lines
* <code>--exclude=<var>pattern</var></code>: exclude specified glob <var>pattern</var>s
* <code>-X <var>file</var></code>: exclude glob patterns listed in <var>file</var>
: {{Note| Default globbing rules: (i) matches after any /, not just at ^; (ii) case-sensitive; (iii) wildcards match /}}

The following common options are allegedly also honored by BusyBox (according to
comments in its source) but aren't declared in its <code>--help</code>:

* <code>-o</code>, <code>--no-same-owner</code>: (BSD doesn't recognize a long option) extract as yourself (default for ordinary users)

* <code>-p</code>, <code>--same-permissions</code>: (BSD doesn't recognize this long option) extract permissions verbatim, instead of subtracting current umask (usually default for root)

* <code>--no-same-permissions</code>: subtract umask from extracted permissions (default for ordinary users)

* <code>--numeric-owner</code>: create using numbers for user/groups

* <code>-k</code>: don't replace existing files when extracting

<ul>
<li><code>--overwrite</code>: (not on BSD) more aggressively overwrite existing files and directory metadata when extracting<br />
tar's default behavior is to first remove existing files (otherwise, all their hardlinks would be modified) and symlinks.
If an existing dir is nonempty, its existing contents won't be removed; tar will instead just update the dir's metadata.
<code>--overwrite</code> instead ''follows'' existing symlinks, and removes anything that impedes extraction except non-empty dirs.
(Gnu's <code>--recursive-unlink</code> removes even those, may replace them with files or symlinks.)
</ul>

Gnu options (also Mac and FreeBSD unless noted) also include:

* <code>--one-file-system</code>: when creating archive, stay on the volume(s) where the specified roots are located, and don't cross mountpoints

* <code>--null</code>: <code>-T</code> reads null-terminated names, ignores "-C/etc" entries

* <code>-l</code>, <code>--check-links</code>: (BSD for <code>-c</code> only) when creating or extracting, print a message if not all hard links are processed

* <code>-S</code>, <code>--sparse</code>: (not on BSD) when creating archive, handle sparse files efficiently

Some Gnu tar formats:
* in future, will default to <code>-H posix/pax</code>, which is the POSIX.1-2001 format
* v.1.13 - current (v1.26 released Mar 2011, still current in Mar 2012): defaults to <code>-H gnu</code>, which is not very different from the <code>-H oldgnu</code> format.
* <code>-H ustar</code> is the portable POSIX.1-1988 format, some limits including max 256 char filenames, max 8G filesize; supports special files, but not sparse files; also seems not to support xattrs and ACLs (other tar formats are meant to handle ACLs, but see [[#tar_acl|comments below]])

Example of local copy:

{{Cmd|sudo tar -cf- [--one-file-system] /source {{!}} sudo tar -xp[v]f- -C /target}}

The optional flags are:
* <code>--one-file-system</code>: Stay on the volume where {{Path|/source}} is located. This may or may not be the behavior you desire. Note that this feature isn't available on BusyBox <code>tar</code> in any case, though it is available in some other <code>tar</code> implementations.
* <code>-v</code>: verbose

Example of using tar in cpio-style (for which, see below):
{{Cmd|cd /source
sudo find . [-xdev] -depth [-print0] {{!}} sudo tar -cf- -T- [--null] {{!}} sudo tar -xp[v]f- -C /target}}

The next examples add the <code>-z</code> flag, to compress output using gzip. If you have a fast connection, you may omit this; alternatively, you may use <code>-j</code> for bzip2. Example of copying to remote machine:
{{Cmd|sudo tar -czf- [--one-file-system] /source {{!}} ssh root@machine "tar -xpz[v]f- -C /target"}}

Example of copying from remote machine:
{{Cmd|ssh root@machine "tar -czf- [--one-file-system] /source" {{!}} sudo tar -xpz[v]f- -C /target}}

=== cpio ===

cpio operates in one of these four modes:
* <code>-o -H newc</code>: create archive on stdout (or <code>-F <var>archive</var></code>), including filenames supplied from stdin (like <code>tar -cf- -T-</code>)
: <code>-H newc</code>: (on Mac, <code>-H sv4cpio</code> and/or <code>-c</code>) specifies to use SVR4 ASCII format, this is required to create archives in BusyBox
* <code>-i</code>: receive archive from stdin (or <code>-F <var>archive</var></code>), extract specified patterns underneath curdir (like <code>tar -xf-</code>)
* <code>-t</code>: (on some systems, may need <code>-it</code>) receive archive from stdin (or <code>-F <var>archive</var></code>), list contents to stdout (like <code>tar -tf-</code>)
* <code>-p /target</code>: receive list of files from stdin (or <code>-F <var>archive</var></code>), copy them underneath {{Path|/target}} (like <code>tar -cf- -T- | tar -xf- -C /target</code>)

tar vs cpio:
* if you have includes/excludes, it's easier to use <code>find</code> than unfamiliar and non-portable options of Gnu tar
* cpio handles special files (block and char devices, fifos, etc); traditional tar (<code>-H v7</code> format) didn't, also didn't store symbolic owner/group
* when extracting hard links, tar must always include the first
* tar and the <code>-H newc</code> cpio format handle 32-bit inodes; though some older cpio formats didn't
* the <code>-H newc</code> cpio format has max 1024 char filenames (though Gnu cpio can handle arbitrary lengths) and max 4G filesize; the portable <code>-H ustar</code> tar format has max 256 char filenames and max 8G filesize
* in some implementations, cpio wouldn't copy files over 2G properly; don't have details about which implementations/versions are so limited
* <span id="tar_acl">some tar formats are supposed to handle acls, however [http://unix.stackexchange.com/q/391/4801 in practice this doesn't seem very reliable]; not sure what the situation is with cpio; rsync is known to work well for such cases</span>



Example of local copy:
{{Cmd|cd /source
sudo find . [-xdev] -depth [\! -path ./lost+found] [-print0] {{!}} sudo cpio -pdm[vul] [-0] /target}}

The flags are:
* <code>-xdev</code>: stay on the volume where {{Path|/source}} is located; this may or may not be the behavior you desire
* <code>\! -path ./lost+found</code>: omit empty {{Path|./lost+found}}; <code>!</code> only needs to be escaped for some shells, and this sequence may be repeated
* <code>-print0</code> ... <code>-0</code>: permits processing filenames with embedded <code>\n</code>s, not available on BusyBox <code>cpio</code> or Mac
* <code>-d</code>: create leading directories, like <code>mkdir -p</code>
* <code>-m</code>: preserve file's original mtime
* <code>-v</code>: verbose
* <code>-u</code>: overwrite existing files, even if they're newer than files being copied
* <code>-l</code>: create hard links instead of copying. This may or may not be the behavior you desire. Note that this feature isn't available on BusyBox <code>cpio</code> in any case, though it is available in some other <code>cpio</code> implementations.

Other flags are:
* <code>-oA -F <var>archive</var></code>: append to existing archive (not on BusyBox or BSD)
* <code>-L</code>: follow symlinks (not on BusyBox)
* <code>--sparse</code>: write files with blocks of zeros as sparse files (Gnu only)

Example of copying to remote machine:
{{Cmd|cd /source
sudo find . [-xdev] -depth [-print0] {{!}} sudo cpio -o -H newc [-0] {{!}} [gzip -3 {{!}}] ssh root@machine "cd /target; gunzip {{!}} cpio -idm[vu]"}}

=== rsync ===

<code>rsync</code> must be installed on both source and target machines. Example:

{{Cmd|rsync -a[vzx] --delete [--numeric-ids] [-HAX] [--exclude{{=}}/proc --exclude{{=}}/dev --exclude{{=}}/sys] /source/ [root@machine:]/target}}

Note that the trailing {{Path|/}} on {{Path|/source/}} is essential, if you want {{Path|/target}} to end up a clone of {{Path|/source}}.

The flags are:
* <code>-v</code>: verbose
* <code>-z</code>: compress data during transfer
* <code>-x</code> or <code>--one-file-system</code>: skip subdirectories on different volumes
* <code>--numeric-ids</code>: rsync's default is to use symbolic user/groupnames
* <code>-H</code> or <code>--hard-links</code>: preserve hard links in copied data, this can be expensive, and won't break existing hard links on {{Path|/target}} unless rsync needed to write updated data to some of the linked files
* <code>-A</code> or <code>--acls</code>: preserve ACLs
* <code>-X</code> or <code>--xattrs</code>: preserve xattrs

* <code>-8</code> or <code>--8-bit-output</code>: don't escape chars in filename even if they're invalid in current locale
* <code>--partial</code>: keep partially-transferred files
<ul>
<li><code>--inplace</code>: overwrite existing files in place<br />
rsync's default behavior is to write new files and move them into place when complete; this breaks any hard links the existing file may have had and makes a copy-on-write filesystem see the file as entirely new. <code>--inplace</code> specifies an alternate behavior, of writing updated data directly into the existing file.
</ul>
* <code>-h</code> or <code>--human-readable</code>: report statistics in prettier form (use once for powers of 10, twice for powers of 2)
* <code>--progress</code>: show progress of each file transferred, implies <code>-v</code>
* <code>-n</code> or <code>--dry-run</code>

[http://rsync.samba.org/ftp/rsync/rsync.html rsync's full manpage]

rsync's advantages:
* like cp, can handle ACLs and xattrs; many tar and cpio implementations don't
* like cpio, can optionally work over the network
* can resume incomplete transfers, and can do incremental updates if the data is being copied multiple times

Including/excluding from transfers:
* <code>--include=<var>pattern</var>[/]</code>
* <code>--exclude=<var>pattern</var>[/]</code>
Each of these may be repeated. The <var>pattern</var>s may contain <code>*</code>, <code>**</code>, <code>?</code>, <code>[a-z]</code>.
A trailing <code><var>dir</var>/***</code> matches both <var>dir</var> and its contents.
If <var>pattern</var> contains a <code>**</code> or (non-trailing) <code>/</code>, it matches against the full path from {{Path|/target}}, else it only matches against the final path element.

=== lvm ===

"This is one reason I like LVM. Just add the new disk to the volume group, pvmove the logical volumes from the old to new disk, [wait a bit], remove the old disk from the volume group, and then from the system. If it's your boot disk you're replacing then you also need to update your boot loader." 

See [[Setting up Logical Volumes with LVM]].

=== Other tools ===

{{Cmd|cd /source
pax -pe -rw -v [-X] -YZ . /target}}

[[Category:Installation]]
[[Category:Storage]]

Create UEFI boot USB

2018-04-29T19:18:56Z

Mcrute: Fix up command blocks

This article explains how to create an UEFI boot USB with parted and gummiboot.

In this example we will use {{Path|/dev/sdX}}. This will be different depending on your system.

== Create GPT boot partition ==

Install {{Pkg|parted}}
{{Cmd | apk add parted }}

Create a single UEFI boot partitions.
{{warning| this will erase all content of your {{Path|/dev/sdX}}. Make sure that you use correct device}}

{{Cmd | parted --script /dev/sdX mklabel gpt
parted --script --align{{=}}optimal /dev/sdX mkpart ESP fat32 1MiB 512MiB
parted --script /dev/sdX set 1 boot on }}

== Create fat32 filesystem ==

Create a fat32 system with the name `Alpine`.

{{Cmd | mkfs.vfat -n Alpine /dev/sdX1 }}

== Copy content of ISO image to filesystem ==

It is possible to mount the iso image and copy files with {{codeline|cp}} or {{codeline|rsync}} and it is also possible to use {{codeline|7z}} to extract content from the iso. In this example I will use the {{codeline|uniso}} utility from {{Pkg|alpine-conf}} package.

{{Cmd | mount -t vfat /dev/sdX1 /mnt
cd /mnt && uniso < /path/to/alpine-3.4.0-x86_64.iso }}

== Copy gummiboot efi binary ==

{{note| Gummiboot has been forked in Alpine Linux since it has been [https://www.freedesktop.org/wiki/Software/systemd/systemd-boot/ consumed by systemd upstream].}}

UEFI will look for a {{Path|EFI/bootx64.efi}} as a fallback efi loader. We copy {{Path|gummibootx64.efi}} to this location.

{{Cmd | mkdir -p /mnt/EFI/Boot
apk add gummiboot
cp /usr/lib/gummiboot/gummibootx64.efi /mnt/EFI/Boot/bootx64.efi }}

== Create configuration files for boot loader ==

We need create some configuration files for gummiboot.
{{Cmd | mkdir -p loader/entries }}

{{Cat | loader/loader.conf |default alpine
timeout 4
}}

The options are taken from {{Path|boot/syslinux/syslinux.cfg}}

{{Cat | loader/entries/alpine.conf |title Alpine Linux
linux /boot/vmlinuz-hardened
initrd /boot/initramfs-hardened
options modloop{{=}}/boot/modloop-hardened modules{{=}}loop,squashfs,sd-mod,usb-storage quiet
}}

Finally umount the disk
{{Cmd | cd ~ && umount /mnt}}

[[Category:Installation]]

How to setup a Alpine Linux mirror

2018-01-03T16:12:41Z

Mcrute:

== Introduction ==
This document describes how to set up an Alpine Linux mirror and make it available via http and rsync.

We will:
* create the dir where we have the mirror
* set up a cron job to sync with master mirror every hour
* set up lighttpd for http access
* set up rsync so other mirrors can rsync from you

Make sure that you have enough disk space; each v3.x branch has around 20 GiB.

Current (2018-01-03) disk usage:

{|class="wikitable"
!edge
!v2.4
!v2.5
!v2.6
!v2.7
!v3.0
!v3.1
!v3.2
!v3.3
!v3.4
!v3.5
!v3.6
!v3.7
!Total
|-
|71.3G
|18.8G
|10.4G
|13.0G
|16.5G
|16.5G
|17.5G
|14.5G
|19.0G
|23.2G
|32.5G
|34.4G
|71.3G
|'''358.9G'''
|}

== Setting up the cron job ==
Install rsync which will be used to sync from the master mirror.
{{Cmd|apk add rsync}}

Save the following file as ''/etc/periodic/hourly/alpine-mirror''
<pre>
#!/bin/sh

# make sure we never run 2 rsync at the same time
lockfile="/tmp/alpine-mirror.lock"
if [ -z "$flock" ] ; then
exec env flock=1 flock -n $lockfile "$0" "$@"
fi

src=rsync://rsync.alpinelinux.org/alpine/
dest=/var/www/localhost/htdocs/alpine/

# uncomment this to exclude old v2.x branches
#exclude="--exclude v2.*"

mkdir -p "$dest"
/usr/bin/rsync \
--archive \
--update \
--hard-links \
--delete \
--delete-after \
--delay-updates \
--timeout=600 \
$exclude \
"$src" "$dest"

</pre>

(or use [https://gist.github.com/jirutka/288c6fff7c0b8a835d143686207316be this script])

Make it executable:
{{Cmd|<nowiki>chmod +x /etc/periodic/hourly/alpine-mirror</nowiki>}}

Now it will sync every hour. (given cron runs)

== Setting up HTTP access via lighttpd ==

Install the lighttpd server
{{Cmd|apk add lighttpd}}

Enable dir listings by uncommenting the following line in ''/etc/lighttpd/lighttpd.conf'':
dir-listing.activate = "enable"

Also set cache-control to force cache revalidate every 30 mins. Uncomment mod_setenv in ''/etc/lighttpd/lighttpd.conf'':
"mod_setenv",

Add also the following lines to ''/etc/lighttpd/lighttpd.conf'':
setenv.add-response-header += (
"Cache-Control" => "must-revalidate"
)

Start lighttpd and make it start at boot:
{{Cmd|rc-service lighttpd start
rc-update add lighttpd}}

{{Note|You may wish to consider [[Darkhttpd]] as an alternative to [[Lighttpd]]

If so, simply install, start and auto-start the webserver:

{{Cmd|apk add darkhttpd && rc-service darkhttpd start && rc-update add darkhttpd}}

Darkhttpd will, by default, offer directory listings and serve data from /var/www/localhost/htdocs/

See the main article on [[Darkhttpd]] for more configuration options}}

== Setting up rsyncd ==
Add the following lines to ''/etc/rsyncd.conf'':
<pre>
[alpine]
path = /var/www/localhost/htdocs/alpine
comment = My Alpine Linux Mirror
</pre>

Optionally set a bandwidth limit in ''/etc/conf.d/rsyncd''. In this example we limit to 500Kbytes/s (approx 5Mbit/s)
<pre>
RSYNC_OPTS="--bwlimit=500"
</pre>

== Mirror statistics ==

Simple bandwidth statistics can be generated with vnstat.

{{Cmd|apk add vnstat}}

edit /etc/vnstat.conf and replace the interface name with the appropriate one.

Start vnstatd

{{Cmd|/etc/init.d/vnstatd start }}

copy the following script to /etc/periodic/15min/stats and make sure your crond is running.
please not that heredoc should be tab indented or the script will fail. A working copy can be found here: http://tpaste.us/RrMv

<pre>
#!/bin/sh

output="/var/www/localhost/htdocs/.stats"
nic="eth0"

generate_index() {
cat <<-EOF
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta http-equiv="cache-control" content=no-cache">
<meta http-equiv="refresh" content="3000">
<title>Alpine Linux mirror statistics</title>
</head>
<body>
<table border="0">
<tr><td><img src="summary.png" alt="summary"></td><td><img src="hours.png" alt="hours"></td></tr>
<tr><td rowspan="2"><img src="days.png" alt="days"></td><td><img src="top10.png" alt="top10"></td></tr>
<tr><td><img src="months.png" alt="months"></td></tr>
</table>
</body>
</html>
EOF
}

if [ ! -f "$output"/index.html ]; then
mkdir -p $output
generate_index > "$output"/index.html
fi

for type in hours days months top10 summary hsummary vsummary; do
vnstati --${type} -i $nic -o $output/${type}.png
done
</pre>

== Update mirror from mqtt ==

If you want your mirror to be really uptodate compared to our master mirror you can subscribe to Alpine Linux message server "msg.alpinelinux.org" and check for upload messages.
Add mqtt-exec to be able to execute processes when specific topics are being send.

{{Cmd| apk add mqtt-exec}}

mqtt-exec supports running multiple time so we need to setup a specific config.

{{Cmd| ln -s /etc/init.d/mqtt-exec /etc/init.d/mqtt-exec.sync-mirror}}

{{Cmd| ln -s /etc/conf.d/mqtt-exec /etc/conf.d/mqtt-exec.sync-mirror}}

edit /etc/conf.d/mqtt-exec.sync-mirror

<pre>
mqtt_topics="rsync/rsync.alpinelinux.org/#"
exec_user="buildozer"
exec_command="/usr/local/bin/sync-mirror"
</pre>

Copy the following file to /usr/local/bin/sync-mirror and make it executable (dont forget to update the variables).

<pre>
#!/bin/sh

src="rsync://rsync.alpinelinux.org/alpine/"
dest="/var/www/localhost/htdocs/alpine/"
lock="/tmp/sync-mirror.lock"
topic="$1"
dir="$2"

[ -z "$flock" ] && exec env flock=1 flock $lock $0 "$@"

if [ -n "$dir" ] && [ -d "$dest/${dir%/*}" ]; then
logger "Syncing directory: $dir"
src="${src}${dir%/}/"
dest="${dest}${dir%/}/"
else
logger "Syncing all directories"
fi

/usr/bin/rsync \
--archive \
--update \
--verbose \
--progress \
--timeout=600 \
--delay-updates \
--delete-after \
"$src" \
"$dest"
</pre>

And finally start mqtt-exec and let it listen on msg.alpinelinux.org

{{Cmd|/etc/init.d/mqtt-exec.sync-mirror start}}

To make sure you are not missing any packages (in case something goes wrong with MQTT subscription) you can periodically sync all directories by adding the script to cron.

{{Cmd|ln -s /usr/local/bin/sync-mirror /etc/periodic/hourly/sync-mirror}}

Now watch your syslog as it should tell you when it will update directories in your local mirror.

[[Category:Server]]
[[Category:Package Manager]]

APKBUILD Reference

2017-12-29T04:21:19Z

Mcrute:

APKBUILDs are the scripts that are created in order to build Alpine packages using the [[abuild]] tool.

See [[aports]] for details on Alpine's official ports repository.

This page is intended to serve as a reference for creating APKBUILDs; if this is your first time creating a package for Alpine Linux, please see [[Creating an Alpine package]].

= Legend =
The following notes will assist you in understanding this document.

In description text:
* If a variable is not prefixed with a ''$'', it will be represented by italics (i.e., ''srcdir'' ).
* Functions will also be represented by italics, but will also end with a pair of parentheses (i.e., ''build()'' ).
* Shell commands will be represented <code>like this</code>.

= Variables =
{{Note|Variables that contain a path (e.g. ''$srcdir'' and ''$pkgdir'') should always be quoted using double quotes (i.e., ''"$srcdir"''). This is done to prevent things from breaking, should the user have the APKBUILD in a directory path that contains spaces.}}
{{Note|All arbitrary variable and function names should be prefixed with an underscore character ( _ ) to avoid name clashes with the internals of abuild (for example, ''_luaversions'').}}

== abuild-defined variables ==
The following variables are defined by abuild:

==== startdir ====
: The directory where the APKBUILD script is.
==== srcdir ====
: The directory where sources, from the ''source'' variable, are downloaded to and unpacked to.
==== pkgdir ====
: This directory should receive the files for the main package. For example, a normal [http://en.wikipedia.org/wiki/GNU_build_system autotools] package would have <code>make DESTDIR="$pkgdir" install</code> in the ''package()'' function.
==== subpkgdir ====
: This directory should receive the files for a subpackage. This variable should only be used from subpackage functions.
==== builddir ====
: This variable should point to the directory inside the ''srcdir'' where the main package source is unpacked. This is typically ''$srcdir/$pkgname-$pkgver''. It’s used by the default ''prepare()'' function as a working directory when applying patches.

== User-defined variables ==
The following variables should be defined by the user:
==== arch ====
: Package architecture(s) to build for. Can be one of: '''[[x86]], [[x86_64]], [[armhf]], [[aarch64]], [[ppc64le]], [[s390x]], all''', or '''noarch''', where '''all''' means all architectures, and '''noarch''' means it's architecture-independent (e.g., a pure-python package).
: {{Tip|To determine if your APKBUILD can use '''noarch''': First specify '''all''' and then build the package by executing <code>abuild -r</code>. Watch the output towards the end for warnings saying that '''noarch''' can be used. If the main package and all subpackages, if you have any subpackages, give a warning saying that '''noarch''' can be used, then you can use '''noarch'''.}}

==== depends ====
: Run-time dependency package(s) that are not shared-object dependencies. Shared objects dependencies are auto-detected and should not be specified here.
==== depends_dev ====
: Run-time dependency package(s) for the '''$pkgname-dev''' subpackage.

: {{Note|From ncopa on IRC: To find out if you need to add a package to depends_dev have a look at *requires* in usr/lib/pkgconfig/*.pc. With libtool it gets more complicated, but we should delete the .la files. Also check if there are any /usr/bin/*-configure #!/bin/bash #!/usr/bin/perl or Python. Sometimes scripts or similar are generated at build time (i.e autoconf automake) then you normally don't need add those to depends_dev. You can also just add all -dev makedepends to depends_dev but it will slow the build process a little bit (more build dependencies).}}
==== giturl ====
:Git repository from which <code>abuild checkout</code> checks out. You can checkout a specific branch in git by adding <code>-b $branch</code>.
==== install ====
: There are 6 different types of install scripts. Install scripts are named '''$pkgname.action''', where '''action''' can be: '''pre-install, post-install, pre-upgrade, post-upgrade, pre-deinstall''', or '''post-deinstall'''. For example, if ''pkgname'' is set to '''mypackage''' and ''install'' is set to '''$pkgname.post-install''', then a script named '''mypackage.post-install''' must exist along-side the APKBUILD.

: First, a few notes regarding install scripts:
<blockquote>{{Note|When using install scripts, ''$install'' should be included in ''source'' so that checksums can be generated and used for the install scripts specified in ''install''. For example:
<pre>
install="$pkgname.pre-install $pkgname.post-install"
source="http://....
$install"
</pre>}}
{{Note|Always use <code>/bin/sh</code> for the command-line interpreter on the [http://en.wikipedia.org/wiki/Shebang_%28Unix%29 shebang line] of your install scripts.}}</blockquote>

The following are the different types of install scripts in detail:

===== $pkgname.pre-install =====
: This script is executed ''before installing'' the package. Typical use is when the package needs a group and a user to be created. For example:
<blockquote><pre>
#!/bin/sh

addgroup -S clamav 2>/dev/null
adduser -S -D -H -s /bin/false -G clamav -g clamav clamav 2>/dev/null

exit 0
</pre>
{{Note|If the script exits with a failure (e.g., if the user already exists), the package will not be installed and <code>apk</code> will exit with failure, hence the <code>exit 0</code> at the end.}}</blockquote>

===== $pkgname.post-install =====
: This script is executed ''after installing'' the package.

===== $pkgname.pre-upgrade =====
: This script is executed ''before upgrading/downgrading/reinstalling'' the package. Note that exiting with failure will not cause apk to exit with failure, but will mark the package as broken.

===== $pkgname.post-upgrade =====
: This script is executed ''after upgrading/downgrading/reinstalling'' the package.

===== $pkgname.pre-deinstall =====
: This script is executed ''before uninstalling'' the package.
: {{Note|If the script exits with failure, <code>apk</code> will not uninstall the package.}}

===== $pkgname.post-deinstall =====
: This script is executed ''after uninstalling'' the package.

==== install_if ====
:install_if can be used when a package needs to be installed when some packages are already installed or are in the dependency tree. It works in reverse to the ''recommends'' feature, that other package managers provide.

: '''Example:''' When package <code>A</code> has <code>install_if="B C"</code>, and the user runs <code>apk add B C</code>, then package <code>A</code> will get automatically installed.

: '''Example 2:''' A real use-case in Alpine is open-vm-tools. Currently it contains the userspace tools and separate packages for the kernel modules (grsec and vserver). When we install the userspace tools, apk should automatically install the correct kernel modules and will need to figure out for which kernel. This is where install_if jumps in. For any of the kernel modules package we would use:

:<pre>install_if="linux-${_flavor}=${_kernelver} open-vm-tools"</pre>

:This will automatically install the package when the specified packages are installed or are in dependency tree.

==== license ====
: License(s) for the package, for example <code>GPL3+</code>, <code>BSD</code> or <code>MIT</code> [[Creating_an_Alpine_package#license|(details)]].

==== makedepends ====
: Build-time dependency package(s).
==== md5sums/sha256sums/sha512sums ====
: Checksums for the files/URLs listed in ''source''. The checksums are normally generated and updated by executing <code>abuild checksum</code> and should be the last item in the APKBUILD.

New packages should prefer sha512sums and avoid md5sums.
==== options ====
: Build-time options for the package.

: {| class="wikitable"
! Option
! Description
|-
| <code>!archcheck</code>
| Do not try to verify that the architecture of the binary files is the same architecture as abuild should build for. One example where it makes sense to set this are packages with firmware files, that get executed on another CPU (such as WiFi firmware).
|-
| <code>!check</code>
| Do not try to run the <code>check()</code> function. Please always add a short comment after the <code>!check</code> about why it's disabled. [https://github.com/alpinelinux/aports/pull/2322#discussion_r142545300] Creating a very simple check function, that calls <code>program --version</code> is better than disabling tests completely. [https://github.com/alpinelinux/aports/pull/2322#discussion_r142543002]
|-
| <code>!strip</code>
| Avoid stripping symbols from binaries.
|-
| <code>suid</code>
| Allow [https://en.wikipedia.org/wiki/Setuid setuid] binaries.
|-
| <code>!tracedeps</code>
| Do not automatically find dependencies (e.g. by using <code>ldd</code> to find dynamic libraries, which the resulting binary links against).
|}

==== pkgdesc ====
: A brief, one-line description of what the package does.

: Here's an example from the OpenSSH client package:
: <pre>pkgdesc="Port of OpenBSD's free SSH release - client"</pre>
==== pkggroups ====
: System group(s) to be created during build-time. System group(s) should also be created in the '''[[APKBUILD Reference#.24pkgname.pre-install|$pkgname.pre-install]]''' script, so that the system group(s) are also created prior to package installation for run-time use.
==== pkgname ====
: The name of the package. All letters should be lowercase.
: {{Note|When creating an APKBUILD of a module or library for another package, we use some common package prefixes, such as: ''lua-'', ''perl-'', ''php-'', and ''py-''. Search aports for other common prefixes.}}

==== pkgrel ====
: Alpine package release number. Starts at 0 (zero). Always increment ''pkgrel'' when making updates to an aport; reset ''pkgrel'' to 0 (zero) when incrementing ''pkgver''.
==== pkgusers ====
: System user(s) to be created during build-time. System user(s) should also be created in the '''[[APKBUILD Reference#.24pkgname.pre-install|$pkgname.pre-install]]''' script, so that the system user(s) are also created prior to package installation for run-time use.
==== pkgver ====
: The version of the software being packaged.
==== provides ====
: List of package names (and optionally version info) this package provides.

: If package with a version is provided (provides='foo=1.2') apk will consider it as an alternate name and it will automatically consider the package for installation by the alternate name, and conflict with other packages having the same name, or provides.

: If version is not provided (provides='foo'), apk will consider it as virtual package name. Several package with same non-versioned provides can be installed simultaneously. However, none of them will be installed by default when requested by the virtual name - instead, error message is given and user is asked to choose which package providing the virtual name should be installed.
==== replaces ====
: Package(s) that this package replaces. This package will "take over" files owned by packages listed in the ''replaces'' variable. This is useful when files move from one package to another, or when a package gets renamed.
==== replaces_priority ====
: The priority of the replaces. If multiple packages replace each other, then will the package with highest ''replaces_priority'' win.
==== source ====
: The source variable is not only used to list the remote source files to fetch, it is also used to list the local files that abuild will need in order to build the apk. Examples of such local files include: init.d files, conf.d files, install files (see [[APKBUILD Reference#install|install variable]]), patches, and all other necessary files.

: Here are few things to note:

:* When you are finished adding local and/or remote files to ''source'', you can execute the following command to add their checksums to the APKBUILD file:
:: {{Cmd|abuild checksum}}
:: {{Note|When later updating the content of ''source'', or updating a file that is listed in ''source'', you must also update their checksums again with the same command.}}

:* When the remote file is hosted at SourceForge, it's best to specify the special mirrors link used by SourceForge:
:: <pre>http://downloads.sourceforge.net/$pkgname/$pkgname-$pkgver.tar.gz</pre>
:: (or similar depending on the package).

:* You can set target filename (eg 'save as...') by prefixing the URI with ''filename::''. This is useful when the remote filename is not specified in the URI (ie, does not end in '/software-1.0.tar.gz'), such as:
:: <pre>http://oss.example.org/?get=software&ver=1.0</pre>
:: or when the filename is braindead, like githubs' download tags:
:: <pre>https://github.com/software/software/archive/v$pkgver.tar.gz</pre>
:: The above two examples needs a target filename prefix:
:: <pre>$pkgname-$pkgver.tar.gz::http://oss.example.org/?get=software&ver=$pkgver</pre>
:: and:
:: <pre>$pkgname-$pkgver.tar.gz::https://github.com/software/software/archive/v$pkgver.tar.gz</pre>

:* abuild currently supports the following protocols for remote file retrieval:
:** http
:** https
:** ftp

:* abuild currently supports the following archive types/archive file extensions:
:** .tar.gz / .tgz
:** .tar.bz2
:** .tar.lzma
:** .tar.xz
:** .zip

:: {{Note|Legacy APKBUILD scripts define ''source'' variable as "saveas-[brain-dead-url]/[target-filename]" format instead of the modern [target-filename]::[brain-dead-url].<br />''BAD'': source="saveas-http://releases.ddvtech.com/download.php?pack=libmist_dist&ver=RC/$pkgname-$pkgver.tar.gz"<br />''GOOD'': source=$pkgname-$pkgver.tar.gz::http://releases.ddvtech.com/download.php?pack=libmist_dist&ver=RC"}}

==== subpackages ====
: Subpackages built from this APKBUILD. abuild will parse this variable and try to find a subpackage split function. The split function must ''move'' files that do not belong in the main package, from ''$pkgdir'' to ''$subpkgdir''. Files and directories can also be ''copied'' from ''$startdir'' and ''$srcdir'' to ''$subpkgdir''.

: The split function can be specified in 1 of 3 different methods:
:# subpkgname:'''splitfunc'''
:# $pkgname-'''splitfunc'''
:# '''splitfunc'''

: {{Note|Split function names '''cannot''' use hyphens; use the first method above if the subpackage name contains a hyphen (-) character, like this: ''subpkg-name:subpkg_name'', where <code>subpkg-name</code> is the name of the '''subpackage''' and <code>subpkg_name</code> is the name of the '''subpackage's split function'''.}}

: {{Tip|For more information, see the [[APKBUILD_examples:Subpackages|Subpackages example]].}}

==== triggers ====
: Apk-tools can "monitor" directories and execute a trigger if any package installed/uninstalled any file in the monitored dir. The triggers are always execute after the apk action (install, uninstall, upgrade).

: The triggers are specified in the format: ''scriptname''=''pathlist'' where ''scriptname'' is the (sub)package name + .trigger suffix and pathlist is : separated list of the dirs to monitor.

: The '''triggers''' variable must include the triggers for subpackages too if they have any.

: It is possible to use wildcards (*) in the dir list.

==== url ====
: The homepage for the package. This is to help users find upstream documentation and other information regarding the package.

= Functions =
{{Note|All functions should consider the current working directory as undefined, and should therefore use the [[APKBUILD Reference#abuild-defined_variables|abuild-defined directory variables]] to their advantage.}}

== abuild-defined functions ==
The following functions are provided by abuild and can be overridden:

==== fetch() ====
: Downloads remote sources listed in ''source'' to ''SRCDEST'' (''SRCDEST'' is configured in ''/etc/abuild.conf'') and creates symlinks in ''$srcdir''.
==== unpack() ====
: Unpacks .tgz, .tar.gz, .tar.bz2, .tar.lzma, .tar.xz, and .zip archives in ''$srcdir'' to ''$srcdir''.
==== dev() ====
: Subpackage function for the '''$pkgname-dev''' package. Without specifying a custom ''dev()'' function, abuild will call it's internal ''dev()'' function, which in turn calls ''default_dev()'', which will move ''"$pkgdir"/usr/include'', ''*.a'', ''*.la'' and similar files to ''$subpkgdir''.
==== doc() ====
: Subpackage function for the '''$pkgname-doc''' package. Without specifying a custom ''doc()'' function, abuild will call it's internal ''doc()'' function, which in turn calls ''default_doc()'', which will move ''"$pkgdir"/usr/share/doc'', ''"$pkgdir"/usr/share/man'' and similar to ''$subpkgdir''.

== User-defined functions ==
The following functions should be defined by the user:

==== prepare() ====
: {{note|Please adjust old APKBUILDs, which still have a ''prepare()'' function that does the same as the ''default_prepare()'' when you edit them anyway.}}
: '''''Optional''.''' Used for build preparation: patches, etc, should be applied here. When you don't specify a custom ''prepare()'', the built-in ''default_prepare()'' from abuild will be used. It applies patches already (always prepare them in the <code>-p1</code> format), so '''usually it makes sense to not create a custom ''prepare()'' function at all!''' If you do create one, call ''default_prepare()'' inside it:

<pre>
prepare() {
default_prepare
# your custom code here
}
</pre>

==== build() ====
: '''Required.''' This is the compilation stage. This function will be called as the current user (unless the ''package()'' function is missing - for compatibility reasons). If no compilation is needed, this function can contain a single line: <code>return 0</code>
==== check() ====
: '''Recommended.''' This function is called right after the build stage. It should check that the packaged thing is actually working, typically by running (integration) tests, if provided by upstream. If there’s no (easy) way how to test the package, you can declare that it does not want to use ''check()'' by adding "!check" into the ''options'' variable (<code>options="!check"</code>).
==== package() ====
: '''Required.''' This is the packaging stage. Here, the built application and support files should be installed into '''$pkgdir'''. If this is a metapackage, this function can contain a single line: <code>mkdir -p "$pkgdir"</code>

{{Note|Building in fakeroot will reduce performance for parallel builds dramatically. It is for this reason that we split the build and package process into two separate functions.}}

= Examples =
The [[APKBUILD examples]] page will assist you in understanding how to create an APKBUILD.

[[Category:Development]]

Apkindex format

2017-12-23T03:52:44Z

Mcrute:

= The APKINDEX.tar.gz format =
From apk-tools-2.0_pre15 there was added support for package signing. The caused the index format to chage, as it needs to contain a signature for the repository. This document explains how the new index works and how it is created. This is intended to be a reference for abuild developers. Other developers should use the tools provided from abuild.

See the [https://git.alpinelinux.org/cgit/abuild/tree/abuild.in#n1488 abuild source] and [https://git.alpinelinux.org/cgit/abuild/tree/abuild-sign.in#n18 abuild-sign] source for more details about how this indexes and signatures are constructed behind the scenes.

= Creating APKINDEX.tar.gz =
The APKINDEX.tar.gz is created by concatenating 2 other tar.gz files, signature.tar.gz and APKINDEX.unsigned.tar.gz.

{{Cmd|cat signature.tar.gz APKINDEX.unsigned.tar.gz > APKINDEX.tar.gz}}

== Creating signature.tar.gz Manually ==
First we create a signature file for APKINDEX.unsigned.tar.gz, using our [[Abuild_and_Helpers#abuild-keygen|private key]].
{{Cmd|openssl dgst -sha1 -sign ''privatekeyfile'' -out .SIGN.RSA.''nameofpublickey'' APKINDEX.unsigned.tar.gz}}

Then we put this in a tar file, without the ''end-of-tar'' record at the end of the file. This is because we will concatenate this tar archive with the index tar archive.

tar -c .SIGN.RSA.''nameofpublickey'' | abuild-tar --cut | gzip -9 > signature.tar.gz

The name of public key should be the email address of the developer.

== Creating APKINDEX.unsigned.tar.gz ==
The APKINDEX.unsigned.tar.gz is an old 1.9 style index file in a tar archive. This is created with:

{{Cmd|apk index -o APKINDEX.unsigned.tar.gz *.apk}}

== Signing APKINDEX.unsigned.tar.gz ==
Without a signature apk-tools will not trust the index file and will require the <tt>--allow-untrusted</tt> flag for all operations involving the index. To sign an index requires a key, if you don't already have a key the easiest way to generate one is to use the <tt>abuild-keygen</tt> command like so:

{{Cmd|apk add abuild
abuild-keygen -a -i}}

It will prompt you for a filename to save the keypair. Enter file in which to save the key. The standard practice for naming the key is to use your email address or the email address of a maintainer's mailing list as a prefix for the filename of the keypair followed by an alphanumeric suffix, which is generated by the abuild-keygen tool.

For this example, we will use <tt>alpine-devel@example.com-5629d7e6.rsa</tt> which will create the <tt>alpine-devel@example.com-5629d7e6.rsa</tt> and <tt>alpine-devel@example.com-5629d7e6.rsa.pub</tt> keypair and copy the public key to {{Path|/etc/apk/keys/}}. Make sure you keep a copy of your private key somewhere safe because you will need it if you add any packages to the repo in the future since you will need to re-sign the updated index.

Finally, sign the index. Ensure that you provide the full path to the private key or <tt>abuild-sign</tt> will not be able to find it.

{{Cmd|abuild-sign -k ~/alpine-devel@example.com-5629d7e6.rsa /repo/x86_64/APKINDEX.tar.gz}}

For hosts to trust this index ensure that the public component of the key you generated exists in their {{Path|/etc/apk/keys}} directory.

[[Category:Package Manager]] [[Category:Development]]

Install Alpine on Amazon EC2

2017-12-10T05:44:35Z

Mcrute: /* Create an Apkovl File */

EC2 instances are available in a variety of [https://aws.amazon.com/ec2/instance-types/ different sizes] with different specifications but all of them currently run on one of two types of virtualization technology: [https://en.wikipedia.org/wiki/Paravirtualization paravirtualization] (PV) or [https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine hardware virtual machine] (HVM).

PV instances boot with a special boot loader called PV-GRUB, which starts the boot cycle and then chain loads the kernel specified in the menu.lst file on your image. Paravirtual guests can run on host hardware that does not have explicit support for virtualization, but they cannot take advantage of special hardware extensions such as enhanced networking or GPU processing.

HVM instances are presented with a fully virtualized set of hardware and boot by executing the master boot record of the root block device of your image. This virtualization type provides the ability to run an operating system directly on top of a virtual machine without any modification, as if it were run on the bare-metal hardware. The Amazon EC2 host system emulates some or all of the underlying hardware that is presented to the guest. (from: [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html Linux AMI Virtualization Types]).

Only older instance types support PV instances, all new instance types use an HVM hypervisor. For the purposes of setting up Alpine Linux on EC2 this mainly impacts the way the bootloader is configured.

EC2 instances are created from templates called Amazon Machine Images (AMI). AMIs are built using existing running instances and then are turned into AMIs using a snapshot process. Note that an AMI is region specific so you must either follow this guide in each region you plan to deploy Alpine Linux instances or you must copy the AMI using either the console or the AWS command line client to each region you plan to use.

All instance types are 64-bit. 32-bit images will not work.

This guide will create the smallest possible (1GB) EBS-backed image which acts as a virtual USB stick that will boot and run Alpine Linux.
= Initial Setup =
All AMI creation tasks follow the same set of setup steps which require a running Linux instance and an attached EBS volume. The Linux instance will only be used to setup the root filesystem for your Alpine Linux system so any distro works, Amazon Linux is the default so, where relevant, this guide will use that distro.

# Create an EC2 instance, t2.micro running Amazon Linux should suffice
# Create a 1GB EBS volume and attach it to the instance as {{Path|/dev/sdf}} (this will appear on an HVM instance as {{Path|/dev/xvdf}})
# Format the disk as ext4, there is no need to partition the drive: {{Cmd|mkfs.ext4 /dev/xvdf}}
# Mount the drive to the host: {{Cmd|mkdir /mnt/target && mount /dev/xvdf /mnt/target}}
# Fetch a copy of the latest virt release: {{Cmd|curl -o /tmp/alpine-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-virt-3.7.0-x86_64.iso}}
# Mount the release iso: {{Cmd|mkdir /mnt/source && mount -o loop /tmp/alpine-release.iso /mnt/source}}
# Copy the files to the target {{Cmd|cp -av /mnt/source/boot /mnt/target}}

The sudo package is not included in the on-disk repository for the virt image which means that it will not be accessible for installation at system boot time; to enable this you will need a copy of the repository from the extended image. Download this image and copy the <tt>apks</tt> folder to the target.

# Fetch a copy of the latest virt release: {{Cmd|curl -o /tmp/alpine-extended-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-extended-3.7.0-x86_64.iso}}
# Mount the release iso: {{Cmd|mkdir /mnt/source-extended && mount -o loop /tmp/alpine-release-extended.iso /mnt/source-extended}}
# Copy the files to the target {{Cmd|cp -av /mnt/source-extended/apks /mnt/target}}

Because the packages on Amazon Linux are a little older than the ones that ship with Alpine and because you will be creating an apkovl file for initial system configuration, you will either need an existing Alpine Linux system or a copy of the minirootfs image into which you can chroot. To setup a chroot:

# Fetch a copy of the latest minirootfs release: {{Cmd|curl -o /tmp/alpine-minirootfs.tar.gz http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-minirootfs-3.7.0-x86_64.tar.gz}}
# Unpack the root fs: {{Cmd|mkdir /tmp/alpine-chroot && tar -C /tmp/alpine-chroot -xvf /tmp/alpine-minirootfs.tar.gz}}
# Copy the system resolv.conf into the chroot so you will have internet access: {{Cmd|cp /etc/resolv.conf /tmp/alpine-chroot/etc}}

= Create an Apkovl File =
The top level of the EBS volume will be scanned during boot for files named <tt>*.apkovl.tar.gz</tt> and the first one that is found will be unpacked and overlayed onto the file system. The goal of this archive is to configure the system so that you will be able to login after the system boots. At a minimum this means:

* Configure networking to use dhcp and start when the system boots
* Install an SSH daemon
* Configure a user with your SSH keys
* Enable sudo so that you can gain root access

Setup the chroot and chroot into it, then add any desired packages. The <tt>busybox-initscripts</tt> package provides the required files for udhcpc which will allow DHCP to be enabled. The <tt>syslinux</tt> package is required to install the boot-loader but is not required on the final system.

<pre>
mkdir /tmp/alpine-chroot/mnt/target
mount -o bind /mnt/target /tmp/alpine-chroot/mnt/target
mount -t proc none /tmp/alpine-chroot/proc
mount -t devtmpfs none /tmp/alpine-chroot/dev
mount -t sysfs none /tmp/alpine-chroot/sys

chroot /tmp/alpine-chroot sh

apk update
apk add alpine-conf sudo openssh busybox-initscripts syslinux
</pre>

The Alpine init system will start up the core required services if the file {{Path|/etc/.default_boot_services}} exists, so create that.

<pre>
touch etc/.default_boot_services
</pre>

Setup networking:

<pre>
cat > /etc/network/interfaces <<EOF
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
EOF

ln -s /etc/init.d/networking /etc/runlevels/default/
ln -s /etc/init.d/sshd /etc/runlevels/default/
</pre>

Add a user, set an SSH key, and ensure that this user can sudo:

<pre>
adduser -D -s /bin/sh alpine
addgroup alpine wheel

mkdir /home/alpine/.ssh
chmod 700 /home/alpine/.ssh
cat > /home/alpine/.ssh/authorized_keys <<EOF
... your ssh public key(s) ...
EOF
chmod 600 /home/alpine/.ssh/authorized_keys

sed -i '/%wheel/s/^# //' /etc/sudoers
</pre>

Exclude backup files and any files that are going to exist in the unpacked image. Then create an apkovl bundle.

<pre>
lbu exclude \
etc/group- etc/passwd- etc/shadow- \
etc/apk/keys etc/apk/arch etc/apk/repositories \
etc/os-release \
etc/issue \
etc/alpine-release
lbu include /home/alpine/.ssh
lbu package amazon.apkovl.tar.gz
</pre>

Verify that everything you expect to be in the apovl file is there and if not you can [[Manually_editing_a_existing_apkovl|modify the apkovl]].

Finally, copy the apkovl to {{Path|/mnt/target}}.

<pre>
cp amazon.apkovl.tar.gz /mnt/target
</pre>

= EBS Backed HVM AMI =
For HVM instances you will need to install a boot-loader. Any bootloader should do the trick but this guide will use syslinux because it's fast and small and you don't have to support any odd hardware or system layout.

First you will need to ensure that the ext4 driver is loaded and may as well shorten the timeout to speed instance boot time.

{{Cmd|sed -i \
-e '/usb-storage/s//usb-storage,ext4/' \
-e '/^TIMEOUT/s/20/2/' \
/mnt/target/boot/syslinux/syslinux.cfg}}

Finally, install the bootloader: {{Cmd|extlinux -i /mnt/target/boot}}

= EBS Backed PV AMI =
For PV AMIs PV-GRUB will attempt to read {{Path|/boot/grub/menu.lst}} to load the kernel. First create a {{Path|/boot/grub/grub.conf}}:

<pre>
mkdir -p /mnt/target/boot/grub

cat > target/boot/grub/grub.conf <<EOF
default=0
timeout=2
hiddenmenu

title Alpine Linux
root (hd0)
kernel /boot/vmlinuz-virthardened alpine_dev=xvda1:ext4 modules=loop,squashfs,sd-mod,ext4 console=hvc0 pax_nouderef BOOT_IMAGE=/boot/vmlinuz-virthardened
initrd /boot/initramfs-virthardened
EOF

ln -s /mnt/target/boot/grub/grub.conf /mnt/target/boot/grub/menu.lst
</pre>

= Create the AMI =
The following will need to be done in the AWS console.

# Detach the new volume (make note of the volume ID)
# Launch a new instance, use the defaults; you are going to cannibalize it in a moment
# Once the instance starts, stop but do not terminate it
# Under EBS, detach the existing volume, and attach the Alpine Linux volume as {{Path|/dev/xvda}} (for HVM) or {{Path|/dev/sda1}} (for PV)
# Restart the instance
# Log in and make sure it works
# Do any final cleanups necessary. Only make changes appropriate for an AMI, you are going to snapshot this instance and use it as the base for the AMI.
# Stop but do not terminate the instance
# Right click the stopped instance and choose 'Create Image (EBS AMI)'
# Once the AMI creation finishes you can cleanup the instances and EBS volumes used

[[Category:Virtualization]]

Install Alpine on Amazon EC2

2017-12-10T05:30:35Z

Mcrute:

EC2 instances are available in a variety of [https://aws.amazon.com/ec2/instance-types/ different sizes] with different specifications but all of them currently run on one of two types of virtualization technology: [https://en.wikipedia.org/wiki/Paravirtualization paravirtualization] (PV) or [https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine hardware virtual machine] (HVM).

PV instances boot with a special boot loader called PV-GRUB, which starts the boot cycle and then chain loads the kernel specified in the menu.lst file on your image. Paravirtual guests can run on host hardware that does not have explicit support for virtualization, but they cannot take advantage of special hardware extensions such as enhanced networking or GPU processing.

HVM instances are presented with a fully virtualized set of hardware and boot by executing the master boot record of the root block device of your image. This virtualization type provides the ability to run an operating system directly on top of a virtual machine without any modification, as if it were run on the bare-metal hardware. The Amazon EC2 host system emulates some or all of the underlying hardware that is presented to the guest. (from: [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html Linux AMI Virtualization Types]).

Only older instance types support PV instances, all new instance types use an HVM hypervisor. For the purposes of setting up Alpine Linux on EC2 this mainly impacts the way the bootloader is configured.

EC2 instances are created from templates called Amazon Machine Images (AMI). AMIs are built using existing running instances and then are turned into AMIs using a snapshot process. Note that an AMI is region specific so you must either follow this guide in each region you plan to deploy Alpine Linux instances or you must copy the AMI using either the console or the AWS command line client to each region you plan to use.

All instance types are 64-bit. 32-bit images will not work.

This guide will create the smallest possible (1GB) EBS-backed image which acts as a virtual USB stick that will boot and run Alpine Linux.
= Initial Setup =
All AMI creation tasks follow the same set of setup steps which require a running Linux instance and an attached EBS volume. The Linux instance will only be used to setup the root filesystem for your Alpine Linux system so any distro works, Amazon Linux is the default so, where relevant, this guide will use that distro.

# Create an EC2 instance, t2.micro running Amazon Linux should suffice
# Create a 1GB EBS volume and attach it to the instance as {{Path|/dev/sdf}} (this will appear on an HVM instance as {{Path|/dev/xvdf}})
# Format the disk as ext4, there is no need to partition the drive: {{Cmd|mkfs.ext4 /dev/xvdf}}
# Mount the drive to the host: {{Cmd|mkdir /mnt/target && mount /dev/xvdf /mnt/target}}
# Fetch a copy of the latest virt release: {{Cmd|curl -o /tmp/alpine-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-virt-3.7.0-x86_64.iso}}
# Mount the release iso: {{Cmd|mkdir /mnt/source && mount -o loop /tmp/alpine-release.iso /mnt/source}}
# Copy the files to the target {{Cmd|cp -av /mnt/source/boot /mnt/target}}

The sudo package is not included in the on-disk repository for the virt image which means that it will not be accessible for installation at system boot time; to enable this you will need a copy of the repository from the extended image. Download this image and copy the <tt>apks</tt> folder to the target.

# Fetch a copy of the latest virt release: {{Cmd|curl -o /tmp/alpine-extended-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-extended-3.7.0-x86_64.iso}}
# Mount the release iso: {{Cmd|mkdir /mnt/source-extended && mount -o loop /tmp/alpine-release-extended.iso /mnt/source-extended}}
# Copy the files to the target {{Cmd|cp -av /mnt/source-extended/apks /mnt/target}}

Because the packages on Amazon Linux are a little older than the ones that ship with Alpine and because you will be creating an apkovl file for initial system configuration, you will either need an existing Alpine Linux system or a copy of the minirootfs image into which you can chroot. To setup a chroot:

# Fetch a copy of the latest minirootfs release: {{Cmd|curl -o /tmp/alpine-minirootfs.tar.gz http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-minirootfs-3.7.0-x86_64.tar.gz}}
# Unpack the root fs: {{Cmd|mkdir /tmp/alpine-chroot && tar -C /tmp/alpine-chroot -xvf /tmp/alpine-minirootfs.tar.gz}}
# Copy the system resolv.conf into the chroot so you will have internet access: {{Cmd|cp /etc/resolv.conf /tmp/alpine-chroot/etc}}

= Create an Apkovl File =
The top level of the EBS volume will be scanned during boot for files named <tt>*.apkovl.tar.gz</tt> and the first one that is found will be unpacked and overlayed onto the file system. The goal of this archive is to configure the system so that you will be able to login after the system boots. At a minimum this means:

* Configure networking to use dhcp and start when the system boots
* Install an SSH daemon
* Configure a user with your SSH keys
* Enable sudo so that you can gain root access

Setup the chroot and chroot into it, then add any desired packages. The <tt>busybox-initscripts</tt> package provides the required files for udhcpc which will allow DHCP to be enabled.

<pre>
mkdir /tmp/alpine-chroot/mnt/target
mount -o bind /mnt/target /tmp/alpine-chroot/mnt/target
mount -t proc none /tmp/alpine-chroot/proc
mount -t devtmpfs none /tmp/alpine-chroot/dev
mount -t sysfs none /tmp/alpine-chroot/sys

chroot /tmp/alpine-chroot sh

apk update
apk add alpine-conf sudo openssh busybox-initscripts
</pre>

The Alpine init system will start up the core required services if the file {{Path|/etc/.default_boot_services}} exists, so create that.

<pre>
touch etc/.default_boot_services
</pre>

Setup networking:

<pre>
cat > /etc/network/interfaces <<EOF
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
EOF

ln -s /etc/init.d/networking /etc/runlevels/default/
ln -s /etc/init.d/sshd /etc/runlevels/default/
</pre>

Add a user, set an SSH key, and ensure that this user can sudo:

<pre>
adduser -D -s /bin/sh alpine
addgroup alpine wheel

mkdir /home/alpine/.ssh
chmod 700 /home/alpine/.ssh
cat > /home/alpine/.ssh/authorized_keys <<EOF
... your ssh public key(s) ...
EOF
chmod 600 /home/alpine/.ssh/authorized_keys

sed -i '/%wheel/s/^# //' /etc/sudoers
</pre>

Exclude backup files and any files that are going to exist in the unpacked image. Then create an apkovl bundle.

<pre>
lbu exclude \
etc/group- etc/passwd- etc/shadow- \
etc/apk/keys etc/apk/arch etc/apk/repositories \
etc/os-release \
etc/issue \
etc/alpine-release
lbu include /home/alpine/.ssh
lbu package amazon.apkovl.tar.gz
</pre>

Verify that everything you expect to be in the apovl file is there and if not you can [[Manually_editing_a_existing_apkovl|modify the apkovl]].

Finally, copy the apkovl to {{Path|/mnt/target}}.

<pre>
cp amazon.apkovl.tar.gz /mnt/target
</pre>

= EBS Backed HVM AMI =
For HVM instances you will need to install a boot-loader. Any bootloader should do the trick but this guide will use syslinux because it's fast and small and you don't have to support any odd hardware or system layout.

First you will need to ensure that the ext4 driver is loaded and may as well shorten the timeout to speed instance boot time.

{{Cmd|sed -i \
-e '/usb-storage/s//usb-storage,ext4/' \
-e '/^TIMEOUT/s/20/2/' \
/mnt/target/boot/syslinux/syslinux.cfg}}

Finally, install the bootloader: {{Cmd|extlinux -i /mnt/target/boot}}

= EBS Backed PV AMI =
For PV AMIs PV-GRUB will attempt to read {{Path|/boot/grub/menu.lst}} to load the kernel. First create a {{Path|/boot/grub/grub.conf}}:

<pre>
mkdir -p /mnt/target/boot/grub

cat > target/boot/grub/grub.conf <<EOF
default=0
timeout=2
hiddenmenu

title Alpine Linux
root (hd0)
kernel /boot/vmlinuz-virthardened alpine_dev=xvda1:ext4 modules=loop,squashfs,sd-mod,ext4 console=hvc0 pax_nouderef BOOT_IMAGE=/boot/vmlinuz-virthardened
initrd /boot/initramfs-virthardened
EOF

ln -s /mnt/target/boot/grub/grub.conf /mnt/target/boot/grub/menu.lst
</pre>

= Create the AMI =
The following will need to be done in the AWS console.

# Detach the new volume (make note of the volume ID)
# Launch a new instance, use the defaults; you are going to cannibalize it in a moment
# Once the instance starts, stop but do not terminate it
# Under EBS, detach the existing volume, and attach the Alpine Linux volume as {{Path|/dev/xvda}} (for HVM) or {{Path|/dev/sda1}} (for PV)
# Restart the instance
# Log in and make sure it works
# Do any final cleanups necessary. Only make changes appropriate for an AMI, you are going to snapshot this instance and use it as the base for the AMI.
# Stop but do not terminate the instance
# Right click the stopped instance and choose 'Create Image (EBS AMI)'
# Once the AMI creation finishes you can cleanup the instances and EBS volumes used

[[Category:Virtualization]]

User:Mcrute

2017-12-10T05:25:26Z

Mcrute:

= Mike Crute =
Software engineer by day, tinkerer by night. I like to document things. Feel free to email me, my email address is my first name at my last name dot us.

My words and opinions are my own and do not reflect the values or opinions of my employers past, present, or future.

User:Mcrute

2017-12-10T05:24:50Z

Mcrute:

User:Mcrute

2017-12-10T05:24:40Z

Mcrute:

= Mike Crute =
Software engineer by day, tinkerer by night. I like to document things. Feel free to email me, my email address is my first name at my last name dot me.

My words and opinions are my own and do not reflect the values or opinions of my employers past or present.

User:Mcrute

2017-12-10T05:24:03Z

Mcrute: Created page with "= Mike Crute = Software engineer by day, tinkerer by night. I like to document things. My words and opinions are my own and do not reflect the values or opinions of my employ..."

= Mike Crute =
Software engineer by day, tinkerer by night. I like to document things.

My words and opinions are my own and do not reflect the values or opinions of my employers past or present.

Install Alpine on Amazon EC2

2017-12-10T05:16:44Z

Mcrute:

EC2 instances are available in a variety of [https://aws.amazon.com/ec2/instance-types/ different sizes] with different specifications but all of them currently run on one of two types of virtualization technology: [https://en.wikipedia.org/wiki/Paravirtualization paravirtualization] (PV) or [https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine hardware virtual machine] (HVM).

PV instances boot with a special boot loader called PV-GRUB, which starts the boot cycle and then chain loads the kernel specified in the menu.lst file on your image. Paravirtual guests can run on host hardware that does not have explicit support for virtualization, but they cannot take advantage of special hardware extensions such as enhanced networking or GPU processing.

HVM instances are presented with a fully virtualized set of hardware and boot by executing the master boot record of the root block device of your image. This virtualization type provides the ability to run an operating system directly on top of a virtual machine without any modification, as if it were run on the bare-metal hardware. The Amazon EC2 host system emulates some or all of the underlying hardware that is presented to the guest. (from: [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html Linux AMI Virtualization Types]).

Only older instance types support PV instances, all new instance types use an HVM hypervisor. For the purposes of setting up Alpine Linux on EC2 this mainly impacts the way the bootloader is configured.

EC2 instances are created from templates called Amazon Machine Images (AMI). AMIs are built using existing running instances and then are turned into AMIs using a snapshot process. Note that an AMI is region specific so you must either follow this guide in each region you plan to deploy Alpine Linux instances or you must copy the AMI using either the console or the AWS command line client to each region you plan to use.

All instance types are 64-bit. 32-bit images will not work.

This guide will create the smallest possible (1GB) EBS-backed image which acts as a virtual USB stick that will boot and run Alpine Linux.
= Initial Setup =
All AMI creation tasks follow the same set of setup steps which require a running Linux instance and an attached EBS volume. The Linux instance will only be used to setup the root filesystem for your Alpine Linux system so any distro works, Amazon Linux is the default so, where relevant, this guide will use that distro.

# Create an EC2 instance, t2.micro running Amazon Linux should suffice
# Create a 1GB EBS volume and attach it to the instance as <tt>/dev/sdf</tt> (this will appear on an HVM instance as <tt>/dev/xvdf</tt>)
# Format the disk as ext4, there is no need to partition the drive: {{Cmd|mkfs.ext4 /dev/xvdf}}
# Mount the drive to the host: {{Cmd|mkdir /mnt/target && mount /dev/xvdf /mnt/target}}
# Fetch a copy of the latest virt release: {{Cmd|curl -o /tmp/alpine-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-virt-3.7.0-x86_64.iso}}
# Mount the release iso: {{Cmd|mkdir /mnt/source && mount -o loop /tmp/alpine-release.iso /mnt/source}}
# Copy the files to the target {{Cmd|cp -av /mnt/source/boot /mnt/target}}

The sudo package is not included in the on-disk repository for the virt image which means that it will not be accessible for installation at system boot time; to enable this you will need a copy of the repository from the extended image. Download this image and copy the <tt>apks</tt> folder to the target.

# Fetch a copy of the latest virt release: {{Cmd|curl -o /tmp/alpine-extended-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-extended-3.7.0-x86_64.iso}}
# Mount the release iso: {{Cmd|mkdir /mnt/source-extended && mount -o loop /tmp/alpine-release-extended.iso /mnt/source-extended}}
# Copy the files to the target {{Cmd|cp -av /mnt/source-extended/apks /mnt/target}}

Because the packages on Amazon Linux are a little older than the ones that ship with Alpine and because you will be creating an apkovl file for initial system configuration, you will either need an existing Alpine Linux system or a copy of the minirootfs image into which you can chroot. To setup a chroot:

# Fetch a copy of the latest minirootfs release: {{Cmd|curl -o /tmp/alpine-minirootfs.tar.gz http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-minirootfs-3.7.0-x86_64.tar.gz}}
# Unpack the root fs: {{Cmd|mkdir /tmp/alpine-chroot && tar -C /tmp/alpine-chroot -xvf /tmp/alpine-minirootfs.tar.gz}}
# Copy the system resolv.conf into the chroot so you will have internet access: {{Cmd|cp /etc/resolv.conf /tmp/alpine-chroot/etc}}

= Create an Apkovl File =
The top level of the EBS volume will be scanned during boot for files named <tt>*.apkovl.tar.gz</tt> and the first one that is found will be unpacked and overlayed onto the file system. The goal of this archive is to configure the system so that you will be able to login after the system boots. At a minimum this means:

* Configure networking to use dhcp and start when the system boots
* Install an SSH daemon
* Configure a user with your SSH keys
* Enable sudo so that you can gain root access

Setup the chroot and chroot into it, then add any desired packages. The <tt>busybox-initscripts</tt> package provides the required files for udhcpc which will allow DHCP to be enabled.

<pre>
mkdir /tmp/alpine-chroot/mnt/target
mount -o bind /mnt/target /tmp/alpine-chroot/mnt/target
mount -t proc none /tmp/alpine-chroot/proc
mount -t devtmpfs none /tmp/alpine-chroot/dev
mount -t sysfs none /tmp/alpine-chroot/sys

chroot /tmp/alpine-chroot sh

apk update
apk add alpine-conf sudo openssh busybox-initscripts
</pre>

The Alpine init system will start up the core required services if the file <tt>/etc/.default_boot_services</tt> exists, so create that.

<pre>
touch etc/.default_boot_services
</pre>

Setup networking:

<pre>
cat > /etc/network/interfaces <<EOF
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
EOF

ln -s /etc/init.d/networking /etc/runlevels/default/
ln -s /etc/init.d/sshd /etc/runlevels/default/
</pre>

Add a user, set an SSH key, and ensure that this user can sudo:

<pre>
adduser -D -s /bin/sh alpine
addgroup alpine wheel

mkdir /home/alpine/.ssh
chmod 700 /home/alpine/.ssh
cat > /home/alpine/.ssh/authorized_keys <<EOF
... your ssh public key(s) ...
EOF
chmod 600 /home/alpine/.ssh/authorized_keys

sed -i '/%wheel/s/^# //' /etc/sudoers
</pre>

Exclude backup files and any files that are going to exist in the unpacked image. Then create an apkovl bundle.

<pre>
lbu exclude \
etc/group- etc/passwd- etc/shadow- \
etc/apk/keys etc/apk/arch etc/apk/repositories \
etc/os-release \
etc/issue \
etc/alpine-release
lbu include /home/alpine/.ssh
lbu package amazon.apkovl.tar.gz
</pre>

Verify that everything you expect to be in the apovl file is there and if not you can [[Manually_editing_a_existing_apkovl|modify the apkovl]].

Finally, copy the apkovl to <tt>/mnt/target</tt>.

<pre>
cp amazon.apkovl.tar.gz /mnt/target
</pre>

= EBS Backed HVM AMI =
For HVM instances you will need to install a boot-loader. Any bootloader should do the trick but this guide will use syslinux because it's fast and small and you don't have to support any odd hardware or system layout.

First you will need to ensure that the ext4 driver is loaded and may as well shorten the timeout to speed instance boot time.

{{Cmd|sed -i \
-e '/usb-storage/s//usb-storage,ext4/' \
-e '/^TIMEOUT/s/20/2/' \
/mnt/target/boot/syslinux/syslinux.cfg}}

Finally, install the bootloader: {{Cmd|extlinux -i /mnt/target/boot}}

= EBS Backed PV AMI =
For PV AMIs PV-GRUB will attempt to read <tt>/boot/grub/menu.lst</tt> to load the kernel. First create a <tt>/boot/grub/grub.conf</tt>:

<pre>
mkdir -p /mnt/target/boot/grub

cat > target/boot/grub/grub.conf <<EOF
default=0
timeout=2
hiddenmenu

title Alpine Linux
root (hd0)
kernel /boot/vmlinuz-virthardened alpine_dev=xvda1:ext4 modules=loop,squashfs,sd-mod,ext4 console=hvc0 pax_nouderef BOOT_IMAGE=/boot/vmlinuz-virthardened
initrd /boot/initramfs-virthardened
EOF

ln -s /mnt/target/boot/grub/grub.conf /mnt/target/boot/grub/menu.lst
</pre>

= Create the AMI =
The following will need to be done in the AWS console.

# Detach the new volume (make note of the volume ID)
# Launch a new instance, use the defaults; you are going to cannibalize it in a moment
# Once the instance starts, stop but do not terminate it
# Under EBS, detach the existing volume, and attach the Alpine Linux volume as <tt>/dev/xvda</tt> (for HVM) or <tt>/dev/sda1</tt> (for PV)
# Restart the instance
# Log in and make sure it works
# Do any final cleanups necessary. Only make changes appropriate for an AMI, you are going to snapshot this instance and use it as the base for the AMI.
# Stop but do not terminate the instance
# Right click the stopped instance and choose 'Create Image (EBS AMI)'
# Once the AMI creation finishes you can cleanup the instances and EBS volumes used

[[Category:Virtualization]]