Install Alpine on Amazon EC2: Difference between revisions

From Alpine Linux
(update kernel/initrd filename for AL3.3)
(need wikiformating and integration.. added note header)
(11 intermediate revisions by 7 users not shown)
Line 1: Line 1:
{{Draft}}
{{Move| Alpine Install: from a image to a Amazon EC2 instance |This article need wikiformating consisten with setups procedure and naming formating, see [[Alpine_newbie_install_manual]].}}


The goal here is to have a "1GB" (the smallest possible) EBS 'virtual usb stick' that can boot and run Alpine Linux.
EC2 instances are available in a variety of [https://aws.amazon.com/ec2/instance-types/ different sizes] with different specifications but all of them currently run on one of two types of virtualization technology: [https://en.wikipedia.org/wiki/Paravirtualization paravirtualization] (PV) or [https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine hardware virtual machine] (HVM).


= Create an EBS backed Alpine Linux AMI =
PV instances boot with a special boot loader called PV-GRUB, which starts the boot cycle and then chain loads the kernel specified in the menu.lst file on your image. Paravirtual guests can run on host hardware that does not have explicit support for virtualization, but they cannot take advantage of special hardware extensions such as enhanced networking or GPU processing.


{{Note|You need to do this process at least once in each availability region.  EBS can't be shared between Ireland and California, for instance.}}
HVM instances are presented with a fully virtualized set of hardware and boot by executing the master boot record of the root block device of your image. This virtualization type provides the ability to run an operating system directly on top of a virtual machine without any modification, as if it were run on the bare-metal hardware. The Amazon EC2 host system emulates some or all of the underlying hardware that is presented to the guest. (from: [https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtualization_types.html Linux AMI Virtualization Types]).
 
Only older instance types support PV instances, all new instance types use an HVM hypervisor. For the purposes of setting up Alpine Linux on EC2 this mainly impacts the way the bootloader is configured.
 
EC2 instances are created from templates called Amazon Machine Images (AMI). AMIs are built using existing running instances and then are turned into AMIs using a snapshot process. Note that an AMI is region specific so you must either follow this guide in each region you plan to deploy Alpine Linux instances or you must copy the AMI using either the console or the AWS command line client to each region you plan to use.
 
All instance types are 64-bit. 32-bit images will not work.
 
This guide will create the smallest possible (1GB) EBS-backed image which acts as a virtual USB stick that will boot and run Alpine Linux.   
= Initial Setup =
All AMI creation tasks follow the same set of setup steps which require a running Linux instance and an attached EBS volume. The Linux instance will only be used to set up the root filesystem for your Alpine Linux system so any distro works, Amazon Linux is the default so, where relevant, this guide will use that distro.
 
# Create an EC2 instance, t2.micro running Amazon Linux should suffice
# Create a 1GB EBS volume and attach it to the instance as {{Path|/dev/sdf}} (this will appear on an HVM instance as {{Path|/dev/xvdf}})
# Format the disk as ext4, there is no need to partition the drive: {{Cmd|mkfs.ext4 /dev/xvdf}}
# Mount the drive to the host: {{Cmd|mkdir /mnt/target && mount /dev/xvdf /mnt/target}}
# Fetch a copy of the latest virt release: {{Cmd|curl -o /tmp/alpine-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-virt-3.7.0-x86_64.iso}}
# Mount the release iso: {{Cmd|mkdir /mnt/source && mount -o loop /tmp/alpine-release.iso /mnt/source}}
# Copy the files to the target {{Cmd|cp -av /mnt/source/boot /mnt/target}}
 
The sudo package is not included in the on-disk repository for the virt image which means that it will not be accessible for installation at system boot time; to enable this you will need a copy of the repository from the extended image. Download this image and copy the <tt>apks</tt> folder to the target.
 
# Fetch a copy of the latest extended release: {{Cmd|curl -o /tmp/alpine-extended-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-extended-3.7.0-x86_64.iso}}
# Mount the release iso: {{Cmd|mkdir /mnt/source-extended && mount -o loop /tmp/alpine-extended-release.iso /mnt/source-extended}}
# Copy the files to the target {{Cmd|cp -av /mnt/source-extended/apks /mnt/target}}
 
Because the packages on Amazon Linux are a little older than the ones that ship with Alpine and because you will be creating an apkovl file for initial system configuration, you will either need an existing Alpine Linux system or a copy of the minirootfs image into which you can chroot. To set up a chroot:
 
# Fetch a copy of the latest minirootfs release: {{Cmd|curl -o /tmp/alpine-minirootfs.tar.gz http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-minirootfs-3.7.0-x86_64.tar.gz}}
# Unpack the root fs: {{Cmd|mkdir /tmp/alpine-chroot && tar -C /tmp/alpine-chroot -xvf /tmp/alpine-minirootfs.tar.gz}}
# Copy the system resolv.conf into the chroot so you will have internet access: {{Cmd|cp /etc/resolv.conf /tmp/alpine-chroot/etc}}
 
= Create an Apkovl File =
The top level of the EBS volume will be scanned during boot for files named <tt>*.apkovl.tar.gz</tt> and the first one that is found will be unpacked and overlayed onto the file system. The goal of this archive is to configure the system so that you will be able to login after the system boots. At a minimum this means:
 
* Configure networking to use dhcp and start when the system boots
* Install an SSH daemon
* Configure a user with your SSH keys
* Enable sudo so that you can gain root access
 
Set up the chroot and chroot into it, then add any desired packages. The <tt>busybox-initscripts</tt> package provides the required files for udhcpc which will allow DHCP to be enabled. The <tt>syslinux</tt> package is required to install the boot-loader but is not required on the final system.


* Create an Amazon instance in the desired availability region.  A micro instance is fine - we will need it only long enough to create our EBS usb stick.
* Create a new 1GB EBS volume
* Attach the new volume to the running instance
* The new volume will have a name like /dev/xvdf or such
* Format the volume as ext4 {{Cmd|mke2fs -t ext4 /dev/xvdf}}  ''Do not partition it - just format the whole volume''
* wget a '''x86_64''' iso and extract it to the new volume. 32bit will not work.
<pre>
<pre>
wget http://dl-4.alpinelinux.org/alpine/v2.4/releases/x86_64/alpine-2.4.5-x86_64.iso
mkdir /tmp/alpine-chroot/mnt/target
mkdir target
mount -o bind /mnt/target /tmp/alpine-chroot/mnt/target
mkdir source
mount -t proc none /tmp/alpine-chroot/proc
mount /dev/xvdf target
mount -t devtmpfs none /tmp/alpine-chroot/dev
mount -o loop alpine-2.4.5-x86_64.iso source
mount -t sysfs none /tmp/alpine-chroot/sys
cp -av source/boot target
 
cp -av source/apks target
chroot /tmp/alpine-chroot sh
umount source
 
apk update
apk add alpine-conf sudo openssh busybox-initscripts syslinux
</pre>
</pre>
* Create a grub.conf on the new partition.
 
The Alpine init system will start up the core required services if the file {{Path|/etc/.default_boot_services}} exists, so create that.
 
<pre>
<pre>
mkdir -p target/boot/grub
touch etc/.default_boot_services
cat - >target/boot/grub/grub.conf <<EOF  
</pre>
 
Set up networking:
 
<pre>
cat > /etc/network/interfaces <<EOF
auto lo
iface lo inet loopback
 
auto eth0
iface eth0 inet dhcp
EOF
 
ln -s /etc/init.d/networking /etc/runlevels/default/
ln -s /etc/init.d/sshd /etc/runlevels/default/
</pre>
 
Add a user, set an SSH key, and ensure that this user can sudo:
 
<pre>
adduser -D -s /bin/sh alpine
addgroup alpine wheel
 
mkdir /home/alpine/.ssh
chmod 700 /home/alpine/.ssh
chown alpine /home/alpine/.ssh
cat > /home/alpine/.ssh/authorized_keys <<EOF
... your ssh public key(s) ...
EOF
chmod 600 /home/alpine/.ssh/authorized_keys
chown alpine:alpine /home/alpine/.ssh/authorized_keys
 
passwd -u alpine
 
sed -i '/%wheel/s/^# //' /etc/sudoers
</pre>
 
Exclude backup files and any files that are going to exist in the unpacked image. Then create an apkovl bundle.
 
<pre>
lbu exclude \
    etc/group- etc/passwd- etc/shadow- \
    etc/apk/keys etc/apk/arch etc/apk/repositories \
    etc/os-release \
    etc/issue \
    etc/alpine-release
lbu include /home/alpine/.ssh
lbu package amazon.apkovl.tar.gz
</pre>
 
Verify that everything you expect to be in the apovl file is there and if not you can [[Manually_editing_a_existing_apkovl|modify the apkovl]].
 
Finally, copy the apkovl to {{Path|/mnt/target}}.
 
<pre>
cp amazon.apkovl.tar.gz /mnt/target
</pre>
 
= EBS Backed HVM AMI =
For HVM instances you will need to install a boot-loader. Any bootloader should do the trick but this guide will use syslinux because it's fast and small and you don't have to support any odd hardware or system layout.
 
First you will need to ensure that the ext4 driver is loaded and may as well shorten the timeout to speed instance boot time.
 
{{Cmd|sed -i \
    -e '/usb-storage/s//usb-storage,ext4/' \
    -e '/^TIMEOUT/s/20/2/' \
    /mnt/target/boot/syslinux/syslinux.cfg}}
 
Finally, install the bootloader: {{Cmd|extlinux -i /mnt/target/boot}}
 
= EBS Backed PV AMI =
For PV AMIs PV-GRUB will attempt to read {{Path|/boot/grub/menu.lst}} to load the kernel. First create a {{Path|/boot/grub/grub.conf}}:
 
<pre>
mkdir -p /mnt/target/boot/grub
 
cat > target/boot/grub/grub.conf <<EOF  
default=0
default=0
timeout=3
timeout=2
hiddenmenu
hiddenmenu


title Alpine Linux
title Alpine Linux
root (hd0)
root (hd0)
kernel /boot/grsec alpine_dev=xvda1:ext4 modules=loop,squashfs,sd-mod,ext4 console=hvc0 pax_nouderef BOOT_IMAGE=/boot/vmlinuz-grsec
kernel /boot/vmlinuz-virthardened alpine_dev=xvda1:ext4 modules=loop,squashfs,sd-mod,ext4 console=hvc0 pax_nouderef BOOT_IMAGE=/boot/vmlinuz-virthardened
initrd /boot/initramfs-grsec
initrd /boot/initramfs-virthardened
EOF
EOF
ln -s /mnt/target/boot/grub/grub.conf  /mnt/target/boot/grub/menu.lst
</pre>
</pre>
:* Syslinux automatically adds BOOT_IMAGE to the kernel command line; grub does not, so make sure you specify it in the grub.conf
:* You do not need any other grub files - just boot.conf
* symlink the grub.conf to menu.lst
<pre>
ln -sf ./grub.conf target/boot/grub/menu.lst
</pre>
* Create an amazon.apkovl.tar.gz file to put on the target
** This is probably easiest on a local alpine linux instance.  Make sure the following are configured:
*** eth0 uses dhcp
*** networking is set to autostart
*** sshd is installed and set to autostart
*** Your ssh public key is in /root/.ssh/authorized_keys
*** The root password is set to something
*** lbu include root/.ssh
*** (optional) - Delete the /etc/ssh/*key* files, so they are created on the new box
** {{Cmd|lbu package amazon.apkovl.tar.gz}} {{Warning|If you are packaging on a 32bit box, manually delete etc/apk/arch from the apkovl.tar.gz file}}
** Copy amazon.apkovl.tar.gz to target/
* Unmount target
* '''Do the following from the Amazon web interface'''
** Detach the new volume
** Make note of the volume ID
** Launch NEW instance.  Use defaults, amazon linux, micro; we are going to canibalize it in a bit, so defaults are fine here.
** Once the instance starts, ''stop'' but ''do not terminate'' the instance.
** Under EBS, detach the existing volume, and attach the alpine linux volume as /dev/sda1 (note the 1 at the end)
** Restart the instance
* Log in and make sure it works
* Do any final cleanups necessary, and if necessary lbu ci
** Only make configs that are appropriate for an AMI, we are going to snapshot this instance and create an AMI out of it
* Again from the Amazon web interface
** Delete the 8GB volume that is no longer needed
** ''Stop'' but do not terminate the instance
** Right click the stopped instance and choose 'Create Image (EBS AMI)'
*** Image name should be unique for the image - example AlpineLinux-2.4.5
*** Description can be anything - example 'Base AlpineLinux Installation - no services'
* Done.


= Create the AMI =
The following will need to be done in the AWS console.


# Detach the new volume (make note of the volume ID)
# Launch a new instance, use the defaults; you are going to cannibalize it in a moment
# Once the instance starts, stop but do not terminate it
# Under EBS, detach the existing volume, and attach the Alpine Linux volume as {{Path|/dev/xvda}} (for HVM) or {{Path|/dev/sda1}} (for PV)
# Restart the instance
# Log in and make sure it works
# Do any final cleanups necessary. Only make changes appropriate for an AMI, you are going to snapshot this instance and use it as the base for the AMI.
# Stop but do not terminate the instance
# Right click the stopped instance and choose 'Create Image (EBS AMI)'
# Once the AMI creation finishes you can cleanup the instances and EBS volumes used


[[Category:Virtualization]]
[[Category:Virtualization]]

Revision as of 13:43, 15 November 2019

This page is proposed for moving ...

It should be renamed to [[ Alpine Install: from a image to a Amazon EC2 instance ]]. This article need wikiformating consisten with setups procedure and naming formating, see Alpine_newbie_install_manual. (Discuss)

EC2 instances are available in a variety of different sizes with different specifications but all of them currently run on one of two types of virtualization technology: paravirtualization (PV) or hardware virtual machine (HVM).

PV instances boot with a special boot loader called PV-GRUB, which starts the boot cycle and then chain loads the kernel specified in the menu.lst file on your image. Paravirtual guests can run on host hardware that does not have explicit support for virtualization, but they cannot take advantage of special hardware extensions such as enhanced networking or GPU processing.

HVM instances are presented with a fully virtualized set of hardware and boot by executing the master boot record of the root block device of your image. This virtualization type provides the ability to run an operating system directly on top of a virtual machine without any modification, as if it were run on the bare-metal hardware. The Amazon EC2 host system emulates some or all of the underlying hardware that is presented to the guest. (from: Linux AMI Virtualization Types).

Only older instance types support PV instances, all new instance types use an HVM hypervisor. For the purposes of setting up Alpine Linux on EC2 this mainly impacts the way the bootloader is configured.

EC2 instances are created from templates called Amazon Machine Images (AMI). AMIs are built using existing running instances and then are turned into AMIs using a snapshot process. Note that an AMI is region specific so you must either follow this guide in each region you plan to deploy Alpine Linux instances or you must copy the AMI using either the console or the AWS command line client to each region you plan to use.

All instance types are 64-bit. 32-bit images will not work.

This guide will create the smallest possible (1GB) EBS-backed image which acts as a virtual USB stick that will boot and run Alpine Linux.

Initial Setup

All AMI creation tasks follow the same set of setup steps which require a running Linux instance and an attached EBS volume. The Linux instance will only be used to set up the root filesystem for your Alpine Linux system so any distro works, Amazon Linux is the default so, where relevant, this guide will use that distro.

  1. Create an EC2 instance, t2.micro running Amazon Linux should suffice
  2. Create a 1GB EBS volume and attach it to the instance as /dev/sdf (this will appear on an HVM instance as /dev/xvdf)
  3. Format the disk as ext4, there is no need to partition the drive:

    mkfs.ext4 /dev/xvdf

  4. Mount the drive to the host:

    mkdir /mnt/target && mount /dev/xvdf /mnt/target

  5. Fetch a copy of the latest virt release:

    curl -o /tmp/alpine-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-virt-3.7.0-x86_64.iso

  6. Mount the release iso:

    mkdir /mnt/source && mount -o loop /tmp/alpine-release.iso /mnt/source

  7. Copy the files to the target

    cp -av /mnt/source/boot /mnt/target

The sudo package is not included in the on-disk repository for the virt image which means that it will not be accessible for installation at system boot time; to enable this you will need a copy of the repository from the extended image. Download this image and copy the apks folder to the target.

  1. Fetch a copy of the latest extended release:

    curl -o /tmp/alpine-extended-release.iso http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-extended-3.7.0-x86_64.iso

  2. Mount the release iso:

    mkdir /mnt/source-extended && mount -o loop /tmp/alpine-extended-release.iso /mnt/source-extended

  3. Copy the files to the target

    cp -av /mnt/source-extended/apks /mnt/target

Because the packages on Amazon Linux are a little older than the ones that ship with Alpine and because you will be creating an apkovl file for initial system configuration, you will either need an existing Alpine Linux system or a copy of the minirootfs image into which you can chroot. To set up a chroot:

  1. Fetch a copy of the latest minirootfs release:

    curl -o /tmp/alpine-minirootfs.tar.gz http://dl-4.alpinelinux.org/alpine/v3.7/releases/x86_64/alpine-minirootfs-3.7.0-x86_64.tar.gz

  2. Unpack the root fs:

    mkdir /tmp/alpine-chroot && tar -C /tmp/alpine-chroot -xvf /tmp/alpine-minirootfs.tar.gz

  3. Copy the system resolv.conf into the chroot so you will have internet access:

    cp /etc/resolv.conf /tmp/alpine-chroot/etc

Create an Apkovl File

The top level of the EBS volume will be scanned during boot for files named *.apkovl.tar.gz and the first one that is found will be unpacked and overlayed onto the file system. The goal of this archive is to configure the system so that you will be able to login after the system boots. At a minimum this means:

  • Configure networking to use dhcp and start when the system boots
  • Install an SSH daemon
  • Configure a user with your SSH keys
  • Enable sudo so that you can gain root access

Set up the chroot and chroot into it, then add any desired packages. The busybox-initscripts package provides the required files for udhcpc which will allow DHCP to be enabled. The syslinux package is required to install the boot-loader but is not required on the final system.

mkdir /tmp/alpine-chroot/mnt/target
mount -o bind /mnt/target /tmp/alpine-chroot/mnt/target
mount -t proc none /tmp/alpine-chroot/proc
mount -t devtmpfs none /tmp/alpine-chroot/dev
mount -t sysfs none /tmp/alpine-chroot/sys

chroot /tmp/alpine-chroot sh

apk update
apk add alpine-conf sudo openssh busybox-initscripts syslinux

The Alpine init system will start up the core required services if the file /etc/.default_boot_services exists, so create that.

touch etc/.default_boot_services

Set up networking:

cat > /etc/network/interfaces <<EOF
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
EOF

ln -s /etc/init.d/networking /etc/runlevels/default/
ln -s /etc/init.d/sshd /etc/runlevels/default/

Add a user, set an SSH key, and ensure that this user can sudo:

adduser -D -s /bin/sh alpine
addgroup alpine wheel

mkdir /home/alpine/.ssh
chmod 700 /home/alpine/.ssh
chown alpine /home/alpine/.ssh
cat > /home/alpine/.ssh/authorized_keys <<EOF
... your ssh public key(s) ...
EOF
chmod 600 /home/alpine/.ssh/authorized_keys
chown alpine:alpine /home/alpine/.ssh/authorized_keys

passwd -u alpine

sed -i '/%wheel/s/^# //' /etc/sudoers

Exclude backup files and any files that are going to exist in the unpacked image. Then create an apkovl bundle.

lbu exclude \
    etc/group- etc/passwd- etc/shadow- \
    etc/apk/keys etc/apk/arch etc/apk/repositories \
    etc/os-release \
    etc/issue \
    etc/alpine-release
lbu include /home/alpine/.ssh
lbu package amazon.apkovl.tar.gz

Verify that everything you expect to be in the apovl file is there and if not you can modify the apkovl.

Finally, copy the apkovl to /mnt/target.

cp amazon.apkovl.tar.gz /mnt/target

EBS Backed HVM AMI

For HVM instances you will need to install a boot-loader. Any bootloader should do the trick but this guide will use syslinux because it's fast and small and you don't have to support any odd hardware or system layout.

First you will need to ensure that the ext4 driver is loaded and may as well shorten the timeout to speed instance boot time.

sed -i \ -e '/usb-storage/s//usb-storage,ext4/' \ -e '/^TIMEOUT/s/20/2/' \ /mnt/target/boot/syslinux/syslinux.cfg

Finally, install the bootloader:

extlinux -i /mnt/target/boot

EBS Backed PV AMI

For PV AMIs PV-GRUB will attempt to read /boot/grub/menu.lst to load the kernel. First create a /boot/grub/grub.conf:

mkdir -p /mnt/target/boot/grub

cat > target/boot/grub/grub.conf <<EOF 
default=0
timeout=2
hiddenmenu

title Alpine Linux
root (hd0)
kernel /boot/vmlinuz-virthardened alpine_dev=xvda1:ext4 modules=loop,squashfs,sd-mod,ext4 console=hvc0 pax_nouderef BOOT_IMAGE=/boot/vmlinuz-virthardened
initrd /boot/initramfs-virthardened
EOF

ln -s /mnt/target/boot/grub/grub.conf  /mnt/target/boot/grub/menu.lst

Create the AMI

The following will need to be done in the AWS console.

  1. Detach the new volume (make note of the volume ID)
  2. Launch a new instance, use the defaults; you are going to cannibalize it in a moment
  3. Once the instance starts, stop but do not terminate it
  4. Under EBS, detach the existing volume, and attach the Alpine Linux volume as /dev/xvda (for HVM) or /dev/sda1 (for PV)
  5. Restart the instance
  6. Log in and make sure it works
  7. Do any final cleanups necessary. Only make changes appropriate for an AMI, you are going to snapshot this instance and use it as the base for the AMI.
  8. Stop but do not terminate the instance
  9. Right click the stopped instance and choose 'Create Image (EBS AMI)'
  10. Once the AMI creation finishes you can cleanup the instances and EBS volumes used