Difference between revisions of "KVM"

From Alpine Linux
Jump to: navigation, search
m
 
(12 intermediate revisions by 7 users not shown)
Line 1: Line 1:
[https://www.linux-kvm.org/page/Main_Page KVM] is an open source virtualization solution in a kernel module. KVM can virtualize x86, PowerPC, and S390 guests.
+
[https://www.linux-kvm.org/page/Main_Page KVM] is an free and open source virtualization solution in a kernel module. Although it is often simply referred to as KVM, the actual hypervisor is [https://www.qemu.org QEMU]. QEMU runs from user-space, but can integrate with KVM, providing better performance by leveraging the hardware from kernel-space. QEMU can virtualize x86, PowerPC, and S390 guests, amongst others. [https://libvirt.org Libvirt] is a management framework that integrates with QEMU/KVM, [https://wiki.alpinelinux.org/wiki/LXC LXC], [https://wiki.alpinelinux.org/wiki/Xen_Dom0 Xen] and others.
  
 +
== Installation ==
 +
The following commands provide '''libvirt''' as well as '''QEMU with emulation for x86_64''' and '''qemu-img''', a necessary component for using various disk formats such as qcow2. Without qemu-img, only raw disks are available. It can also convert images between several formats like vhdx and vmdk.
 +
{{Cmd|<nowiki># apk add libvirt-daemon qemu-img qemu-system-x86_64
 +
# rc-update add libvirtd</nowiki>}}
  
= Installing KVM =
+
== Networking ==
It is possible to install just qemu and libvirt for local management, but most users will probably want libvirt-deamon and/or virt-manager for a GUI.  
+
By default, libvirt uses NAT for VM connectivity. If you want to use the default configuration, you need to load the tun module.
{{Cmd|sudo apk add qemu-system-x86_64 libvirt
+
{{Cmd|# modprobe tun}}
sudo apk add libvirt-daemon dbus}}
 
  
Enable the KVM kernel module
+
If you prefer bridging a guest over your Ethernet interface, you need to make a [https://wiki.alpinelinux.org/wiki/Bridge#Configuration_file bridge].
{{Cmd|sudo modprobe kvm-intel}}
 
or
 
{{Cmd|sudo modprobe kvm-amd}}
 
  
 +
It's quite common to use bridges with KVM environments but when IPv6 is used, Alpine will assign itself a link-local address as well as an SLAAC address in case there's a router sending Router Advertisements. You don't want this because you don't want to have the KVM host an IP address in every network it serves to guests. Unfortunately you cannot just disable IPv6 for the bridge via sysctl because it might not be up at boottime when sysctl fires. There's a workaround though which is to put a post-up hook into the /etc/network/interfaces file like this:
 +
auto brlan
 +
iface brlan inet manual
 +
        bridge-ports eth1.5
 +
        bridge-stp 0
 +
        post-up ip -6 a flush dev brlan; sysctl -w net.ipv6.conf.brlan.disable_ipv6=1
  
Add your user to the follwing groups so you can mange the vms
+
== Management ==
{{Cmd|sudo addgroup user kvm
+
For (non-root) management, you will need to add your user to the libvirt group.
sudo addgroup user qemu
+
{{Cmd|# addgroup user libvirt}}
sudo addgroup user libvirt}}
 
  
 +
You can use libvirt's virsh on the CLI. It can execute commands as well as run as an interactive shell. Read its manual page and/or use the "help" command for more info. Some basic commands are:
 +
 +
{{Cmd|<nowiki>virsh help
 +
virsh list --all
 +
virsh start $domain
 +
virsh shutdown $domain</nowiki>
 +
}}
 +
 +
The libvirt project provides a GUI for managing hosts, called virt-manager. It handles local systems as well as remote ones via SSH.
 +
{{Cmd|<nowiki># apk add dbus polkit virt-manager
 +
# rc-update add dbus</nowiki>}}
 +
 +
In order to use libvirtd to remotely control KVM over ssh PolicyKit needs a .pkla informing it that this is allowed.
 +
Write the following file to /etc/polkit-1/localauthority/50-local.d/50-libvirt-ssh-remote-access-policy.pkla
 +
{{Cmd|<nowiki>[Remote libvirt SSH access]
 +
Identity=unix-group:libvirt
 +
Action=org.libvirt.unix.manage
 +
ResultAny=yes
 +
ResultInactive=yes
 +
ResultActive=yes</nowiki>
 +
}}
 +
 +
== vfio ==
 +
 +
VFIO is more flexible way to do PCI passthrough. Let's suppose you want to use following ethernet card as PCI device in a VM.
 +
 +
# lspci | grep 02:00.0
 +
02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
 +
# lspci -n -s 02:00.0
 +
02:00.0 0200: 8086:10c9 (rev 01)
 +
 +
Modify following file to instruct ''mkinitfs'' to load following module with the options and rebuild kernel ramdisk.
 +
 +
# cat /etc/modprobe.d/vfio.conf <<EOF
 +
options vfio-pci ids=8086:10c9
 +
options vfio_iommu_type1 allow_unsafe_interrupts=1
 +
softdep igb pre: vfio-pci
 +
EOF
 +
# mkinitfs
 +
 +
Now modify GRUB, include ''intel_iommu=o iommu=pt'' for Intel platform (AMD uses ''amd_iommu=on'') and add VFIO option for your device.
 +
 +
# grep ^GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub
 +
GRUB_CMDLINE_LINUX_DEFAULT="modules=sd-mod,usb-storage,ext4,raid1 nomodeset rootfstype=ext4 intel_iommu=on iommu=pt console=ttyS0,115200 vfio-pci.ids=8086:10c9"
 +
# grub-mkconfig /boot/grub/grub.cfg
 +
 +
Reboot and check dmesg.
 +
 +
# grep -i -e DMAR -e IOMMU /var/log/dmesg
 +
[    0.343795] DMAR: Host address width 36
 +
[    0.343797] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
 +
[    0.343804] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c90780106f0462 ecap f020e3
 +
[    0.343806] DMAR: RMRR base: 0x000000000ed000 end: 0x000000000effff
 +
[    0.343807] DMAR: RMRR base: 0x000000bf7ed000 end: 0x000000bf7fffff
 +
[    0.553830] iommu: Default domain type: Passthrough (set via kernel command line)
 +
[    0.902477] DMAR: No ATSR found
 +
[    0.902563] DMAR: dmar0: Using Queued invalidation
 +
...
 +
[    0.903256] pci 0000:02:00.0: Adding to iommu group 12
 +
...
 +
[    0.903768] DMAR: Intel(R) Virtualization Technology for Directed I/O
 +
 +
If you do not run libvirt VMs under ''root'' (''egrep '^#*user' /etc/libvirt/qemu.conf''), then you must have correct permission on ''/dev/vfio/<iommu_group>'', eg. ''/dev/vfio/12''. You have to tune ''/etc/mdev.conf'' or UDEV rules.
 +
 +
# virsh dumpxml vm01 | xmllint --xpath '//*/hostdev' -
 +
<hostdev mode="subsystem" type="pci" managed="yes">
 +
      <driver name="vfio"/>
 +
      <source>
 +
        <address domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
 +
      </source>
 +
      <alias name="hostdev0"/>
 +
      <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/>
 +
    </hostdev>
 +
<hostdev mode="subsystem" type="pci" managed="yes">
 +
      <driver name="vfio"/>
 +
      <source>
 +
        <address domain="0x0000" bus="0x02" slot="0x00" function="0x1"/>
 +
      </source>
 +
      <alias name="hostdev1"/>
 +
      <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
 +
    </hostdev>
 +
 +
A lot of info at [https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF].
  
 
[[Category:Virtualization]]
 
[[Category:Virtualization]]

Latest revision as of 22:29, 14 April 2020

KVM is an free and open source virtualization solution in a kernel module. Although it is often simply referred to as KVM, the actual hypervisor is QEMU. QEMU runs from user-space, but can integrate with KVM, providing better performance by leveraging the hardware from kernel-space. QEMU can virtualize x86, PowerPC, and S390 guests, amongst others. Libvirt is a management framework that integrates with QEMU/KVM, LXC, Xen and others.

Installation

The following commands provide libvirt as well as QEMU with emulation for x86_64 and qemu-img, a necessary component for using various disk formats such as qcow2. Without qemu-img, only raw disks are available. It can also convert images between several formats like vhdx and vmdk.

# apk add libvirt-daemon qemu-img qemu-system-x86_64 # rc-update add libvirtd

Networking

By default, libvirt uses NAT for VM connectivity. If you want to use the default configuration, you need to load the tun module.

# modprobe tun

If you prefer bridging a guest over your Ethernet interface, you need to make a bridge.

It's quite common to use bridges with KVM environments but when IPv6 is used, Alpine will assign itself a link-local address as well as an SLAAC address in case there's a router sending Router Advertisements. You don't want this because you don't want to have the KVM host an IP address in every network it serves to guests. Unfortunately you cannot just disable IPv6 for the bridge via sysctl because it might not be up at boottime when sysctl fires. There's a workaround though which is to put a post-up hook into the /etc/network/interfaces file like this:

auto brlan
iface brlan inet manual
       bridge-ports eth1.5
       bridge-stp 0
       post-up ip -6 a flush dev brlan; sysctl -w net.ipv6.conf.brlan.disable_ipv6=1

Management

For (non-root) management, you will need to add your user to the libvirt group.

# addgroup user libvirt

You can use libvirt's virsh on the CLI. It can execute commands as well as run as an interactive shell. Read its manual page and/or use the "help" command for more info. Some basic commands are:

virsh help virsh list --all virsh start $domain virsh shutdown $domain

The libvirt project provides a GUI for managing hosts, called virt-manager. It handles local systems as well as remote ones via SSH.

# apk add dbus polkit virt-manager # rc-update add dbus

In order to use libvirtd to remotely control KVM over ssh PolicyKit needs a .pkla informing it that this is allowed. Write the following file to /etc/polkit-1/localauthority/50-local.d/50-libvirt-ssh-remote-access-policy.pkla

[Remote libvirt SSH access] Identity=unix-group:libvirt Action=org.libvirt.unix.manage ResultAny=yes ResultInactive=yes ResultActive=yes

vfio

VFIO is more flexible way to do PCI passthrough. Let's suppose you want to use following ethernet card as PCI device in a VM.

# lspci | grep 02:00.0
02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
# lspci -n -s 02:00.0
02:00.0 0200: 8086:10c9 (rev 01)

Modify following file to instruct mkinitfs to load following module with the options and rebuild kernel ramdisk.

# cat /etc/modprobe.d/vfio.conf <<EOF
options vfio-pci ids=8086:10c9
options vfio_iommu_type1 allow_unsafe_interrupts=1
softdep igb pre: vfio-pci
EOF
# mkinitfs

Now modify GRUB, include intel_iommu=o iommu=pt for Intel platform (AMD uses amd_iommu=on) and add VFIO option for your device.

# grep ^GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="modules=sd-mod,usb-storage,ext4,raid1 nomodeset rootfstype=ext4 intel_iommu=on iommu=pt console=ttyS0,115200 vfio-pci.ids=8086:10c9"
# grub-mkconfig /boot/grub/grub.cfg

Reboot and check dmesg.

# grep -i -e DMAR -e IOMMU /var/log/dmesg
[    0.343795] DMAR: Host address width 36
[    0.343797] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    0.343804] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap c90780106f0462 ecap f020e3
[    0.343806] DMAR: RMRR base: 0x000000000ed000 end: 0x000000000effff
[    0.343807] DMAR: RMRR base: 0x000000bf7ed000 end: 0x000000bf7fffff
[    0.553830] iommu: Default domain type: Passthrough (set via kernel command line)
[    0.902477] DMAR: No ATSR found
[    0.902563] DMAR: dmar0: Using Queued invalidation
...
[    0.903256] pci 0000:02:00.0: Adding to iommu group 12
...
[    0.903768] DMAR: Intel(R) Virtualization Technology for Directed I/O

If you do not run libvirt VMs under root (egrep '^#*user' /etc/libvirt/qemu.conf), then you must have correct permission on /dev/vfio/<iommu_group>, eg. /dev/vfio/12. You have to tune /etc/mdev.conf or UDEV rules.

# virsh dumpxml vm01 | xmllint --xpath '//*/hostdev' -
<hostdev mode="subsystem" type="pci" managed="yes">
      <driver name="vfio"/>
      <source>
        <address domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
      </source>
      <alias name="hostdev0"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x06" function="0x0"/>
    </hostdev>
<hostdev mode="subsystem" type="pci" managed="yes">
      <driver name="vfio"/>
      <source>
        <address domain="0x0000" bus="0x02" slot="0x00" function="0x1"/>
      </source>
      <alias name="hostdev1"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x08" function="0x0"/>
    </hostdev>

A lot of info at [1].