Talk:LXC: Difference between revisions

From Alpine Linux
(→‎Alternative Network Setup: got macvlan bridge mode working)
(→‎What Works, What Doesnt: iptables works inside a container)
Line 108: Line 108:
** Can ping from one guest to another
** Can ping from one guest to another
** No communication allowed between host and guests (this is a plus in our case - managment vlan != user vlan)
** No communication allowed between host and guests (this is a plus in our case - managment vlan != user vlan)
** if iptables modules are loaded in the host, each guest can create its own iptables rules (awall for all! sweet)
* Con
* Con
** Real /sys is mounted - the guest can shut down the host<br />
** Real /sys is mounted - the guest can shut down the host<br />
in guest1:  echo /sbin/poweroff > /sys/kernel/uevent_helper<br />
in guest1:  echo /sbin/poweroff > /sys/kernel/uevent_helper<br />
in host: /etc/init.d/lxc.guest1 stop
in host: /etc/init.d/lxc.guest1 stop

Revision as of 23:58, 24 February 2013

Alternative Network Setup

These are notes on macvlan on a box with real vlans. The goal here is to have the host on a management vlan, and several guests each on other vlans. There's no need for the host to talk to the guests. I wanted to try to see if the guest could get dhcp addresses. Something like this:

Setup:

host dhcp on vlan 8
guest1 dhcp on vlan 64
guest2 dhcp on vlan 129
guest3 dhcp (different address) on vlan 64
  • Host's /etc/network/interfaces file
auto lo
iface lo inet loopback
 
# MGMT vlan
auto eth0.8
iface eth0.8 inet dhcp
     hostname lxchost

# USR vlan - we bring it up, but dont assign an address
auto eth0.65
iface eth0.65 inet manual
   up ip link set $IFACE addr de:ad:be:ef:ca:fe
   up ip link set $IFACE up
   down ip link set $IFACE down

# VoIP vlan - we bring it up, but dont assign an address
auto eth0.129
iface eth0.129 inet manual
   up ip link set $IFACE addr 0f:f1:ce:c0:ff:ee
   up ip link set $IFACE up
   down ip link set $IFACE down
  • Here's /etc/lxc/lxc.conf
lxc.network.type   =   macvlan
lxc.network.macvlan.mode = bridge  # allow guests on the same vlan to see each other
lxc.network.link   =   eth0.65
lxc.network.name   =   eth0
# lxc.network.flags  =   up       # We will bring the interface up inside the container
# lxc.network.ipv4   =   0.0.0.0  # We are going to do dhcp later
  • Create the guests
for a in `seq 1 3`; do 
  lxc-create -n guest${a} -f /etc/lxc/lxc.conf -t alpine
  ln -s /etc/init.d/lxc /etc/init.d/lxc.guest${a}
done
  • vi /var/lib/lxc/guest2/config
  change lxc.network.link to eth0.129
  • Start and enter the first guest (this is where the fun starts)
/etc/init.d/lxc.guest1 start
lxc-console -n guest1

Fun inside the guest

  • The /etc/networking/interfaces file is already set up for dhcp, so let's just restart networking:
guest1:~# /etc/init.d/networking restart
* Stopping networking ...
*   eth0 ...
cat: can't open '/var/run/udhcpc.eth0.pid': No such file or directory
ifdown: warning: no dhcp clients found and stopped  [ !! ]
 * Starting networking ...
 *   eth0 ...
cat: can't open '/sys/class/net/eth0/ifindex': No such file or directory
/usr/share/udhcpc/default.script: line 125: arithmetic syntax error
/usr/share/udhcpc/default.script: line 125: arithmetic syntax error 
  • But.. lookie there... we do have a real ip address.
  • The reason for the syntax errors is we don't have sys/class/net mounted... So let's mount it and try again....
guest1:~# mount -t sysfs none /sys
guest1:~# /etc/init.d/networking restart
 * Stopping networking ...
 *   eth0 ...   [ ok ]
 * Starting networking ...
 *   eth0 ...   [ ok ]
guest1:~# 
  • We just opened ourselves to a world of hurt. But more on that later
  • Let's see if we can make this 'just work'. We're going to do some weird things, don't worry... its not standard
 guest1:~# rc-update add networking
 guest1:~# echo "sysfs		/sys		sysfs	auto,defaults 0 0" >>/etc/fstab
 guest1:~# cat - << EOF >/etc/network/interfaces
   #auto lo
   iface lo inet loopback
   auto eth0
   iface eth0 inet dhcp
       pre-up /bin/mount -a
       hostname guest1
   EOF
 ctrl-a q
 lxchost# /etc/init.d/lxc.guest1 restart
 lxchost# lxc-console -n guest1
  • We have Networking!
  • Repeat the configuration for guest2 and 3

What Works, What Doesnt

  • Pro
    • Each guest has its own mac address
    • Can ping from one guest to another
    • No communication allowed between host and guests (this is a plus in our case - managment vlan != user vlan)
    • if iptables modules are loaded in the host, each guest can create its own iptables rules (awall for all! sweet)
  • Con
    • Real /sys is mounted - the guest can shut down the host

in guest1: echo /sbin/poweroff > /sys/kernel/uevent_helper
in host: /etc/init.d/lxc.guest1 stop