Talk:LXC - Revision history

Zcrayfish: Unsigned comment attribution. Separated some comments with HR. nowiki'd a dead link.

2023-08-28T01:48:57Z

Unsigned comment attribution. Separated some comments with HR. nowiki'd a dead link.

← Older revision		Revision as of 01:48, 28 August 2023
Line 54:		Line 54:

	# Capabilities to drop (for instance, to stop the guest from mounting sys)		# Capabilities to drop (for instance, to stop the guest from mounting sys)
	# Taken from http://sourceforge.net/mailarchive/message.php?msg_id=28285704		# Taken from <nowiki>http://sourceforge.net/mailarchive/message.php?msg_id=28285704</nowiki>
	# sys_boot is not listed here, as it causes problems when the host tries to stop the guest		# sys_boot is not listed here, as it causes problems when the host tries to stop the guest

Line 99:		Line 99:
	** No communication allowed between host and guests because we are not using a bridge interface (this is a plus in our case - managment vlan != user vlan)		** No communication allowed between host and guests because we are not using a bridge interface (this is a plus in our case - managment vlan != user vlan)

			<small><span class="autosigned">— Preceding [[Help:Signature\|unsigned]] comment added by [[User:Nangel\|Nangel]] ([[User talk:Nangel\|{{int:talkpagelinktext}}]] • [[Special:Contributions/Nangel\|{{int:contribslink}}]]) 02:00, 12 April 2013</span></small>
			-----
	== About lxc-attach ==		== About lxc-attach ==

Line 109:		Line 111:
	Or is it a bug in AL LXC?		Or is it a bug in AL LXC?

			<small><span class="autosigned">— Preceding [[Help:Signature\|unsigned]] comment added by [[User:Jch\|Jch]] ([[User talk:Jch\|{{int:talkpagelinktext}}]] • [[Special:Contributions/Jch\|{{int:contribslink}}]]) 17:11, 3 June 2015</span></small>‎
			-----

	=== Update about lxc-attach ===		=== Update about lxc-attach ===
Line 134:		Line 138:
	<pre>sysctl -p</pre>		<pre>sysctl -p</pre>

			<small><span class="autosigned">— Preceding [[Help:Signature\|unsigned]] comment added by [[User:Fcolista\|Fcolista]] ([[User talk:Fcolista\|{{int:talkpagelinktext}}]] • [[Special:Contributions/Fcolista\|{{int:contribslink}}]]) 02:22, 28 April 2016‎</span></small>
	== Unprivileged containers ==		== Unprivileged containers ==

Pickfire: Add unpriviledged containers with shadow-uidmap

2017-02-23T16:05:28Z

Add unpriviledged containers with shadow-uidmap

← Older revision		Revision as of 16:05, 23 February 2017
Line 133:		Line 133:

	<pre>sysctl -p</pre>		<pre>sysctl -p</pre>

			== Unprivileged containers ==

			To use unprivileged containers, one needs to install shadow-uidmap and add 'name:100000:65536' to both /etc/subuid and /etc/subgid. Or they will get errors like:

			unshare: Operation not permitted
			read pipe: Permission denied
			lxc-create: lxccontainer.c: do_create_container_dir: 985 Failed to chown container dir
			lxc-create: tools/lxc_create.c: main: 318 Error creating container test

			[[User:Pickfire\|Pickfire]] ([[User talk:Pickfire\|talk]]) 16:05, 23 February 2017 (UTC)

Fcolista: /* About lxc-attach */

2016-04-28T06:22:55Z

About lxc-attach

← Older revision		Revision as of 06:22, 28 April 2016
Line 108:		Line 108:
	What did I possibly wrong?<br/>		What did I possibly wrong?<br/>
	Or is it a bug in AL LXC?		Or is it a bug in AL LXC?


			=== Update about lxc-attach ===

			'''LXC-host: lxc-attach fail with "lxc_attach_to_ns: 270 Operation not permitted - failed to set namespace 'pid'"'''

			'''Issue:''' When you try to run lxc-attach, this fails. "use of CAP_SYS_ADMIN in chroot denied for /usr/bin/lxc-attach" appears in dmesg.<br/>
			'''Cause:''' This issue due to grsecurity restriction in the lxc host.<br/>
			'''Workaround:''' Add the following settings to your sysctl.conf file:<br/>
			<pre>
			kernel.grsecurity.chroot_caps=0
			kernel.grsecurity.chroot_deny_chmod=0
			</pre>

			Since those settings are read only at lxc host boot, and they have been applied in a second time, some of the lxc hosts might not have those settings loaded yet.
			A simple workaround can be:

			<pre>
			echo 0 > /proc/sys/kernel/grsecurity/chroot_caps
			echo 0 > /proc/sys/kernel/grsecurity/chroot_deny_chroot
			</pre>

			or simply run:

			<pre>sysctl -p</pre>

Jch: /* About lxc-attach */ new section

2015-06-03T21:11:05Z

About lxc-attach: new section

← Older revision		Revision as of 21:11, 3 June 2015
Line 98:		Line 98:
	* Con		* Con
	** No communication allowed between host and guests because we are not using a bridge interface (this is a plus in our case - managment vlan != user vlan)		** No communication allowed between host and guests because we are not using a bridge interface (this is a plus in our case - managment vlan != user vlan)

			== About lxc-attach ==

			I cannot conncect to any AL LXC build under AL... the response is always <pre>
			infra:~# lxc-attach --name=git -- "ps ax"
			lxc_container: attach.c: lxc_attach_to_ns: 196 Operation not permitted - failed to set namespace 'pid'
			lxc_container: attach.c: lxc_attach: 844 failed to enter the namespace
			</pre>
			What did I possibly wrong?<br/>
			Or is it a bug in AL LXC?

Nangel: /* Fun inside the guest */ Remove network start (fixed in 2.5 current) - add new note about missing device files

2013-04-12T06:00:12Z

Fun inside the guest: Remove network start (fixed in 2.5 current) - add new note about missing device files

← Older revision		Revision as of 06:00, 12 April 2013
Line 76:		Line 76:
	=== Fun inside the guest ===		=== Fun inside the guest ===

	~~If you just start networking, you'll get:~~		* /dev/null is currently created as a regular file
	~~cat: can't open '~~/~~sys~~/~~class/net/eth0/ifindex': No such~~ file ~~or directory~~		* /dev/zero doesn't exist
	~~/usr~~/~~share~~/~~udhcpc/default.script: line 125: arithmetic syntax error~~
	~~/usr/share/udhcpc/default.script: line 125: arithmetic syntax error~~
	~~This is because /sys is not mounted, and we don~~'t ~~[http://blog.bofh.it/debian/id_413 really want to do that].~~

	~~Fortunately~~, ~~there~~'~~s a simple fix:~~		To create these, do the following from ''the host''

	--- /~~usr~~/~~share~~/~~udhcpc~~/~~default.script.orig~~		<pre>
	~~+++~~ /~~usr~~/~~share~~/~~udhcpc/default.script~~		rm -f /var/lib/lxc/[guest-name]/rootfs/dev/null
	@@ -~~39,7 +39,7 @@~~		rm -f /var/lib/lxc/[guest-name]/rootfs/dev/zero
	}		mknod /var/lib/lxc/[guest-name]/rootfs/dev/zero c 1 5
			mknod /var/lib/lxc/[guest-name]/rootfs/dev/null c 1 3
	~~if_index() {~~		</pre>
	~~- cat~~ /~~sys~~/~~class~~/~~net~~/~~$interface~~/~~ifindex~~
	~~+ cat~~ /~~sys~~/~~class~~/~~net~~/~~$interface~~/~~ifindex \|\| echo 0~~		We do this in the host because our default config drops mknod capabilites in the guest.
	}

	~~calc_metric() {~~
	* Let's make the /~~etc~~/~~network~~/~~interfaces proper~~
	~~guest1:~# cat~~ - ~~<< EOF >~~/~~etc~~/~~network~~/~~interfaces~~
	~~auto lo~~
	~~iface lo inet loopback~~

	~~auto eth0~~
	~~iface eth0 inet dhcp~~
	~~hostname guest1~~
	~~EOF~~
	* And add networking:
	~~guest1:~# rc-update add networking~~
	* And test
	~~ctrl-a q~~
	~~lxchost#~~ /~~etc/init.d/lxc.guest1 restart~~
	~~lxchost# lxc-console -n guest1~~
	* We ~~have Networking!~~
	* Repeat the ~~patch and configuration for guest2 and 3~~

	=== What Works, What Doesnt ===		=== What Works, What Doesnt ===

Nangel: Found the problem in udhcp.default script - each guest now has networking, no sys mounting, and individual firewall rules.

2013-02-25T03:29:23Z

Found the problem in udhcp.default script - each guest now has networking, no sys mounting, and individual firewall rules.

← Older revision		Revision as of 03:29, 25 February 2013
Line 2:		Line 2:
	= Alternative Network Setup =		= Alternative Network Setup =

	These are notes on macvlan on a box with real vlans. The goal here is to have the host on a management vlan, and several guests each on other vlans. There's no need for the host to talk to the guests. ~~I wanted~~ to ~~try~~ to ~~see if the~~ guest ~~could~~ get ~~dhcp addresses~~. Something like this:		These are notes on macvlan on a box with real vlans. The goal here is to have the host on a management vlan, and several guests each on other vlans. There's no need for the host to talk to the guests. The host resides on the "OOB" network, and if the host needs to talk to a guest, it does so with lxc-console, like having a KVM. Each guest should get its address from the DHCP server on the appropriate vlan.Something like this:

	Setup:		Setup:
Line 17:		Line 17:
	\|-		\|-
	\| guest3		\| guest3
	\| dhcp (different address) ~~on vlan 64~~		\| dhcp on vlan64 (different address)
	\|}		\|}

Line 45:		Line 45:
	* Here's /etc/lxc/lxc.conf		* Here's /etc/lxc/lxc.conf
	lxc.network.type = macvlan		lxc.network.type = macvlan
	~~lxc.network.macvlan.mode = bridge~~ # ~~allow~~ guests on the same vlan to see each other		# Allow guests on the same vlan to see each other
	lxc.network.link = eth0.65		lxc.network.macvlan.mode = bridge
	lxc.network.name = eth0		lxc.network.link = eth0.65
	# lxc.network.flags = up # ~~We will~~ bring the interface ~~up inside~~ the container		lxc.network.name = eth0
	# lxc.network.ipv4 = 0.0.0.0 # ~~We are going~~ to ~~do dhcp later~~		# lxc.network.hwaddr = de:ad:be:ef:c0:00 # macvlan will make one up, but possible if wanted
			# lxc.network.flags = up # Do NOT bring up the interface, we will do so within the container
			# lxc.network.ipv4 = 0.0.0.0 # Do NOT assign an address, we do so within the container

			# Capabilities to drop (for instance, to stop the guest from mounting sys)
			# Taken from http://sourceforge.net/mailarchive/message.php?msg_id=28285704
			# sys_boot is not listed here, as it causes problems when the host tries to stop the guest

			# If you trust the guest, then you can get by without dropping capabilities

			lxc.cap.drop= sys_admin audit_control audit_write fsetid ipc_lock
			lxc.cap.drop= ipc_owner lease linux_immutable mac_admin mac_override mknod setfcap
			lxc.cap.drop= setpcap sys_module sys_nice sys_pacct sys_ptrace sys_rawio
			lxc.cap.drop= sys_tty_config sys_time
	* Create the guests		* Create the guests
	for a in `seq 1 3`; do		for a in `seq 1 3`; do
Line 57:		Line 70:
	* vi /var/lib/lxc/guest2/config		* vi /var/lib/lxc/guest2/config
	change lxc.network.link to eth0.129		change lxc.network.link to eth0.129

	* Start and enter the first guest (this is where the fun starts)		* Start and enter the first guest (this is where the fun starts)
	/etc/init.d/lxc.guest1 start		/etc/init.d/lxc.guest1 start
Line 64:		Line 76:
	=== Fun inside the guest ===		=== Fun inside the guest ===

	* The /etc/networking~~/interfaces file is already set up for dhcp~~, ~~so let's just restart networking:~~		If you just start networking, you'll get:
	~~guest1:~# /etc/init.d/networking restart~~
	* Stopping networking ...
	* eth0 ...
	~~cat: can't open '/var/run/udhcpc.eth0.pid~~': ~~No such file or directory~~
	~~ifdown: warning: no dhcp clients found and stopped [ !! ]~~
	* Starting networking ...
	* eth0 ...
	cat: can't open '/sys/class/net/eth0/ifindex': No such file or directory		cat: can't open '/sys/class/net/eth0/ifindex': No such file or directory
	/usr/share/udhcpc/default.script: line 125: arithmetic syntax error		/usr/share/udhcpc/default.script: line 125: arithmetic syntax error
	/usr/share/udhcpc/default.script: line 125: arithmetic syntax error		/usr/share/udhcpc/default.script: line 125: arithmetic syntax error
	* But.. lookie there... we do have a real ip address.		This is because /sys is not mounted, and we don't [http://blog.bofh.it/debian/id_413 really want to do that].
	* The reason for the syntax errors is we don't ~~have sys~~/~~class~~/~~net mounted~~... ~~So let~~'s ~~mount it and try again....~~
	~~guest1:~# mount~~ -~~t sysfs none~~ /~~sys~~		Fortunately, there's a simple fix:
	~~guest1:~#~~ /~~etc~~/~~init~~.~~d/networking restart~~
	* Stopping networking ...		--- /usr/share/udhcpc/default.script.orig
	* eth0 ... [ ok ]		+++ /usr/share/udhcpc/default.script
	* Starting networking ...		@@ -39,7 +39,7 @@
	* eth0 ... [ ok ]		}
	~~guest1:~#~~
	* We just opened ourselves to a [http://~~blog.bofh.it~~/~~debian~~/~~id_413 world of hurt]. But more on that later~~		if_index() {
	* Let's ~~see if we can~~ make ~~this 'just work'. We're going to do some weird things, don't worry... its not standard~~		- cat /sys/class/net/$interface/ifindex
	~~guest1:~# rc-update add networking~~		+ cat /sys/class/net/$interface/ifindex \|\| echo 0
	~~guest1:~# echo "sysfs~~ /~~sys sysfs auto,defaults 0 0" >>~~/~~etc~~/~~fstab~~		}

			calc_metric() {
			* Let's make the /etc/network/interfaces proper
	guest1:~# cat - << EOF >/etc/network/interfaces		guest1:~# cat - << EOF >/etc/network/interfaces
	#auto lo		auto lo
	iface lo inet loopback		iface lo inet loopback

	auto eth0		auto eth0
	iface eth0 inet dhcp		iface eth0 inet dhcp
	~~pre-up /bin/mount -a~~
	hostname guest1		hostname guest1
	EOF		EOF
			* And add networking:
			guest1:~# rc-update add networking
			* And test
	ctrl-a q		ctrl-a q
	lxchost# /etc/init.d/lxc.guest1 restart		lxchost# /etc/init.d/lxc.guest1 restart
	lxchost# lxc-console -n guest1		lxchost# lxc-console -n guest1
	* We have Networking!		* We have Networking!
	* Repeat the configuration for guest2 and 3		* Repeat the patch and configuration for guest2 and 3

	=== What Works, What Doesnt ===		=== What Works, What Doesnt ===
	* Pro		* Pro
	** Each guest has its own mac address		** Each guest has its own mac address
	** ~~Can ping from one~~ guest ~~to another~~		** Network connectivity between each guest
	** No communication allowed between host and guests (this is a plus in our case - managment vlan != user vlan)		** No communication allowed between host and guests (this is a plus in our case - managment vlan != user vlan)
	** if iptables modules are loaded in the host, each guest can create its own iptables rules (awall for all! sweet)		** if iptables modules are loaded in the host, each guest can create its own iptables rules (awall for all! sweet)
	* Con		* Con
	** ~~Real /sys~~ is ~~mounted~~ - ~~the guest can shut down the host<br />~~		** No communication allowed between host and guests because we are not using a bridge interface (this is a plus in our case - managment vlan != user vlan)
	~~in guest1: echo /sbin/poweroff > /sys/kernel/uevent_helper<br />~~
	~~in host: /etc/init.d/lxc.guest1 stop~~

Nangel: /* What Works, What Doesnt */ iptables works inside a container

2013-02-24T23:58:53Z

What Works, What Doesnt: iptables works inside a container

← Older revision		Revision as of 23:58, 24 February 2013
Line 108:		Line 108:
	** Can ping from one guest to another		** Can ping from one guest to another
	** No communication allowed between host and guests (this is a plus in our case - managment vlan != user vlan)		** No communication allowed between host and guests (this is a plus in our case - managment vlan != user vlan)
			** if iptables modules are loaded in the host, each guest can create its own iptables rules (awall for all! sweet)
	* Con		* Con
	** Real /sys is mounted - the guest can shut down the host<br />		** Real /sys is mounted - the guest can shut down the host<br />
	in guest1: echo /sbin/poweroff > /sys/kernel/uevent_helper<br />		in guest1: echo /sbin/poweroff > /sys/kernel/uevent_helper<br />
	in host: /etc/init.d/lxc.guest1 stop		in host: /etc/init.d/lxc.guest1 stop

Nangel: /* Alternative Network Setup */ got macvlan bridge mode working

2013-02-24T23:27:31Z

Alternative Network Setup: got macvlan bridge mode working

← Older revision		Revision as of 23:27, 24 February 2013
Line 45:		Line 45:
	* Here's /etc/lxc/lxc.conf		* Here's /etc/lxc/lxc.conf
	lxc.network.type = macvlan		lxc.network.type = macvlan
			lxc.network.macvlan.mode = bridge # allow guests on the same vlan to see each other
	lxc.network.link = eth0.65		lxc.network.link = eth0.65
	lxc.network.name = eth0		lxc.network.name = eth0
Line 105:		Line 106:
	* Pro		* Pro
	** Each guest has its own mac address		** Each guest has its own mac address
			** Can ping from one guest to another
			** No communication allowed between host and guests (this is a plus in our case - managment vlan != user vlan)
	* Con		* Con
	** Can't ping from one guest to another		** Real /sys is mounted - the guest can shut down the host<br />
	** Real /sys is mounted - the guest can shut down the host		in guest1: echo /sbin/poweroff > /sys/kernel/uevent_helper<br />
			in host: /etc/init.d/lxc.guest1 stop

Nangel: Notes on macvlan networking

2013-02-24T22:21:21Z

Notes on macvlan networking

New page

= Alternative Network Setup =

These are notes on macvlan on a box with real vlans. The goal here is to have the host on a management vlan, and several guests each on other vlans. There's no need for the host to talk to the guests. I wanted to try to see if the guest could get dhcp addresses. Something like this:

Setup:
{| class="wikitable" border="1"
|-
| host
| dhcp on vlan 8
|-
| guest1
| dhcp on vlan 64
|-
| guest2
| dhcp on vlan 129
|-
| guest3
| dhcp (different address) on vlan 64
|}

* Host's /etc/network/interfaces file
auto lo
iface lo inet loopback

# MGMT vlan
auto eth0.8
iface eth0.8 inet dhcp
hostname lxchost

# USR vlan - we bring it up, but dont assign an address
auto eth0.65
iface eth0.65 inet manual
up ip link set $IFACE addr de:ad:be:ef:ca:fe
up ip link set $IFACE up
down ip link set $IFACE down

# VoIP vlan - we bring it up, but dont assign an address
auto eth0.129
iface eth0.129 inet manual
up ip link set $IFACE addr 0f:f1:ce:c0:ff:ee
up ip link set $IFACE up
down ip link set $IFACE down

* Here's /etc/lxc/lxc.conf
lxc.network.type = macvlan
lxc.network.link = eth0.65
lxc.network.name = eth0
# lxc.network.flags = up # We will bring the interface up inside the container
# lxc.network.ipv4 = 0.0.0.0 # We are going to do dhcp later
* Create the guests
for a in `seq 1 3`; do
lxc-create -n guest${a} -f /etc/lxc/lxc.conf -t alpine
ln -s /etc/init.d/lxc /etc/init.d/lxc.guest${a}
done
* vi /var/lib/lxc/guest2/config
change lxc.network.link to eth0.129

* Start and enter the first guest (this is where the fun starts)
/etc/init.d/lxc.guest1 start
lxc-console -n guest1

=== Fun inside the guest ===

* The /etc/networking/interfaces file is already set up for dhcp, so let's just restart networking:
guest1:~# /etc/init.d/networking restart
* Stopping networking ...
* eth0 ...
cat: can't open '/var/run/udhcpc.eth0.pid': No such file or directory
ifdown: warning: no dhcp clients found and stopped [ !! ]
* Starting networking ...
* eth0 ...
cat: can't open '/sys/class/net/eth0/ifindex': No such file or directory
/usr/share/udhcpc/default.script: line 125: arithmetic syntax error
/usr/share/udhcpc/default.script: line 125: arithmetic syntax error
* But.. lookie there... we do have a real ip address.
* The reason for the syntax errors is we don't have sys/class/net mounted... So let's mount it and try again....
guest1:~# mount -t sysfs none /sys
guest1:~# /etc/init.d/networking restart
* Stopping networking ...
* eth0 ... [ ok ]
* Starting networking ...
* eth0 ... [ ok ]
guest1:~#
* We just opened ourselves to a [http://blog.bofh.it/debian/id_413 world of hurt]. But more on that later
* Let's see if we can make this 'just work'. We're going to do some weird things, don't worry... its not standard
guest1:~# rc-update add networking
guest1:~# echo "sysfs /sys sysfs auto,defaults 0 0" >>/etc/fstab
guest1:~# cat - << EOF >/etc/network/interfaces
#auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp
pre-up /bin/mount -a
hostname guest1
EOF
ctrl-a q
lxchost# /etc/init.d/lxc.guest1 restart
lxchost# lxc-console -n guest1
* We have Networking!
* Repeat the configuration for guest2 and 3

=== What Works, What Doesnt ===
* Pro
** Each guest has its own mac address
* Con
** Can't ping from one guest to another
** Real /sys is mounted - the guest can shut down the host