Fault Tolerant Routing with Alpine Linux
This document explains how to setup a fault-tolerant router using Alpine Linux. It has been tested with Alpine Linux 2.2.3.
Hardware and Network Setup
The network used in this example is as follows:
- Will run (at least initially) IPv4
- Pre-existing border router that performs NAT from public IP(s) to the 10.0.0.0/8 network and has the address 10.0.0.1/24 on the transit network
- The border router will have a default route to the internal network via 10.0.0.2 (the virtual IP address of the routers being set up in this doc)
- The transit network between the border router and the fault-tolerant routers described in this document, will be on 10.0.0.0/24
- The routers will also connect several internal subnets on the network:
- It's assumed that 10.0.0.0/24 and 10.0.1.0/24 are connected via dedicated interfaces (eth0 and eth1, respectively), while 10.0.2.0/24 and 10.0.3.0/24 share an interface(eth2), but traffic is segregated using 802.1q vlan tagging, with 10.0.2.0/24 using vlan id 2, and 10.0.3.0/24 using vlan id 3.
- Finally, all computers in subnets 10.0.1.0/24, 10.0.2.0/24 and 10.0.3.0/24 are set up with a default gateway of 10.0.x.1
Two computers will be needed, with at least three NICs in them (more if you are connecting more network segments together), to act as routers.
First, setup Alpine on a USB key or CF card on both computers. Connect both computers initially to 10.0.0.0/24 on eth0, and assign them ip addresses of 10.0.0.3/24 and 10.0.0.4/24 in the setup-alpine script(for router1 and router2, respectively). Do not configure other interfaces initially. Ensure that both machines are pingable.
Next, connect them both to 10.0.1.0/24 on eth1, and assign them ip addresses of 10.0.1.2/24 and 10.0.1.3/24, respectively. Ensure that they can also ping each other. At this point, /etc/network/interfaces (for router1) should look like this:
auto eth0 iface eth0 inet static address 10.0.0.2 netmask 255.255.255.0 gateway 10.0.0.1 auto eth1 iface eth1 inet static address 10.0.1.2 netmask 255.255.255.0
Finally, get the last two networks connected (the ip addresses given are for router1. The following steps should be performed on router2):
modprobe 8021q echo "8021q" >> /etc/modules cat >> /etc/network/interfaces << EOF auto eth2 iface eth2 inet manual up ip link set eth2 up up ifup eth2.2 || true up ifup eth2.3 || true down ifdown eth2.3 || true down ifdown eth2.2 || true down ip link set dev eth2 down iface eth2.2 inet static pre-up vconfig add eth2 2 address 10.0.2.2 netmask 255.255.255.0 post-down vconfig rem $IFACE iface eth2.3 inet static pre-up vconfig add eth2 3 address 10.0.3.2 netmask 255.255.255.0 post-down vconfig rem $IFACE EOF
Test that you can also ping between the two interfaces.
Start ip forwarding
Next, turn them into simple routers by enabling ip forwarding (do that on each box):
If you follow the ucarp section below, you'll also need to disable rp_filter (RFC3704 Ingress Filtering). Since this how-to is designed for an internal router this should (or might not) be acceptable. Available options for this setting can be viewed at http://git.kernel.org/?p=linux/kernel/git/torvalds/linux-2.6.git;a=blob;f=Documentation/networking/ip-sysctl.txt;hb=HEAD#l855.
Install and start ucarp
Ucarp will provide a virtual IP address for each subnet that the routers will share. That way, if either router fails, network connectivity stays up.
- Copy the scripts for each the interface:
- edit the /etc/conf/ucarp.eth0 file:
REALIP= VHID=1 VIP=10.0.0.2 PASSWORD=Password
- edit the /etc/conf/ucarp.eth1 file:
REALIP= VHID=2 VIP=10.0.1.1 PASSWORD=Password
- edit the /etc/conf/ucarp.eth2.2 file:
REALIP= VHID=3 VIP=10.0.2.1 PASSWORD=Password
- edit the /etc/conf/ucarp.eth2.3 file:
REALIP= VHID=4 VIP=10.0.3.1 PASSWORD=Password
- Create etc/ucarp/vip-up-eth0.sh (and copy this script for each interface: vip-up-eth1.sh, vip-up-eth2.2.sh, vip-up-eth2.3.sh):
#!/bin/sh # Add the VIP address ip addr add $2/24 dev $1 for a in 330 440 550; do beep -f $a -l 100; done
- Create /etc/ucarp/vip-down-eth0.sh (and copy this script for each interface: vip-down-eth1.sh, vip-down-eth2.2.sh, vip-down-eth2.3.sh):
#!/bin/sh # Remove the VIP address ip addr del $2/24 dev $1 for a in 550 440 330; do beep -f $a -l 100; done
- Make the scripts executable
- Start ucarp and save the changes
- Do the above steps for each router
At this point, you should have connectivity from your border router through to hosts on the internal subnets, and hosts on your subnets should be able to ping all interfaces on each router.
Install and configure Shorewall iptables frontend
We will refer to the Shorewall zones by letters to keep the config files simple. They will correspond to subnets and interfaces as follows:
A = eth0 = 10.0.0.0/24 B = eth1 = 10.0.1.0/24 C_2 = eth2.2 = 10.0.2.0/24 C_3 = eth2.3 = 10.0.3.0/24
Install the needed package
#FIRST LINE A_IF=eth0 B_IF=eth1 C_2_IF=eth2.2 C_3_IF=eth2.3 #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONCE - DO NOT REMOVE
#FIRST LINE A $A_IF detect dhcp B $B_IF detect dhcp C_2 $C_2_IF detect dhcp C_3 $C_3_IF detect dhcp #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONCE - DO NOT REMOVE
#FIRST LINE fw firewall A ipv4 B ipv4 C_2 ipv4 C_3 ipv4 #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONCE - DO NOT REMOVE
Edit /etc/shorewall/policy: (note: this allows all traffic between internal subnets and all traffic outbound to the Internet, but blocks all traffic inbound by default, this policy might need to be adjusted in your case)
#FIRST LINE A all REJECT INFO B A ACCEPT B C_2 ACCEPT B C_3 ACCEPT C_2 A ACCEPT C_2 B ACCEPT C_2 C_3 ACCEPT C_3 A ACCEPT C_3 B ACCEPT C_3 C_2 ACCEPT all all REJECT INFO #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONCE - DO NOT REMOVE
Edit /etc/shorewall/rules: (the following simply allows SSH traffic in to your two routers - more will probably be needed to accept or reject traffic based on your needs)
#FIRST LINE ACCEPT A fw tcp 22 #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONCE - DO NOT REMOVE
#FIRST LINE /bin/echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter /bin/echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter /bin/echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter /bin/echo 0 > /proc/sys/net/ipv4/conf/eth2.2/rp_filter /bin/echo 0 > /proc/sys/net/ipv4/conf/eth2.3/rp_filter #LAST LINE - ADD YOUR ENTRIES ABOVE THIS ONCE - DO NOT REMOVE
Finally, edit STARTUP_ENABLED line in /etc/shorewall/shorewall.conf:
Check config for errors then start shorewall:
This will allow DHCP broadcasts from one subnet to be sent on to a DHCP server in another subnet (that way your DHCP server won't need an interface in each subnet).
First, install dhcrelay:
Next, adjust /etc/conf.d/dhcrelay: (assuming you don't need DHCP on the 10.0.0.0/24 subnet, and that your DHCP servers are 10.0.1.100 and 10.0.1.101)
IFACE="eth1 eth2.2 eth2.3" DHCRELAY_SERVERS="10.0.1.100 10.0.1.101"
Finally, set up your scopes on 10.0.1.100 and 10.0.1.101 for the 10.0.1.0/24, 10.0.2.0/24, and 10.0.3.0/24 subnets and test by requesting a lease from a computer in either 10.0.2.0/24 or 10.0.3.0/24.