Multi ISP - Revision history

Bt129: /* Shorewall Providers */

2021-07-26T19:57:47Z

Shorewall Providers

← Older revision		Revision as of 19:57, 26 July 2021
Line 92:		Line 92:


	We do '''not''' want shorewall to mess with the routing tables (which we needed it to do with older versions of pingu). We can tell shorewall not to flush the routing tables that pingu will manage by setting DUPLICATE column to '-', GATEWAY column to 'none' and ~~let~~ COPY column be empty.		We do '''not''' want shorewall to mess with the routing tables (which we needed it to do with older versions of pingu). We can tell shorewall not to flush the routing tables that pingu will manage by setting DUPLICATE column to '-', GATEWAY column to 'none' and leaving the COPY column empty.

	In this example eth0 is attatched to ISP1, eth3 is attatched to ISP2.		In this example eth0 is attatched to ISP1, eth3 is attatched to ISP2.

Bt129: /* Shorewall Interfaces */

2021-07-26T19:55:37Z

Shorewall Interfaces

← Older revision		Revision as of 19:55, 26 July 2021
Line 66:		Line 66:
	LAN_IF=eth2		LAN_IF=eth2

	Add it ~~also~~ to ''/etc/shorewall/interfaces'':		Add it to ''/etc/shorewall/interfaces'':

	INET $ISP1_IF detect dhcp		INET $ISP1_IF detect dhcp
Line 73:		Line 73:
	LAN $LAN_IF detect dhcp		LAN $LAN_IF detect dhcp

	The dhcp option is not really needed unless dhcp is actually used. We recommend leaving the dhcp option on in case the ISP changes to DHCP ~~in the future~~.		The dhcp option is not really needed unless dhcp is actually used. We recommend leaving the dhcp option on in case the ISP changes to DHCP.

	== Shorewall Masquerade ==		== Shorewall Masquerade ==

Fab: transform into table

2014-06-11T15:36:38Z

transform into table

@@ Line 3: / Line 3: @@
 '''Please note:''' Alpine Linux v2.4.4 or newer is required.
- * eth0 = ''ISP1'' - Internet with static IP address (we will use 1.2.3.4/24 and gw 1.2.3.1 as example)
+{| cellpadding="5" border="1" class="wikitable"
- * eth1 = DMZ (192.168.0.1)
+|-
- * eth2 = LAN (192.168.1.1)
+! Interface
- * eth3 = ''ISP2'' - Internet with dynamic IP address.
+! Name
 == Network Interfaces ==

Dubiousjim: add Category:Networking

2012-09-23T01:17:34Z

add Category:Networking

← Older revision		Revision as of 01:17, 23 September 2012
Line 183:		Line 183:

	...		...

			[[Category:Networking]]

Larena: formatting fix

2012-07-17T14:53:56Z

formatting fix

← Older revision		Revision as of 14:53, 17 July 2012
Line 15:		Line 15:
	auto eth0		auto eth0
	iface eth0 inet static		iface eth0 inet static
	address 1.2.3.4		address 1.2.3.4
	netmask 255.255.255.0		netmask 255.255.255.0
	gateway 1.2.3.1		gateway 1.2.3.1
	metric 200		metric 200


	The DHCP client will automatically add a metric value to the default gateway. It will pick 200 + interface index. Therefore we don't need to worry about that for DHCP.		The DHCP client will automatically add a metric value to the default gateway. It will pick 200 + interface index. Therefore we don't need to worry about that for DHCP.


	auto eth3		auto eth3
	iface eth0 inet dhcp		iface eth0 inet dhcp


	For PPP interfaces there is a ''defaultroute-metric'' option that can set the metric value.		For PPP interfaces there is a ''defaultroute-metric'' option that can set the metric value.

Larena: typos

2012-07-17T14:52:41Z

typos

← Older revision		Revision as of 14:52, 17 July 2012
Line 58:		Line 58:
	We must masquerade the internet addresses so that every ISP gets a correct source address. This is done in ''/etc/shorewall/masq''.		We must masquerade the internet addresses so that every ISP gets a correct source address. This is done in ''/etc/shorewall/masq''.

	$~~ISP1~~ 0.0.0.0/0		$ISP1_IF 0.0.0.0/0
	$~~ISP2~~ 0.0.0.0/0		$ISP2_IF 0.0.0.0/0

	== Shorewall Policy ==		== Shorewall Policy ==

Larena: formatting fix

2012-07-17T14:52:14Z

formatting fix

← Older revision		Revision as of 14:52, 17 July 2012
Line 155:		Line 155:
	# ping google dns via ISP1		# ping google dns via ISP1
	host 8.8.8.8 {		host 8.8.8.8 {
	interval 60		interval 60
	bind-interface eth0		bind-interface eth0
	}		}

	# ping opendns via ISP1		# ping opendns via ISP1
	host 208.67.222.222 {		host 208.67.222.222 {
	interval 60		interval 60
	bind-interface eth0		bind-interface eth0
	}		}

Larena: first draft

2012-07-17T14:48:06Z

first draft

New page

This document describes how to configure your Alpine Linux router with multiple ISPs, with or without failover and load balancing.

'''Please note:''' Alpine Linux v2.4.4 or newer is required.

* eth0 = ''ISP1'' - Internet with static IP address (we will use 1.2.3.4/24 and gw 1.2.3.1 as example)
* eth1 = DMZ (192.168.0.1)
* eth2 = LAN (192.168.1.1)
* eth3 = ''ISP2'' - Internet with dynamic IP address.

== Network Interfaces ==
If there are multiple static IP addresses, or to be exact, if there are multiple default gateways, you must specify a ''metric'' value. The gateway with the lowest metric will win.

''/etc/network/interfaces'':

auto eth0
iface eth0 inet static
address 1.2.3.4
netmask 255.255.255.0
gateway 1.2.3.1
metric 200

The DHCP client will automatically add a metric value to the default gateway. It will pick 200 + interface index. Therefore we don't need to worry about that for DHCP.

auto eth3
iface eth0 inet dhcp

For PPP interfaces there is a ''defaultroute-metric'' option that can set the metric value.

==Shorewall Zones==
In ''/etc/shorewall/zones'', create the zones:

INET ipv4
DMZ ipv4
LAN ipv4

== Shorewall Interfaces ==
We need to make shorewall aware of the second ISP interface.
In ''/etc/shorewall/params'', add ISP1 under ISP2.

ISP1_IF=eth0
ISP2_IF=eth3
DMZ_IF=eth1
LAN_IF=eth2

Add it also to ''/etc/shorewall/interfaces'':

INET $ISP1_IF detect dhcp
INET $ISP2_IF detect dhcp
DMZ $DMZ_IF detect dhcp
LAN $LAN_IF detect dhcp

The dhcp option is not really needed unless dhcp is actually used. We recommend leaving the dhcp option on in case the ISP changes to DHCP in the future.

== Shorewall Masquerade ==
We must masquerade the internet addresses so that every ISP gets a correct source address. This is done in ''/etc/shorewall/masq''.

$ISP1 0.0.0.0/0
$ISP2 0.0.0.0/0

== Shorewall Policy ==
We disallow routing between the 2 ISP's by adding a drop policy in ''/etc/shorewall/policy''.

...
INET INET DROP
...

== Shorewall Providers ==
The ''/etc/shorewall/providers'' defines the 2 ISP's and allows firewall packet marking (the firewall can mark certain packages which can later be used for the routing decision). Shorewall will also let the firewall mark the incomming connections in order to keep track of what gateway the response should go via. This is needed in case you do DNAT of incomming traffic on the secondary ISP. Without the connection tracking the outgoing traffic would always go via the primary ISP.

We do '''not''' want shorewall to mess with the routing tables (which we needed it to do with older versions of pingu). We can tell shorewall not to flush the routing tables that pingu will manage by setting DUPLICATE column to '-', GATEWAY column to 'none' and let COPY column be empty.

In this example eth0 is attatched to ISP1, eth3 is attatched to ISP2.

#NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY
ISP1 1 1 - eth0 none track,optional,loose
ISP2 2 2 - eth3 none track,optional,loose

After shorewall has restarted it should be possible to see the route rules for the firewall marks using the command ''ip rule'':

...
10001: from all fwmark 0x1 lookup ISP1
10002: from all fwmark 0x2 lookup ISP2
...

== Shorewall Route Rules ==
We also use ''/etc/shorewall/route_rules'' to set up some static route rules. This is to force local traffic to ignore the alternate route tables for the ISP's.

#SOURCE DEST PROVIDER PRIORITY

# prevent local traffic go via providers' route tables
- 192.168.0.1/24 main 1000
- 192.168.1.1/24 main 1000

== Make Shorewall not touch default gateway ==
Shorewall will store the default gateway(s) on startup and try restore on stopping. Unfortunally it does it badly, restulting in a default gateway getting lost on stop. Additionally, the default gateway might have changed during runtime, due to dhcp lease renewal. We need configure shorewall to not do anything with the default gateways at all and let pingu handle this.

Add the following to ''/etc/shorewall/shorewall.conf'':

RESTORE_DEFAULT_ROUTE=No

== Policy Routing with Pingu ==
We need to make sure that where ISP1's IP address is used as source address, the ISP1's default gateway must be used. Otherwise we might send packets with ISP1 IP source address via ISP2. The response would come through the ISP2's interfaces and firewall would block.

Since IP address and default gateway might change (DHCP lease renewal) we need a daemon that monitors the source address of the gateways via the interfaces. This is what pingu does.

apk add pingu

We configure pingu to monitor our eth0 and eth3 interfaces in ''/etc/pingu/pingu.conf'':

interface eth0 {
# route-table must correspond with NUMBER column in /etc/shorewall/providers
route-table 1
# the rule-priority must be a higher number than the priority in /etc/shorewall/route_rules
rule-priority 20000
}

interface eth3 {
# route-table must correspond with NUMBER column in /etc/shorewall/providers
route-table 2
rule-priority 20000
}

When pingu is started you should be able to see the routing rules with

ip rule

...
20000: from 1.2.3.4 lookup ISP1
20000: from 192.168.254.4 lookup ISP2
...

and see the ISP route tables with

ip route show table 1

default via 1.2.3.1 dev eth0 metric 200
1.2.3.0/24 dev eth0 proto kernel scope link metric 200

== Internet Failover with Pingu ==
To do failover we need a way to detect if an ISP is still or not. We configure some hosts that pingu will ping for us at regular intervals. When a host stops to respond we consider that host offline. We can bind a ping host to an interface in pingu and when enough hosts stops to respond, pingu will remove that gateway from main route table and a gateway will lower metric will take over. We normally want ping more than one host since a single host might down without it meaning that the rest of internet is down.

We also want pick hosts that is unlikely to change. DNS servers are good in this aspect. They seldomly change since it would require all clients to do reconfiguration. In this example we will use google's DNS and opendns hosts.

Add the hosts to ''/etc/pingu/pingu.conf'' and bind to primary ISP. We also set the ping timeout to 2.5 seconds.

...

# Ping responses that takes more than 2.5 seconds are considered lost.
timeout 2.5

# ping google dns via ISP1
host 8.8.8.8 {
interval 60
bind-interface eth0
}

# ping opendns via ISP1
host 208.67.222.222 {
interval 60
bind-interface eth0
}

Now, if both hosts stops to respond to pings, ISP-1 will be considered down and all gateways via eth0 will be removed from main route table. Note that the gateway will not be removed from the route table '1'. This is so we can continue try ping via eth0 so we can detect that the ISP is back online. When ISP starts working again, the gateways will be added back to main route table again.

== Load Balancing ==
It is possible to give the impression of load balancing by using multipath routes. It will not let a http file download use both ISPs for double download speed but it will let some users surf via ISP1 and others via ISP2, resulting that bandwith usage is spread over both ISPs.

Note that balancing will not be perfect, as it is route based, and routes are cached. This means that routes to often-used sites will always be over the same provider. It also means that the source IP might change so websites that requires login (webmail, banks etc) will break if they check if the clients ip is consistent. Banking sites often do this. It is recommended to '''avoid this feature''' if possible.

To enable load balance, simply add the 'load-balance' keyword to all the interfaces that should be balanced in ''/etc/pingu/pingu.conf'':

interface eth0 {
...
load-balance
}

interface eth3 {
...
load-balance
}

...