Dynamic Multipoint VPN (DMVPN): Difference between revisions
(Enable IP forwarding) |
(Mostly formatting) |
||
Line 8: | Line 8: | ||
= Terminology = | = Terminology = | ||
;NBMA: ''Non-Broadcast Multi-Access'' network as described in [http://tools.ietf.org/html/rfc2332 RFC 2332] | |||
;Hub: the ''Next Hop Server'' (NHS) performing the Next Hop Resolution Protocol service within the NBMA cloud. | |||
;Spoke: the ''Next Hop Resolution Protocol Client'' (NHC) which initiates NHRP requests of various types in order to obtain access to the NHRP service. | |||
{{Tip|At the time of this writing the recommended Alpine version for building a DMVPN should be at minimum 2.4.11. Don't use 2.5.x, or 2.6.0 since the kernel has in-tunnel IP fragmentation issues. Alpine 2.6.1 or later should be okay instead.}} | {{Tip|At the time of this writing the recommended Alpine version for building a DMVPN should be at minimum 2.4.11. Don't use 2.5.x, or 2.6.0 since the kernel has in-tunnel IP fragmentation issues. Alpine 2.6.1 or later should be okay instead.}} | ||
Line 21: | Line 21: | ||
If you are looking for hundreds of megabits of throughput for your VPN with a limited budget, you should consider using [http://www.via.com.tw/en/initiatives/padlock/hardware.jsp VIA Padlock] engine present in VIA processor C7, Eden, Nano and Quad. If you need gigabits throughput you should go instead for an Intel Xeon processor with [http://software.intel.com/en-us/articles/intel-advanced-encryption-standard-instructions-aes-ni AES-NI] and [http://software.intel.com/en-us/articles/intel-sha-extensions SHA Extensions] | If you are looking for hundreds of megabits of throughput for your VPN with a limited budget, you should consider using [http://www.via.com.tw/en/initiatives/padlock/hardware.jsp VIA Padlock] engine present in VIA processor C7, Eden, Nano and Quad. If you need gigabits throughput you should go instead for an Intel Xeon processor with [http://software.intel.com/en-us/articles/intel-advanced-encryption-standard-instructions-aes-ni AES-NI] and [http://software.intel.com/en-us/articles/intel-sha-extensions SHA Extensions] | ||
For supporting VIA Padlock engine | For supporting VIA Padlock engine enable its modules: | ||
{{Cmd|echo -e "padlock_aes\npadlock-sha" >> /etc/modules}} | {{Cmd|echo -e "padlock_aes\npadlock-sha" >> /etc/modules}} | ||
Line 34: | Line 35: | ||
}} | }} | ||
Set appropriate permission for your certificate files: | |||
{{Cmd|chmod 600 *.pem *.pfx}} | {{Cmd|chmod 600 *.pem *.pfx}} | ||
Line 54: | Line 55: | ||
bond0.257 = ISP2<br> | bond0.257 = ISP2<br> | ||
Boot Alpine in [[Installation#Basics|diskless mode]] and run <code>setup-alpine</code> | Boot Alpine in [[Installation#Basics|diskless mode]] and run <code>[[setup-alpine]]</code> | ||
{|class="wikitable" | {|class="wikitable" | ||
Line 132: | Line 133: | ||
Update the bonding configuration: | Update the bonding configuration: | ||
{{cmd|echo bonding mode{{=}}balance-tlb miimon{{=}}100 updelay{{=}}500 >> /etc/modules}} | |||
== Physically install == | == Physically install == | ||
At this point, you're ready to connect the VPN Spoke Node to the network if you haven't already done so. Please set up an 802.1q capable switch with the VLANs listed in AlpineSetup section. Once done, tag all of the VLANs on one port. Connect that port to eth0. Then, connect your first ISP's CPE to a switchport with VLAN 256 untagged. | At this point, you're ready to connect the VPN Spoke Node to the network if you haven't already done so. Please set up an 802.1q capable switch with the VLANs listed in AlpineSetup section. Once done, tag all of the VLANs on one port. Connect that port to <code>eth0</code>. Then, connect your first ISP's CPE to a switchport with VLAN 256 untagged. | ||
== SSH == | == SSH == | ||
Remove password authentication and DNS reverse lookup: | Remove password authentication and DNS reverse lookup: | ||
{{Cmd|sed -i "s/.PasswordAuthentication yes/PasswordAuthentication no/" /etc/ssh/sshd_config | {{Cmd|sed -i "s/.PasswordAuthentication yes/PasswordAuthentication no/" /etc/ssh/sshd_config | ||
sed -i "s/.UseDNS yes/UseDNS no/" /etc/ssh/sshd_config}} | sed -i "s/.UseDNS yes/UseDNS no/" /etc/ssh/sshd_config}} | ||
== Recursive DNS == | == Recursive DNS == | ||
Install package(s): | |||
{{Cmd|apk add -U unbound}} | {{Cmd|apk add -U unbound}} | ||
With your favorite editor open <code>/etc/unbound/unbound.conf</code> and add the following configuration. If you have a domain that you want unbound to resolve but is internal to your network only, the stub-zone stanza is present: | With your favorite editor open <code>/etc/unbound/unbound.conf</code> and add the following configuration. If you have a domain that you want unbound to resolve but is internal to your network only, the stub-zone stanza is present: | ||
{{cat|/etc/unbound/unbound.conf| | |||
server: | server: | ||
verbosity: 1 | verbosity: 1 | ||
Line 187: | Line 192: | ||
remote-control: | remote-control: | ||
control-enable: no | control-enable: no | ||
}} | |||
Start unbound: | Start unbound and start using unbound on this host: | ||
{{Cmd|/etc/init.d/unbound start | {{Cmd|/etc/init.d/unbound start | ||
Line 198: | Line 203: | ||
If you have a DNS zone that is only resolvable internally to your network, you will need a 2nd IP address on your LAN interface, and use NSD to host the zone. | If you have a DNS zone that is only resolvable internally to your network, you will need a 2nd IP address on your LAN interface, and use NSD to host the zone. | ||
First, add the following to the end of the bond0.8 stanza | First, add the following to the end of the <code>bond0.8</code> stanza: | ||
{{cat|/etc/network/interfaces| | |||
auto bond0.8 | auto bond0.8 | ||
... | ... | ||
... | ... | ||
up ip addr add 10.1.0.2/24 dev bond0.8 | up ip addr add 10.1.0.2/24 dev bond0.8 | ||
}} | |||
Install package(s): | |||
{{Cmd|apk add nsd}} | {{Cmd|apk add nsd}} | ||
Create /etc/nsd/nsd.conf: | Create <code>/etc/nsd/nsd.conf</code>: | ||
{{cat|/etc/nsd/nsd.conf| | |||
server: | server: | ||
ip-address: 10.1.0.2 | ip-address: 10.1.0.2 | ||
Line 223: | Line 230: | ||
name: location1.example.net | name: location1.example.net | ||
zonefile: location1.example.net.zone | zonefile: location1.example.net.zone | ||
</ | }} | ||
Create zonefile in <code>/etc/nsd/location1.example.net.zone</code>: | |||
{{cat|/etc/nsd/location1.example.net.zone| | |||
;## location1.example.net authoritative zone | ;## location1.example.net authoritative zone | ||
Line 244: | Line 252: | ||
ns IN A 10.1.0.2 | ns IN A 10.1.0.2 | ||
mail IN A 10.1.0.4 | mail IN A 10.1.0.4 | ||
}} | |||
Check configuration then start: | Check configuration then start: | ||
{{Cmd|nsd-checkconf /etc/nsd/nsd.conf | {{Cmd|nsd-checkconf /etc/nsd/nsd.conf | ||
nsdc rebuild | nsdc rebuild | ||
Line 255: | Line 264: | ||
With your favorite editor open <code>/etc/network/interfaces</code> and add the following: | With your favorite editor open <code>/etc/network/interfaces</code> and add the following: | ||
{{cat|/etc/network/interfaces| | |||
auto gre1 | |||
iface gre1 inet static | |||
pre-up ip tunnel add $IFACE mode gre ttl 64 tos inherit key 12.34.56.78 || true | |||
address 172.16.1.1 | |||
netmask 255.255.0.0 | |||
post-down ip tunnel del $IFACE || true | |||
}} | |||
Bring up the new <code>gre1</code> interface: | |||
{{Cmd|ifup gre1}} | {{Cmd|ifup gre1}} | ||
== IPSEC == | == IPSEC == | ||
Install package(s): | |||
{{Cmd|apk add ipsec-tools}} | {{Cmd|apk add ipsec-tools}} | ||
With your favorite editor create <code>/etc/ipsec.conf</code> and set the content to the following: | With your favorite editor create <code>/etc/ipsec.conf</code> and set the content to the following: | ||
{{cat|/etc/ipsec.conf| | |||
spdflush; | |||
spdadd 0.0.0.0/0 0.0.0.0/0 gre -P out ipsec esp/transport//require; | |||
spdadd 0.0.0.0/0 0.0.0.0/0 gre -P in ipsec esp/transport//require; | |||
}} | |||
Create missing directory: | |||
{{Cmd|mkdir /etc/racoon/}} | {{Cmd|mkdir /etc/racoon/}} | ||
Extract your pfx into /etc/racoon, using the filenames '''ca.pem''', '''cert.pem''', and '''key.pem'''. | Extract your pfx into <code>/etc/racoon</code>, using the filenames '''<code>ca.pem</code>''', '''<code>cert.pem</code>''', and '''<code>key.pem</code>'''. | ||
With your favorite editor create <code>/etc/racoon/racoon.conf</code> and set the content to the following: | With your favorite editor create <code>/etc/racoon/racoon.conf</code> and set the content to the following: | ||
{{cat|/etc/racoon/racoon.conf| | |||
path certificate "/etc/racoon/"; | path certificate "/etc/racoon/"; | ||
remote anonymous { | remote anonymous { | ||
Line 313: | Line 330: | ||
compression_algorithm deflate; | compression_algorithm deflate; | ||
} | } | ||
}} | |||
Edit /etc/conf.d/racoon and unset RACOON_PSK_FILE: | Edit <code>/etc/conf.d/racoon</code> and unset <code>RACOON_PSK_FILE</code>: | ||
{{cat|/etc/conf.d/racoon| | |||
... | ... | ||
RACOON_PSK_FILE= | RACOON_PSK_FILE{{=}} | ||
... | ... | ||
}} | |||
Start service(s): | |||
{{Cmd|/etc/init.d/racoon start | {{Cmd|/etc/init.d/racoon start | ||
Line 329: | Line 346: | ||
== Next Hop Resolution Protocol (NHRP) == | == Next Hop Resolution Protocol (NHRP) == | ||
Install package(s): | |||
{{Cmd|apk add opennhrp}} | {{Cmd|apk add opennhrp}} | ||
With your favorite editor open <code>/etc/opennhrp/opennhrp.conf</code> and change the content to the following: | With your favorite editor open <code>/etc/opennhrp/opennhrp.conf</code> and change the content to the following: | ||
{{cat|/etc/opennhrp/opennhrp.conf| | |||
interface gre1 | interface gre1 | ||
dynamic-map 172.16.0.0/16 hub.example.com | dynamic-map 172.16.0.0/16 hub.example.com | ||
Line 345: | Line 364: | ||
interface bond0.64 | interface bond0.64 | ||
shortcut-destination | shortcut-destination | ||
}} | |||
You must have a DNS A record ''hub.example.com'' for each hub node IP address. | You must have a DNS A record ''<code>hub.example.com</code>'' for each hub node IP address. | ||
With your favorite editor open <code>/etc/opennhrp/opennhrp-script</code> and change the content to the following: | With your favorite editor open <code>/etc/opennhrp/opennhrp-script</code> and change the content to the following: | ||
{{cat|/etc/opennhrp/opennhrp-script| | |||
#!/bin/sh | #!/bin/sh | ||
Line 422: | Line 441: | ||
exit 0 | exit 0 | ||
}} | |||
Make it executable and start service(s): | |||
{{Cmd|chmod +x /etc/opennhrp/opennhrp-script | {{Cmd|chmod +x /etc/opennhrp/opennhrp-script | ||
Line 432: | Line 450: | ||
== BGP == | == BGP == | ||
Install package(s): | |||
{{Cmd|apk add quagga | {{Cmd|apk add quagga | ||
touch /etc/quagga/zebra.conf}} | touch /etc/quagga/zebra.conf}} | ||
With your favorite editor open <code>/etc/quagga/bgpd.conf</code> and change the content to the following (replace <code>strongpassword</code> with a password of your choice and %HUB_GRE_IP% with the '''Hub''' node GRE IP address): | With your favorite editor open <code>/etc/quagga/bgpd.conf</code> and change the content to the following (replace <code>strongpassword</code> with a password of your choice and <code>%HUB_GRE_IP%</code> with the '''Hub''' node GRE IP address): | ||
* Add the line <code>neighbor %HUB_GRE_IP% remote-as 65000</code> for each '''Hub''' host you have in your NBMA cloud. | |||
{{cat|/etc/quagga/bgpd.conf| | |||
password strongpassword | password strongpassword | ||
enable password strongpassword | enable password strongpassword | ||
Line 455: | Line 476: | ||
neighbor %HUB_GRE_IP% remote-as 65000 | neighbor %HUB_GRE_IP% remote-as 65000 | ||
... | ... | ||
}} | |||
Start service(s): | |||
{{Cmd|/etc/init.d/bgpd start | {{Cmd|/etc/init.d/bgpd start | ||
Line 465: | Line 484: | ||
== OpenVPN == | == OpenVPN == | ||
Install package(s): | |||
{{Cmd|echo tun >> /etc/modules | {{Cmd|echo tun >> /etc/modules | ||
modprobe tun | modprobe tun | ||
Line 470: | Line 491: | ||
openssl dhparam -out /etc/openvpn/dh1024.pem 1024}} | openssl dhparam -out /etc/openvpn/dh1024.pem 1024}} | ||
Configure openvpn: | |||
{{cat|/etc/openvpn/openvpn.conf| | |||
dev tun | dev tun | ||
proto udp | proto udp | ||
Line 499: | Line 521: | ||
mute 20 | mute 20 | ||
verb 3 | verb 3 | ||
}} | |||
Start service(s): | |||
{{Cmd|/etc/init.d/openvpn start | {{Cmd|/etc/init.d/openvpn start | ||
Line 505: | Line 529: | ||
== Firewall == | == Firewall == | ||
Install package(s): | |||
{{Cmd|apk add awall}} | {{Cmd|apk add awall}} | ||
Enable IP forwarding | Enable IP forwarding: | ||
{{Cmd|sysctl -w net.ipv4.ip_forward{{=}}1 | {{Cmd|sysctl -w net.ipv4.ip_forward{{=}}1 | ||
sed -i 's/.*net\.ipv4\.ip_forward.*$/net.ipv4.ip_forward {{=}} 1/g' /etc/sysctl.conf}} | sed -i 's/.*net\.ipv4\.ip_forward.*$/net.ipv4.ip_forward {{=}} 1/g' /etc/sysctl.conf}} | ||
With your favorite editor, edit the following files and set their contents as follows: | With your favorite editor, edit the following files and set their contents as follows: | ||
{{cat|/etc/awall/optional/params.json| | |||
{ | { | ||
"description": "params", | "description": "params", | ||
"variable": { | "variable": { | ||
"B_IF" = "bond0.8", | "B_IF" {{=}} "bond0.8", | ||
"C_IF" = "bond0.64", | "C_IF" {{=}} "bond0.64", | ||
"ISP1_IF" = "bond0.256", | "ISP1_IF" {{=}} "bond0.256", | ||
"ISP2_IF" = "bond0.257" | "ISP2_IF" {{=}} "bond0.257" | ||
} | } | ||
} | } | ||
}} | |||
{{cat|/etc/awall/optional/internet-host.json| | |||
{ | { | ||
"description": "Internet host", | "description": "Internet host", | ||
Line 571: | Line 596: | ||
] | ] | ||
} | } | ||
}} | |||
{{cat|/etc/awall/optional/openvpn.json| | |||
{ | { | ||
"description": "OpenVPN support", | "description": "OpenVPN support", | ||
Line 588: | Line 614: | ||
] | ] | ||
} | } | ||
}} | |||
{{cat|/etc/awall/optional/clampmss.json| | |||
{ | { | ||
"description": "Deal with ISPs afraid of ICMP", | "description": "Deal with ISPs afraid of ICMP", | ||
Line 599: | Line 626: | ||
"clamp-mss": [ { "out": "E" } ] | "clamp-mss": [ { "out": "E" } ] | ||
} | } | ||
}} | |||
{{cat|/etc/awall/optional/mark.json| | |||
{ | { | ||
"description": "Mark traffic based on ISP", | "description": "Mark traffic based on ISP", | ||
Line 613: | Line 641: | ||
] | ] | ||
} | } | ||
}} | |||
{{cat|/etc/awall/optional/dmvpn.json| | |||
{ | { | ||
"description": "DMVPN router", | "description": "DMVPN router", | ||
Line 654: | Line 682: | ||
] | ] | ||
} | } | ||
}} | |||
{{cat|/etc/awall/optional/vpnc.json| | |||
{ | { | ||
"description": "VPNc", | "description": "VPNc", | ||
Line 713: | Line 741: | ||
] | ] | ||
} | } | ||
}} | |||
Activate the firewall: | |||
{{Cmd|modprobe ip_tables | {{Cmd|modprobe ip_tables | ||
Line 724: | Line 754: | ||
== ISP Failover == | == ISP Failover == | ||
Install package(s): | |||
{{Cmd|apk add pingu | {{Cmd|apk add pingu | ||
Line 729: | Line 760: | ||
echo -e "2\tisp2">> /etc/iproute2/rt_tables}} | echo -e "2\tisp2">> /etc/iproute2/rt_tables}} | ||
Configure pingu to monitor our bond0.256 and bond0.257 interfaces in <code>/etc/pingu/pingu.conf</code>. Add the hosts to monitor for ISP failover to <code>/etc/pingu/pingu.conf</code> and bind to primary ISP. We also set the ping timeout to 4 seconds.: | Configure pingu to monitor our <code>bond0.256</code> and <code>bond0.257</code> interfaces in <code>/etc/pingu/pingu.conf</code>. Add the hosts to monitor for ISP failover to <code>/etc/pingu/pingu.conf</code> and bind to primary ISP. We also set the ping timeout to 4 seconds.: | ||
{{cat|/etc/pingu/pingu.conf| | |||
timeout 4 | timeout 4 | ||
required 2 | required 2 | ||
Line 752: | Line 784: | ||
rule-priority 20000 | rule-priority 20000 | ||
} | } | ||
}} | |||
</ | Make sure we can reach the public IP from our LAN by adding static route rules for our private net(s). Edit <code>/etc/pingu/route-rules</code>: | ||
{{cat|/etc/pingu/route-rules| | |||
to 10.0.0.0/8 table main prio 1000 | to 10.0.0.0/8 table main prio 1000 | ||
to 172.16.0.0/12 table main prio 1000 | to 172.16.0.0/12 table main prio 1000 | ||
}} | |||
Start service(s): | |||
{{Cmd|/etc/init.d/pingu start | {{Cmd|/etc/init.d/pingu start | ||
rc-update add pingu}} | rc-update add pingu}} | ||
Now, if both hosts stop responding to pings, ISP-1 will be considered down and all gateways via bond0.256 will be removed from main route table. Note that the gateway will not be removed from the route table '1'. This is so we can continue try ping via bond0.256 so we can detect that the ISP is back online. When ISP starts working again, the gateways will be added back to main route table again. | Now, if both hosts stop responding to pings, ISP-1 will be considered down and all gateways via bond0.256 will be removed from main route table. Note that the gateway will not be removed from the route table '1'. This is so we can continue try ping via <code>bond0.256</code> so we can detect that the ISP is back online. When ISP starts working again, the gateways will be added back to main route table again. | ||
== Commit Configuration == | == Commit Configuration == | ||
Commit configuration: | |||
{{Cmd|lbu ci}} | {{Cmd|lbu ci}} | ||
Line 774: | Line 809: | ||
== Routing Tables == | == Routing Tables == | ||
{{Todo|Would we need to change this command - or add some description on why it's documented?}} | |||
{{Cmd|echo -e "42\tnhrp_shortcut\n43\tnhrp_mtu\n44\tquagga\n}} | {{Cmd|echo -e "42\tnhrp_shortcut\n43\tnhrp_mtu\n44\tquagga\n}} | ||
== NHRP == | == NHRP == | ||
With your favorite editor open <code>/etc/opennhrp/opennhrp.conf</code> on Hub 2 and set the content as follows: | With your favorite editor open <code>/etc/opennhrp/opennhrp.conf</code> on Hub 2 and set the content as follows: | ||
{{cat|/etc/opennhrp/opennhrp.conf| | |||
interface gre1 | interface gre1 | ||
map %Hub1_GRE_IP%/%MaskBit% hub1.example.org | map %Hub1_GRE_IP%/%MaskBit% hub1.example.org | ||
Line 785: | Line 823: | ||
redirect | redirect | ||
non-caching | non-caching | ||
}} | |||
Do the same on Hub 1 adding the data relative to Hub 2. | Do the same on Hub 1 adding the data relative to Hub 2. | ||
With your favorite editor open <code>/etc/opennhrp/opennhrp-script</code> and set the content as follows: | With your favorite editor open <code>/etc/opennhrp/opennhrp-script</code> and set the content as follows: | ||
{{cat|/etc/opennhrp/opennhrp-script| | |||
#!/bin/sh | #!/bin/sh | ||
Line 859: | Line 899: | ||
exit 0 | exit 0 | ||
}} | |||
== BGP == | == BGP == | ||
With your favorite editor open <code>/etc/quagga/bgpd.conf</code> on Hub 2 and set the content as follows: | With your favorite editor open <code>/etc/quagga/bgpd.conf</code> on Hub 2 and set the content as follows: | ||
{{cat|/etc/quagga/bgpd.conf| | |||
password zebra | password zebra | ||
enable password zebra | enable password zebra | ||
Line 893: | Line 934: | ||
route-map CORE-IN permit 10 | route-map CORE-IN permit 10 | ||
set metric +100 | set metric +100 | ||
}} | |||
Add the lines <code>neighbor %Spoke1_GRE_IP%...</code> for each spoke node you have. Do the same on Hub 1, changing the relevant data for Hub 2. | Add the lines <code>neighbor %Spoke1_GRE_IP%...</code> for each spoke node you have. Do the same on Hub 1, changing the relevant data for Hub 2. | ||
Line 913: | Line 954: | ||
== Kernel and NHRP Routing Cache Issues == | == Kernel and NHRP Routing Cache Issues == | ||
{{Todo|...}} |
Revision as of 07:58, 5 September 2013
This material is work-in-progress ... Do not follow instructions here until this notice is removed. |
http://alpinelinux.org/about under Why the Name Alpine? states: [ref?]
The first open-source implementation of Cisco's DMVPN, called OpenNHRP, was written for Alpine Linux.
So the aim of this document is to be the reference Linux DMVPN setup, with all the networking services needed for the clients that will use the DMVPN (DNS, DHCP, firewall, etc.).
Terminology
- NBMA
- Non-Broadcast Multi-Access network as described in RFC 2332
- Hub
- the Next Hop Server (NHS) performing the Next Hop Resolution Protocol service within the NBMA cloud.
- Spoke
- the Next Hop Resolution Protocol Client (NHC) which initiates NHRP requests of various types in order to obtain access to the NHRP service.
Hardware
If you are looking for hundreds of megabits of throughput for your VPN with a limited budget, you should consider using VIA Padlock engine present in VIA processor C7, Eden, Nano and Quad. If you need gigabits throughput you should go instead for an Intel Xeon processor with AES-NI and SHA Extensions
For supporting VIA Padlock engine enable its modules:
echo -e "padlock_aes\npadlock-sha" >> /etc/modules
Extract Certificates
We will use certificates for DMVPN and for OpenVPN (RoadWarrior clients). If you are in need to generate your own certificates, please see Generating_SSL_certs_with_ACF. You should use a separate machine for this purpose. If you downloaded the certificates on a Windows machine, you may use WinSCP to copy them on the DMVPN box.
Here are the general purpose instruction for extracting certificates from pfx files:
openssl pkcs12 -in cert.pfx -cacerts -nokeys -out cacert.pem openssl pkcs12 -in cert.pfx -nocerts -nodes -out serverkey.pem openssl pkcs12 -in cert.pfx -nokeys -clcerts -out cert.pem
Set appropriate permission for your certificate files:
chmod 600 *.pem *.pfx
Spoke Node
A local spoke node network has support for multiple ISP connections, along with redundant layer 2 switches. At least one 802.1q capable switch is required, and a second is optional for redundancy purposes. The typical spoke node network looks like:
Alpine Setup
We will setup the network interfaces as follows:
bond0.3 = Management (not implemented below yet)
bond0.8 = LAN
bond0.64 = DMZ
bond0.80 = Voice (not implemented below yet)
bond0.96 = Internet Access Only (no access to the DMVPN network)(not implemented below yet)
bond0.256 = ISP1
bond0.257 = ISP2
Boot Alpine in diskless mode and run setup-alpine
You will be prompted something like this... | Suggestion on what you could enter... |
---|---|
Select keyboard layout [none]:
|
Type an appropriate layout for you |
Select variant:
|
Type an appropriate layout for you (if prompted) |
Enter system hostname (short form, e.g. 'foo') [localhost]:
|
Enter the hostname, e.g. vpnc |
Available interfaces are: eth0
|
Enter bond0.8 |
Available bond slaves are: eth0 eth1
|
eth0 eth1 |
IP address for bond0? (or 'dhcp', 'none', '?') [dhcp]:
|
Press Enter confirming 'none' |
IP address for bond0.8? (or 'dhcp', 'none', '?') [dhcp]:
|
Enter the IP address of your LAN interface, e.g. 10.1.0.1 |
Netmask? [255.255.255.0]:
|
Press Enter confirming '255.255.255.0' or type another appropriate subnet mask |
Gateway? (or 'none') [none]:
|
Press Enter confirming 'none' |
Do you want to do any manual network configuration? [no]
|
yes |
Make a copy of the bond0.8 configuration for bond0.64, bond0.256 and bond0.257 (optional) interfaces. Don't forget to add a gateway and a metric value for ISP interfaces when multiple gateways are set. Save and close the file (:wq) | |
DNS domain name? (e.g. 'bar.com') []:
|
Enter the domain name of your intranet, e.g., example.net |
DNS nameservers(s)? []:
|
8.8.8.8 8.8.4.4 (we will change them later) |
Changing password for root
|
Enter a secure password for the console |
Retype password:
|
Retype the above password |
Which timezone are you in? ('?' for list) [UTC]:
|
Press Enter confirming 'UTC' |
HTTP/FTP proxy URL? (e.g. 'http://proxy:8080', or 'none') [none]
|
Press Enter confirming 'none' |
Enter mirror number (1-9) or URL to add (or r/f/e/done) [f]:
|
Select a mirror close to you and press Enter |
Which SSH server? ('openssh', 'dropbear' or 'none') [openssh]:
|
Press Enter confirming 'openssh' |
Which NTP client to run? ('openntpd', 'chrony' or 'none') [chrony]:
|
Press Enter confirming 'chrony' |
Which disk(s) would you like to use? (or '?' for help or 'none') [none]:
|
Press Enter confirming 'none' or type 'none' if needed |
Enter where to store configs ('floppy', 'usb' or 'none') [usb]:
|
Press Enter confirming 'usb' |
Enter apk cache directory (or '?' or 'none') [/media/usb/cache]:
|
Press Enter confirming '/media/usb/cache' |
Bonding
Update the bonding configuration:
echo bonding mode=balance-tlb miimon=100 updelay=500 >> /etc/modules
Physically install
At this point, you're ready to connect the VPN Spoke Node to the network if you haven't already done so. Please set up an 802.1q capable switch with the VLANs listed in AlpineSetup section. Once done, tag all of the VLANs on one port. Connect that port to eth0
. Then, connect your first ISP's CPE to a switchport with VLAN 256 untagged.
SSH
Remove password authentication and DNS reverse lookup:
sed -i "s/.PasswordAuthentication yes/PasswordAuthentication no/" /etc/ssh/sshd_config sed -i "s/.UseDNS yes/UseDNS no/" /etc/ssh/sshd_config
Recursive DNS
Install package(s):
apk add -U unbound
With your favorite editor open /etc/unbound/unbound.conf
and add the following configuration. If you have a domain that you want unbound to resolve but is internal to your network only, the stub-zone stanza is present:
Contents of /etc/unbound/unbound.conf
Start unbound and start using unbound on this host:
/etc/init.d/unbound start rc-update add unbound echo nameserver 10.1.0.1 > /etc/resolv.conf
Local DNS Zone
If you have a DNS zone that is only resolvable internally to your network, you will need a 2nd IP address on your LAN interface, and use NSD to host the zone.
First, add the following to the end of the bond0.8
stanza:
Contents of /etc/network/interfaces
Install package(s):
apk add nsd
Create /etc/nsd/nsd.conf
:
Contents of /etc/nsd/nsd.conf
Create zonefile in /etc/nsd/location1.example.net.zone
:
Contents of /etc/nsd/location1.example.net.zone
Check configuration then start:
nsd-checkconf /etc/nsd/nsd.conf nsdc rebuild /etc/init.d/nsd start rc-update add nsd
GRE Tunnel
With your favorite editor open /etc/network/interfaces
and add the following:
Contents of /etc/network/interfaces
Bring up the new gre1
interface:
ifup gre1
IPSEC
Install package(s):
apk add ipsec-tools
With your favorite editor create /etc/ipsec.conf
and set the content to the following:
Contents of /etc/ipsec.conf
Create missing directory:
mkdir /etc/racoon/
Extract your pfx into /etc/racoon
, using the filenames ca.pem
, cert.pem
, and key.pem
.
With your favorite editor create /etc/racoon/racoon.conf
and set the content to the following:
Contents of /etc/racoon/racoon.conf
Edit /etc/conf.d/racoon
and unset RACOON_PSK_FILE
:
Contents of /etc/conf.d/racoon
Start service(s):
/etc/init.d/racoon start rc-update add racoon
Next Hop Resolution Protocol (NHRP)
Install package(s):
apk add opennhrp
With your favorite editor open /etc/opennhrp/opennhrp.conf
and change the content to the following:
Contents of /etc/opennhrp/opennhrp.conf
You must have a DNS A record hub.example.com
for each hub node IP address.
With your favorite editor open /etc/opennhrp/opennhrp-script
and change the content to the following:
Contents of /etc/opennhrp/opennhrp-script
Make it executable and start service(s):
chmod +x /etc/opennhrp/opennhrp-script /etc/init.d/opennhrp start rc-update add opennhrp
BGP
Install package(s):
apk add quagga touch /etc/quagga/zebra.conf
With your favorite editor open /etc/quagga/bgpd.conf
and change the content to the following (replace strongpassword
with a password of your choice and %HUB_GRE_IP%
with the Hub node GRE IP address):
- Add the line
neighbor %HUB_GRE_IP% remote-as 65000
for each Hub host you have in your NBMA cloud.
Contents of /etc/quagga/bgpd.conf
Start service(s):
/etc/init.d/bgpd start rc-update add bgpd
OpenVPN
Install package(s):
echo tun >> /etc/modules modprobe tun apk add openvpn openssl openssl dhparam -out /etc/openvpn/dh1024.pem 1024
Configure openvpn:
Contents of /etc/openvpn/openvpn.conf
Start service(s):
/etc/init.d/openvpn start rc-update add openvpn
Firewall
Install package(s):
apk add awall
Enable IP forwarding:
sysctl -w net.ipv4.ip_forward=1 sed -i 's/.*net\.ipv4\.ip_forward.*$/net.ipv4.ip_forward = 1/g' /etc/sysctl.conf
With your favorite editor, edit the following files and set their contents as follows:
Contents of /etc/awall/optional/params.json
Contents of /etc/awall/optional/internet-host.json
Contents of /etc/awall/optional/openvpn.json
Contents of /etc/awall/optional/clampmss.json
Contents of /etc/awall/optional/mark.json
Contents of /etc/awall/optional/dmvpn.json
Contents of /etc/awall/optional/vpnc.json
Activate the firewall:
modprobe ip_tables modprobe iptable_nat awall enable clampmss awall enable openvpn awall enable vpnc awall activate rc-update add iptables
ISP Failover
Install package(s):
apk add pingu echo -e "1\tisp1">> /etc/iproute2/rt_tables echo -e "2\tisp2">> /etc/iproute2/rt_tables
Configure pingu to monitor our bond0.256
and bond0.257
interfaces in /etc/pingu/pingu.conf
. Add the hosts to monitor for ISP failover to /etc/pingu/pingu.conf
and bind to primary ISP. We also set the ping timeout to 4 seconds.:
Contents of /etc/pingu/pingu.conf
Make sure we can reach the public IP from our LAN by adding static route rules for our private net(s). Edit /etc/pingu/route-rules
:
Contents of /etc/pingu/route-rules
Start service(s):
/etc/init.d/pingu start rc-update add pingu
Now, if both hosts stop responding to pings, ISP-1 will be considered down and all gateways via bond0.256 will be removed from main route table. Note that the gateway will not be removed from the route table '1'. This is so we can continue try ping via bond0.256
so we can detect that the ISP is back online. When ISP starts working again, the gateways will be added back to main route table again.
Commit Configuration
Commit configuration:
lbu ci
Hub Node
We will document only what changes from the Spoke node setup.
Routing Tables
echo -e "42\tnhrp_shortcut\n43\tnhrp_mtu\n44\tquagga\n
NHRP
With your favorite editor open /etc/opennhrp/opennhrp.conf
on Hub 2 and set the content as follows:
Contents of /etc/opennhrp/opennhrp.conf
Do the same on Hub 1 adding the data relative to Hub 2.
With your favorite editor open /etc/opennhrp/opennhrp-script
and set the content as follows:
Contents of /etc/opennhrp/opennhrp-script
BGP
With your favorite editor open /etc/quagga/bgpd.conf
on Hub 2 and set the content as follows:
Contents of /etc/quagga/bgpd.conf
Add the lines neighbor %Spoke1_GRE_IP%...
for each spoke node you have. Do the same on Hub 1, changing the relevant data for Hub 2.
Troubleshooting the DMVPN
Broken Path MTU Discovery (PMTUD)
ISPs afraid of ICMP (which is somehow legitimate) often just blindly add no ip unreachables
in their router interfaces, effectively creating a blackhole router that breaks PMTUD, since ICMP Type 3 Code 4 packets (Fragmentation Needed) are dropped. PMTUD is needed by ISAKMP that runs on UDP (TCP works because it uses CLAMPMSS).
For technical details see http://packetlife.net/blog/2008/oct/9/disabling-unreachables-breaks-pmtud/
PMTUD could also be broken due to badly configured DSL modem/routers or bugged firmware. Turning off the firewall on modem itself or any VPN passthrough functionality it may help.
You can easily detect which host is the blackhole router by pinging with DF bit set and with packets of standard MTU size, each hop given in your traceroute to destination:
ping -M do -s 1472 %IP%
iputils
packageIf you don't get a response back (either Echo-Response or Fragmentation-Needed) there's firewall dropping ICMP packets. If it answers to normal ping packets (DF bit cleared), most likely you have hit a blackhole router.