Page tree

This section consists of setup examples with firewall-based load balancing methods.

Example 1

Failover With Firewall Marking


This example demonstrates how to set up failover with a firewall mangle, filter and NAT rules.

Detailed Section Overview

IP address

In this example, our provider assigned two upstream links, one connected to ether1 and other to ether2. Our local network has two subnets 192.168.1.0/24 and 192.168.2.0/24

/ip address
add address=10.1.101.18/30 interface=ether1 
add address=10.1.200.18/30 interface=ether2
add address=192.168.1.1/24 interface=ether3
add address=192.168.2.1/24 interface=ether4

Mangle, NAT, and Filter rules

Connections going through the ether1 interface is marked as "first" and packets going through the ether2 is marked as "other":

/ip firewall mangle
add action=mark-connection chain=forward connection-mark=no-mark new-connection-mark=first out-interface=ether1 passthrough=yes
add action=mark-connection chain=forward connection-mark=no-mark new-connection-mark=other out-interface=ether2 passthrough=yes

Instead of masquerade, we will use src-nat for our local networks, because we do not want to purge connections which is one of masquerades main features when a primary link fails. We will restrict them with firewall rules (later in this example):

/ip firewall nat
add action=src-nat chain=srcnat out-interface=ether1 to-address=10.1.101.18
add action=src-nat chain=srcnat out-interface=ether2 to-address=10.1.200.18

When the primary link will fail, we will reject all the established connections, so new connections will pass through the secondary link. The same behavior will happen when a primary link will come back again and here we will prevent local IP leakage to a public network, which is one of masquerades disadvantages:

/ip firewall filter
add action=reject chain=forward connection-mark=other out-interface=ether1 reject-with=icmp-network-unreachable
add action=reject chain=forward connection-mark=first out-interface=ether2 reject-with=icmp-network-unreachable

Routes

We will add two default routes. With distance parameter we set route preference:

/ip route add gateway=10.1.101.17 distance=1 check-gateway=ping
/ip route add gateway=10.1.200.17 distance=2

Example 2


This example demonstrates how to set up load balancing if the provider is giving IP addresses from the same subnet for all links.

Detailed Section Overview

IP address

In this example, our provider assigned two upstream links, one connected to ether1 and other to ether2. Both links have IP addresses from the same subnet. Our ether3 network has two subnets 192.168.1.0/24 and 192.168.2.0/2:

/ip address
add address=10.1.101.18/24 interface=ether1 
add address=10.1.200.10/24 interface=ether2
add address=192.168.1.1/24 interface=ether3
add address=192.168.2.1/24 interface=ether4

After the IP address is set up, the connected route will be installed as ECMP rout:

[admin@MikroTik] > /ip route print detail
 0 ADC  dst-address=10.1.101.0/24 pref-src=10.1.101.18 gateway=ether1,ether2 
        gateway-status=ether1 reachable,ether2 reachable distance=0 scope=10

Mangle and NAT

In our example, very simple policy routing is used. Clients from 192.168.1.0/24 subnet are marked to use the "first" routing table and 192.168.2.0/24 to use the "other" subnet:

/ip firewall mangle
add action=mark-routing chain=prerouting src-address=192.168.1.0/24 new-routing-mark=first
add action=mark-routing chain=prerouting src-address=192.168.2.0/24 new-routing-mark=other

And masquerade our local network:

/ip firewall nat
add action=masquerade chain=srcnat out-interface=ether1
add action=masquerade chain=srcnat out-interface=ether2

The same can be achieved by setting up route rules instead of mangle.

Routes

We are adding two gateways, one to resolve in "first" routing table and another to "other" routing table:

/ip route
add gateway=10.1.101.1%ether1 routing-mark=first
add gateway=10.1.101.1%ether2 routing-mark=other

Interesting part of these routes is how we set the gateway - gateway=10.1.101.1%ether1 means that gateway 10.1.101.1 will be explicitly reachable over ether:

[admin@MikroTik] > /ip route print detail 
Flags: X - disabled, A - active, D - dynamic, 
C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, 
B - blackhole, U - unreachable, P - prohibit 
 0 A S  dst-address=0.0.0.0/0 gateway=10.1.101.1%ether2 
        gateway-status=10.1.101.1 reachable ether2 distance=1 scope=30 
        target-scope=10 routing-mark=other 

 1 A S  dst-address=0.0.0.0/0 gateway=10.1.101.1%ether1 
        gateway-status=10.1.101.1 reachable ether1 distance=1 scope=30 
        target-scope=10 routing-mark=first 

Finally, we have one additional entry specifying that traffic from the router itself (the traffic without any routing marks) will be resolved in the main routing table:

/ip route add gateway=10.1.101.1

Example 3 (PCC)

Load Balancing With Per Connection Classifier

PCC matcher will allow you to divide traffic into equal streams with the ability to keep packets with a specific set of options in one particular stream (you can specify this set of options from src-address, src-port, dst-address, dst-port, etc.)

Detailed Section Overview

IP address

The router has two upstream ether1 and ether2 interfaces with the addresses of 10.111.0.2/24 and 10.112.0.2/24. The ether3 interface has an IP address of 192.168.1.1/24.

/ip address
add address=192.168.1.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=ether3
add address=10.111.0.2/30 network=10.111.0.0 broadcast=10.111.0.255 interface=ether1
add address=10.112.0.2/30 network=10.112.0.0 broadcast=10.112.0.255 interface=ether2

Mangle

With policy routing, it is possible to force all traffic to the specific gateway, even if traffic is destined to the host (other than gateway) from the connected networks. This way routing loop will be generated and communications with those hosts will be impossible. To avoid this situation we need to allow usage of the default routing table for traffic to connected networks:

/ip firewall mangle
add chain=prerouting dst-address=10.111.0.0/30  action=accept in-interface=ether3
add chain=prerouting dst-address=10.112.0.0/30  action=accept in-interface=ether3

First, it is necessary to manage connections initiated from outside - replies must leave via the same interface (from the same Public IP) request came. We will mark all new incoming connections, to remember what was the interface.

/ip firewall mangle
add chain=prerouting in-interface=ether1 connection-mark=no-mark action=mark-connection new-connection-mark=ISP1_conn
add chain=prerouting in-interface=ether2 connection-mark=no-mark action=mark-connection new-connection-mark=ISP2_conn

Action mark-routing can be used only in mangle chain output and prerouting, but mangle chain prerouting is capturing all traffic that is going to the router itself. To avoid this we will use dst-address-type=!ether3. And with the help of the new PCC, we will divide traffic into two groups based on source and destination addresses:

/ip firewall mangle 
add chain=prerouting in-interface=ether3 connection-mark=no-mark dst-address-type=!ether3 per-connection-classifier=both-addresses:2/0 action=mark-connection new-connection-mark=ISP1_conn 
add chain=prerouting in-interface=ether3 connection-mark=no-mark dst-address-type=!ether3 per-connection-classifier=both-addresses:2/1 action=mark-connection new-connection-mark=ISP2_conn

Then we need to mark all packets from those connections with a proper mark. As policy routing is required only for traffic going to the Internet, do not forget to specify the in-interface option:

/ip firewall mangle
add chain=prerouting connection-mark=ISP1_conn in-interface=ether3 action=mark-routing new-routing-mark=to_ISP1
add chain=prerouting connection-mark=ISP2_conn in-interface=ether3 action=mark-routing new-routing-mark=to_ISP2
add chain=output connection-mark=ISP1_conn action=mark-routing new-routing-mark=to_ISP1     
add chain=output connection-mark=ISP2_conn action=mark-routing new-routing-mark=to_ISP2

Routes

Create a route for each routing-mark

/ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 routing-mark=to_ISP1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 routing-mark=to_ISP2 check-gateway=ping

To enable failover, it is necessary to have routes that will jump in as soon as others will become inactive on gateway failure. (and that will happen only if the check-gateway option is active)

/ip route
add dst-address=0.0.0.0/0 gateway=10.111.0.1 distance=1 check-gateway=ping
add dst-address=0.0.0.0/0 gateway=10.112.0.1 distance=2 check-gateway=ping

NAT

As routing decision is already made we just need rules that will fix src-addresses for all outgoing packets. If this packet will leave via wlan1 it will be NATed to 10.112.0.2, if via wlan2 then NATed to 10.111.0.2:

/ip firewall nat 
add chain=srcnat out-interface=ether1 action=masquerade
add chain=srcnat out-interface=ether2 action=masquerade

Example 4

Load Balancing With NTH

We will use the same setup from Example 3, but instead of Per Connection Classifier we will use NTH.  It has only two parameters 'every' and 'packet'.  Rule nth=2,1 rule will match every first packet of 2, hence, 50% of all the traffic that is matched by the rule. 

Detailed Section Overview

IP address

The router has two upstream ether1 and ether2 interfaces with the addresses of 10.111.0.2/24 and 10.112.0.2/24. The ether3 interface has an IP address of 192.168.1.1/24.

/ip address
add address=192.168.1.1/24 network=192.168.0.0 broadcast=192.168.0.255 interface=ether3
add address=10.111.0.2/30 network=10.111.0.0 broadcast=10.111.0.255 interface=ether1
add address=10.112.0.2/30 network=10.112.0.0 broadcast=10.112.0.255 interface=ether2

Mangle

All traffic from customers having their IP address previously placed in the address list "odd" is instantly marked with connection and routing marks "odd". Afterward, the traffic is excluded from processing against successive mangle rules in the prerouting chain:

/ip firewall mangle 
add chain=prerouting src-address-list=odd in-interface=ether3 action=mark-connection new-connection-mark=odd passthrough=yes 
add chain=prerouting src-address-list=odd in-interface=ether3 action=mark-routing new-routing-mark=odd 

Similar configuration, only for customers having their IP address previously placed in the address list "even":

/ip firewall mangle 
add chain=prerouting src-address-list=even in-interface=ether3 action=mark-connection new-connection-mark=even passthrough=yes 
add chain=prerouting src-address-list=even in-interface=ether3 action=mark-routing new-routing-mark=even 

First we take every second packet that establishes a new session (note connection-state=new), and mark it with connection mark "odd". Consequently all successive packets belonging to the same session will carry the connection mark "odd". Note that we are passing these packets to the second and third rules (passthrough=yes). The second rule adds IP address of the client to the address-list to enable all successive sessions to go through the same gateway. The third rule places the routing mark "odd" on all packets that belong to the "odd" connection and stops processing all other mangle rules for these packets in the prerouting chain:

/ip firewall mangle 
add chain=prerouting in-interface=ether3 connection-state=new nth=2,1 action=mark-connection new-connection-mark=odd passthrough=yes
add chain=prerouting in-interface=ether3 action=add-src-to-address-list address-list=odd address-list-timeout=1d connection-mark=odd passthrough=yes 
add chain=prerouting in-interface=ether3 connection-mark=odd action=mark-routing new-routing-mark=odd passthrough=no

The next rules do the same for the remaining half of the traffic as the first three rules for the first half of the traffic. The code above effectively means that each new connection initiated through the router from the ether3 network will be marked as either "odd" or "even" with both routing and connection marks.

/ip firewall mangle 
add chain=prerouting in-interface=ether3 connection-state=new nth=2,2 action=mark-connection new-connection-mark=even passthrough=yes
add chain=prerouting in-interface=ether3 action=add-src-to-address-list address-list=even address-list-timeout=1d connection-mark=even passthrough=yes 
add chain=prerouting in-interface=ether3 connection-mark=even action=mark-routing new-routing-mark=even passthrough=no

The above works fine. There are however some situations where you might find that the same IP address is listed under both the ODD and EVEN scr-address-lists. This behavior causes issues with apps that require persistent connections. A simple remedy for this situation is to add the following statement to your mangle rules:

/ ip firewall mangle
add chain=prerouting in-interface=ether3 connection-state=new nth=2,2 src-address-list=!odd action=mark-connection new-connection-mark=even passthrough=yes

This will ensure that the new connection will not already be part of the ODD src-address-list. You will have to do the same for the ODD mangle rule thus excluding IP's already part of the EVEN scr-address-list.

NAT

Fix the source address according to the outgoing interface.

/ ip firewall nat 
add chain=srcnat out-interface=ether1 action=masquerade
add chain=srcnat out-interface=ether2 action=masquerade

Routing

For all traffic marked "odd" (consequently having 10.111.0.2 translated source address) we use 10.111.0.1 gateway. In the same manner all traffic marked "even" is routed through the 10.112.0.1 gateway.

/ ip route 
add dst-address=0.0.0.0/0 gateway=10.111.0.1 scope=255 target-scope=10 routing-mark=odd 
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10 routing-mark=even

Finally, we have one additional entry specifying that traffic from the router itself (the traffic without any routing marks) should go to 10.112.0.1 gateway.

/ip route
add dst-address=0.0.0.0/0 gateway=10.112.0.1 scope=255 target-scope=10
  • No labels