This section consists of setup examples with firewall-based load balancing methods.
This example demonstrates how to set up failover with a firewall mangle, filter and NAT rules.
In this example, our provider assigned two upstream links, one connected to ether1 and other to ether2. Our local network has two subnets 192.168.1.0/24 and 192.168.2.0/24
Connections going through the ether1 interface is marked as "first" and packets going through the ether2 is marked as "other":
When the primary link will fail, we will reject all the established connections, so new connections will pass through the secondary link. The same behavior will happen when a primary link will come back again and here we will prevent local IP leakage to a public network, which is one of:
Instead of masquerade, we will use src-nat for our local networks, because we do not want to purge connections which are one of the masquerade's main features when a primary link fails. We will restrict them with firewall rules (later in this example):
We will add two default routes. With distance parameter we set route preference:
This example demonstrates how to set up load balancing if the provider is giving IP addresses from the same subnet for all links.
In this example, our provider assigned two upstream links, one connected to ether1 and the other to ether2. Both links have IP addresses from the same subnet. Our local network has two subnets 192.168.1.0/24 and 192.168.2.0/2:
After the IP address is set up, the connected route will be installed as an ECMP route:
In our example, very simple policy routing is used. Let`s start with adding routing tables for each mangle mark:
Clients from the 192.168.1.0/24 subnet are marked to use the "first" routing table and 192.168.2.0/24 to use the "other" subnet:
And masquerade our local network:
The same can be achieved by setting up route rules instead of mangle.
We are adding two gateways, one to resolve in the "first" routing table and another to the "other" routing table:
PCC matcher will allow you to divide traffic into equal streams with the ability to keep packets with a specific set of options in one particular stream (you can specify this set of options from src-address, src-port, dst-address, dst-port, etc.) PCC takes selected fields from the IP header, and with the help of a hashing algorithm converts selected fields into 32-bit values. This value then is divided by a specified Denominator and the remainder then is compared to a specified Remainder, if equal the packet will be captured. You can choose from src-address, dst-address, src-port, dst-port from the header to use in this operation.
per-connection-classifier= PerConnectionClassifier ::= [!]ValuesToHash:Denominator/Remainder Remainder ::= 0..4294967295 (integer number) Denominator ::= 1..4294967295 (integer number) ValuesToHash ::= both-addresses|both-ports|dst-address-and-port| src-address|src-port|both-addresses-and-ports|dst-address|dst-port|src-address-and-port
The router has two upstream ether1 and ether2 interfaces with the addresses of 10.111.0.2/24 and 10.112.0.2/24. The ether3 interface has an IP address of 192.168.1.1/24.
With policy routing, it is possible to force all traffic to the specific gateway, even if traffic is destined to the host (other than gateway) from the connected networks. This way routing loop will be generated and communications with those hosts will be impossible. To avoid this situation we need to allow usage of the default routing table for traffic to connected networks:
First, it is necessary to manage connections initiated from outside - replies must leave via the same interface (from the same Public IP) request came. We will mark all new incoming connections, to remember what was the interface.
Before configuring mark-routing, we have to create a routing table for each of them:
Action mark-routing can be used only in mangle chain output and prerouting, but mangle chain prerouting is capturing all traffic that is going to the router itself. To avoid this we will use dst-address-type=!local. And with the help of the new PCC, we will divide traffic into two groups based on source and destination addresses:
Then we need to mark all packets from those connections with a proper mark. As policy routing is required only for traffic going to the Internet, do not forget to specify the in-interface option:
As routing decision is already made we just need rules that will fix src-addresses for all outgoing packets. If this packet will leave via wlan1 it will be NATed to 10.112.0.2, if via wlan2 then NATed to 10.111.0.2:
Create a route for each routing-mark
Every rule has its own counter. When the rule receives a packet counter for a current rule is increased by one. If the counter matches the value of 'every' packet will be matched and the counter will be set to zero.
To match 50% of all traffic only with one rule:
To split traffic into more than two parts we can use the following configuration. The first rule sees all packets and matches 1/3 of all, the second rule sees 2/3 of packets and matches half of them, the third rule sees and matches all packets that passed through the first two rules ( 1/3 of all packets ):
This example is a different version of the round-robin load balancing example. It adds persistent user sessions, i.e. a particular user would use the same source IP address for all outgoing connections. Consider the following network layout:
The router has two upstream (WAN) interfaces with the addresses 10.111.0.2/24 and 10.112.0.2/24. The LAN interface has the name "Local" and an IP address of 192.168.1.1/24.
All traffic from customers having their IP address previously placed in the address list "odd" is instantly marked with connection and routing marks "odd". Afterward, the traffic is excluded from processing against successive mangle rules in the prerouting chain:
The same configuration as above, only for customers having their IP address previously placed in the address list "even":
First, we take every second packet that establishes a new session (note connection-state=new), and mark it with the connection mark "odd". Consequently, all successive packets belonging to the same session will carry the connection mark "odd". Note that we are passing these packets to the second and third rules (passthrough=yes). The second rule adds the IP address of the client to the address list to enable all successive sessions to go through the same gateway. The third rule places the routing mark "odd" on all packets that belong to the "odd" connection and stops processing all other mangle rules for these packets in the prerouting chain:
These rules do the same for the remaining half of the traffic as the first three rules for the first half of the traffic.
The following code effectively means that each new connection initiated through the router from the local network will be marked as either "odd" or "even" with both routing and connection marks.
The above works fine. There are however some situations where you might find that the same IP address is listed under both the ODD and EVEN scr-address-lists. This behavior causes issues with apps that require persistent connections. A simple remedy for this situation is to add the following statement to your mangle rules and this will ensure that the new connection will not already be part of the ODD src-address-list. You will have to do the same for the ODD mangle rule thus excluding IPs already part of the EVEN scr-address-list:
Fix the source address according to the outgoing interface:
For all traffic marked "odd" (consequently having 10.111.0.2 translated source address) we use 10.111.0.1 gateway. In the same manner, all traffic marked "even" is routed through the 10.112.0.1 gateway.
Finally, we have one additional entry specifying that traffic from the router itself (the traffic without any routing marks) should go to the 10.111.0.2 gateway: