...
- Hotspot-in - allows to capture traffic that otherwise would be discarded by connection tracking - this way our Hotspot feature is able to provide connectivity even if networks settings are an incomplete mess ;
- RAW Prerouting - RAW table prerouting chain;
- Connection tracking - packet is processed by connection tracking;
- Mangle Prerouting - Mangle prerouting chain;
- Mangle Input - Mangle input chain;
- Filter Input - Firewall filter input chain;
- HTB Global - Queue tree;
- Simple Queues - is a feature that can be used to limit traffic for a particular target;
- TTL - indicates an exact place where the Time To Live (TTL) of the routed packet is reduced by 1 if TTL becomes 0, a packet will be discarded;
- Mangle Forward - Mangle forward chain;
- Filter Forward - Mangle Filter forward chain;
- Accounting - Authentication, Authorization, and Accounting feature processing;
- RAW Output - RAW table output chain;
- Mangle Output - Mangle output chain;
- Filter Output - Firewall filter output chain;
- Routing Adjustment - this is a workaround that allows to set up policy routing in mangle chain output (routing-mark) ;
- Mangle Postrouting - Mangle postrouting chain;
- Src Nat - Network Address Translation srcnat chain;
- Dst Nat - Network Address Translation dstnat chain;
- Hotspot-out - undo all that was done by hotspot-in for the packets that are going back to the client;
...
Section | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
Info |
---|
For RouterOS v6: Tagged packets might get decapsulated on the "BRIDGING DECISION" block, which means these packets will no longer match the |
Bridge Input
Bridge input is a process that takes place when a packet is destined for the bridge interface. Most commonly this happens when you need to reach some services that are running on the bridge interface (e.g. a DHCP server) or you need to route traffic to other networks. The very first steps are similar to the bridge forward process - after receiving a packet on the in-interface, the device determines that the in-interface is a bridge port, so it gets passed through the bridging process:
When bridge Tagged packets might get decapsulated on the "BRIDGING DECISION" block, which means these packets will no longer match the |
Bridge Input
Bridge input is a process that takes place when a packet is destined for the bridge interface. Most commonly this happens when you need to reach some services that are running on the bridge interface (e.g. a DHCP server) or you need to route traffic to other networks. The very first steps are similar to the bridge forward process - after receiving a packet on the in-interface, the device determines that the in-interface is a bridge port, so it gets passed through the bridging process:
Section | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
Section | ||||||||||
|
Bridge Output
Bridge output is a process that takes place when a packet should exit the device through one or multiple bridge ports. Most commonly this happens when a bridge interface itself tries to reach a device connected to a certain bridge port (e.g. when a DHCP server running on a bridge interface is responding to a DHCP client). After a packet is processed on other higher-level RouterOS processes and the device finally determines that the output interface is a bridge, the packet gets passed through the bridging process:
|
Bridge Output
Bridge output is a process that takes place when a packet should exit the device through one or multiple bridge ports. Most commonly this happens when a bridge interface itself tries to reach a device connected to a certain bridge port (e.g. when a DHCP server running on a bridge interface is responding to a DHCP client). After a packet is processed on other higher-level RouterOS processes and the device finally determines that the output interface is a bridge, the packet gets passed through the bridging process:
Section | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
Section | ||||||||||
|
...
Section | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|
|
...
On the previous topic, we solely discussed a software bridging that requires the main CPU processing to forward packets through the correct bridge port. Most of the MikroTik devices are equipped with dedicated switching hardware, the so-called switch chip or switch ASIC. This allows us to offload some of the bridging functions, like packet forwarding between bridge ports or packet filtering, to this specialized hardware chip without consuming any CPU resources. In RouterOS, we have named this function Bridge Hardware (HW) Offloading. Different MikroTik devices might have different switch chips and each chip has a different set of features available, so make sure to visit this article to get more details - Bridge Hardware Offloading.
Note |
---|
Interface HTB will not work correctly when the out-interface is hardware offloaded and the bridge Fast Path is not active. |
...
- switching decision - widely depends on the switch model. This block controls all the switching-related tasks, like host learning, packet forwarding, filtering, rate-limiting, VLAN tagging/untagging, mirroring, etc. Certain switch configurations can alter the packet flow;
- switch-cpu port - a special purpose switch port for communication between the main CPU and other switch ports. Note that the switch-cpu port does not show up anywhere on RouterOS except for the switch menu, none of the software-related configurations (e.g. interface-list) can be applied to this port. Packets that reach the CPU are automatically associated with the physical in-interface.
The hardware offloading, however, does not restrict a device to only hardware limited features, rather it is possible to take advantage of the hardware and software processing at the same time. This does require a profound understanding of how packets travel through the switch chip and when exactly they are passed to the main CPU.
Info |
---|
Switch features found in the "/interface/ethernet/switch" menu and its sub-menus, like ACL rules, mirroring, ingress/egress rate limiters, QoS, and L3HW (except inter-VLAN routing) may not rely on bridge hardware offloading. Therefore, they can potentially be applied to interfaces not configured within a hardware-offloaded bridge. |
Switch ForwardSwitch Forward
We will further discuss a packet flow when bridge hardware offloading is enabled and a packet is forwarded between two switched ports on a single switch chip. This is the most common and also the simplest example:
...
Section | ||
---|---|---|
|
Push Label
Section | ||
---|---|---|
|
Logical Interfaces
So far we looked at examples when in or out interfaces are actual physical interfaces (Ethernet, wireless), but how packets will flow if the router receives tunnel encapsulated packets?
Let's assume that there is an IPIP packet coming into the router. Since it is a regular ipv4 packet it will be processed through all routing-related facilities ( until "J" in the diagram). Then the router will look if the packet needs to be decapsulated., in this case, it is an IPIP packet so "yes" send the packet to decapsulation. After that packet will go another loop through all the facilities but this time as a decapsulated IPv4 packet.
It is very important because the packet actually travels through the firewall twice, so if there is a strict firewall, then there should be "accept" rules for IPIP encapsulated packets as well as decapsulated IP packets.
Info |
---|
Packet encapsulation and decapsulation using a bridge with enabled |
IPSec Policies
MPLS IP VPN
In VPNv4 setups packet arriving at PE router that needs to be forwarded to the CE router is not a typical "forward".
If incoming label and destination is bound to the VRF Then after MPLS label is popped and:
- destination address is local to the router, then packet is moved to LOCAL_IN
- destination address is in the CE network, then packet is moved to LOCAL_OUT
Warning |
---|
Forwarded packets from MPLS cloud to the CE network will not show up in the forward. |
For example traffic from src:111.15.0.1 to dst:111.13.0.1
Code Block | ||
---|---|---|
| ||
[admin@CCR2004_2XS] /mpls/forwarding-table> print
Flags: L - LDP, P - VPN
Columns: LABEL, VRF, PREFIX, NEXTHOPS
# LABEL VRF PREFIX NEXTHOPS
0 P 17 myVrf 111.13.0.0/24
4 L 20 main 203.0.113.2 { label=impl-null; nh=111.11.0.1; interface=sfp-sfpplus1 }
[admin@CCR2004_2XS] /mpls/forwarding-table>
...
[admin@CCR2004_2XS] /ip/route> print detail
Flags: D - dynamic; X - disabled, I - inactive, A - active; c - connect, s - static, r - rip, b - bgp, o - ospf, i - is-is, d - dhcp, v - vpn, m - modem, y - bgp-mpls-vpn;
H - hw-offloaded; + - ecmp
DAc dst-address=111.11.0.0/24 routing-table=main gateway=sfp-sfpplus1 immediate-gw=sfp-sfpplus1 distance=0 scope=10 suppress-hw-offload=no
local-address=111.11.0.2%sfp-sfpplus1
DAc dst-address=203.0.113.1/32 routing-table=main gateway=lo immediate-gw=lo distance=0 scope=10 suppress-hw-offload=no local-address=203.0.113.1%lo
DAo dst-address=203.0.113.2/32 routing-table=main gateway=111.11.0.1%sfp-sfpplus1 immediate-gw=111.11.0.1%sfp-sfpplus1 distance=110 scope=20 target-scope=10
suppress-hw-offload=no
DAc dst-address=111.13.0.0/24 routing-table=myVrf gateway=sfp-sfpplus2@myVrf immediate-gw=sfp-sfpplus2 distance=0 scope=10 suppress-hw-offload=no
local-address=111.13.0.2%sfp-sfpplus2@myVrf
DAy dst-address=111.15.0.0/24 routing-table=myVrf gateway=203.0.113.2 immediate-gw=111.11.0.1%sfp-sfpplus1 distance=200 scope=40 target-scope=30 suppress-hw-offload=no [admin@CCR2004_2XS] /ip/route> |
Packet will be seen in the output and postrouting chains, because now it is locally originated packet with source MAC address equal to vrfInterface:
Code Block | ||
---|---|---|
| ||
08:10:55 firewall,info output: in:(unknown 0) out:sfp-sfpplus2, connection-state:established src-mac f2:b5:e9:17:18:3b, proto ICMP (type 8, code 0), 111.15.0.1->111.13.0.1, len 56
08:10:55 firewall,info postrouting: in:myVrf out:sfp-sfpplus2, connection-state:established src-mac f2:b5:e9:17:18:3b, proto ICMP (type 8, code 0), 111.15.0.1->111.13.0.1, len 56 |
On the other hand, for packets routed in the direction CE→PE will be seen in the "forward" as any other routed IP traffic before being sent to the MPLS:
Code Block | ||
---|---|---|
| ||
08:10:55 firewall,info prerouting: in:sfp-sfpplus2 out:(unknown 0), connection-state:established src-mac dc:2c:6e:46:f8:93, proto ICMP (type 0, code 0), 111.13.0.1->111.15.0.1, len 56
08:10:55 firewall,info forward: in:sfp-sfpplus2 out:sfp-sfpplus1, connection-state:established src-mac dc:2c:6e:46:f8:93, proto ICMP (type 0, code 0), 111.13.0.1->111.15.0.1, len 56
08:10:55 firewall,info postrouting: in:sfp-sfpplus2 out:sfp-sfpplus1, connection-state:established src-mac dc:2c:6e:46:f8:93, proto ICMP (type 0, code 0), 111.13.0.1->111.15.0.1, len 56
|
But there can be an exception. If destination IP of reply packet is local to the router and connection tracking is performing NAT translation then connection tracking will "force" packet to move through the firewall prerouting/forward/postrouting chains.
Logical Interfaces
So far we looked at examples when in or out interfaces are actual physical interfaces (Ethernet, wireless), but how packets will flow if the router receives tunnel encapsulated packets?
Let's assume that there is an IPIP packet coming into the router. Since it is a regular ipv4 packet it will be processed through all routing-related facilities ( until "J" in the diagram). Then the router will look if the packet needs to be decapsulated., in this case, it is an IPIP packet so "yes" send the packet to decapsulation. After that packet will go another loop through all the facilities but this time as a decapsulated IPv4 packet.
It is very important because the packet actually travels through the firewall twice, so if there is a strict firewall, then there should be "accept" rules for IPIP encapsulated packets as well as decapsulated IP packets.
Info |
---|
Packet encapsulation and decapsulation using a bridge with enabled |
IPSec Policies
Let's take a look at another tunnel type - IPSec. This type of VPN does not have logical interfaces but is processed in a similar Let's take a look at another tunnel type - IPSec. This type of VPN does not have logical interfaces but is processed in a similar manner.
Section | |||||||
---|---|---|---|---|---|---|---|
|
...
RouterBoard | Interfaces | |
---|---|---|
RB6xx series | ether1,2RB7xx series | all ports |
RB800 | ether1,2 | |
RB9xx series | all ports | |
RB1000 | all ports | |
RB1100 series | ether1-11 | |
RB2011 series | all ports | |
RB3011 series | all ports | |
CRS series routers | all ports | |
CCR series routers | all ports | |
All devices | wireless interfaces | bridge interfaces | VLAN, VRRP interfaces | bonding interfaces (RX only) | PPPoE, L2TP interfaces | EoIP, GRE, IPIP interfaces. |
EoIP, Gre, IPIP, and L2TP interfaces have per-interface setting allow-fast-path. Allowing a fast path on these interfaces has a side effect of bypassing firewall, connection tracking, simple queues, queue tree with parent=global, IP accounting, IPsec, hotspot universal client, vrf assignment for encapsulated packets that go through a fast-path. Also, packet fragments cannot be received in FastPath.
Note |
---|
FastPath support can be verified by checking the fast-path property value in |
Only interface queue that guarantees FastPath is only-hardware-queue. If you need an interface queue other than hardware then the packet will not go fully FastPath, but there is not a big impact on performance, as "interface queue" is the last step in the packet flow.
The packet may go Half-FastPath by switching from FastPath to SlowPath, but not the other way around. So, for example, if the receiving interface has FastPath support, but the out interface does not, then the router will process the packet by FastPath handlers as far as it can and then proceed with SlowPath. If the receiving interface does not support FastPath but the out interface does, the packet will be processed by SlowPath all the way through the router.
FastTrack
Fasttrack can be decoded as Fast Path + Connection Tracking. It allows marking connections as "fast-tracked", marking packets that belong to fast-tracked connection will be sent fast-path way. The connection table entry for such a connection now will have a fast-tracked flag.
Note |
---|
FastTrack packets bypass firewall, connection tracking, simple queues, queue tree with parent=global, ip traffic-flow(restriction removed in 6.33), IP accounting, IPSec, hotspot universal client, VRF assignment, so it is up to the administrator to make sure FastTrack does not interfere with other configuration! |
To mark a connection as fast-tracked new action was implemented "fasttrack-connection" for firewall filter and mangle. Currently, only IPv4 TCP and UDP connections can be fast-tracked and to maintain connection tracking entries some random packets will still be sent to a slow path. This must be taken into consideration when designing firewalls with enabled "fasttrack".
FastTrack handler also supports source and destination NAT, so special exceptions for NATed connections are not required.
Warning |
---|
Traffic that belongs to a fast-tracked connection travels in FastPath, which means that it will not be visible by other router L3 facilities (firewall, queues, IPsec, IP accounting, VRF assignment, etc). |
The easiest way to start using this feature on home routers is to enable "fasttrack" for all established, related connections:
Code Block | ||||
---|---|---|---|---|
| ||||
/ip firewall filter
add chain=forward action=fasttrack-connection connection-state=established,related \
comment="fasttrack established/related"
add chain=forward action=accept connection-state=established,related \
comment="accept established/related" |
Notice that the first rule marks established/related connections as fast-tracked, the second rule is still required to accept packets belonging to those connections. The reason for this is that, as was mentioned earlier, some random packets from fast-tracked connections are still sent the slow pathway and only UDP and TCP are fast-tracked, but we still want to accept packets for other protocols.
After adding the "FastTrack" rule special dummy rule appeared at the top of the list. This is not an actual rule, it is for visual information showing that some of the traffic is traveling FastPath and will not reach other firewall rules.
These rules appear as soon as there is at least one fast-tracked connection tracking entry and will disappear after the last fast-tracked connection times out in the connection table.
...
All devices | Ethernet interfaces |
wireless interfaces | |
bridge interfaces | |
VLAN, VRRP interfaces | |
bonding interfaces (RX only) | |
PPPoE, L2TP interfaces | |
EoIP, GRE, IPIP, VXLAN interfaces. |
EoIP, Gre, IPIP, VXLAN and L2TP interfaces have per-interface setting allow-fast-path. Allowing a fast path on these interfaces has a side effect of bypassing firewall, connection tracking, simple queues, queue tree with parent=global, IP accounting, IPsec, hotspot universal client, vrf assignment for encapsulated packets that go through a fast-path. Also, packet fragments cannot be received in FastPath.
Note |
---|
FastPath support can be verified by checking the fast-path property value in |
Only interface queue that guarantees FastPath is only-hardware-queue. If you need an interface queue other than hardware then the packet will not go fully FastPath, but there is not a big impact on performance, as "interface queue" is the last step in the packet flow.
The packet may go Half-FastPath by switching from FastPath to SlowPath, but not the other way around. So, for example, if the receiving interface has FastPath support, but the out interface does not, then the router will process the packet by FastPath handlers as far as it can and then proceed with SlowPath. If the receiving interface does not support FastPath but the out interface does, the packet will be processed by SlowPath all the way through the router.
FastTrack
Fasttrack can be decoded as Fast Path + Connection Tracking. It allows marking connections as "fast-tracked", marking packets that belong to fast-tracked connection will be sent fast-path way. The connection table entry for such a connection now will have a fast-tracked flag.
Note |
---|
FastTrack packets bypass firewall, connection tracking, simple queues, queue tree with parent=global, ip traffic-flow(restriction removed in 6.33), IP accounting, IPSec, hotspot universal client, VRF assignment, so it is up to the administrator to make sure FastTrack does not interfere with other configuration! |
To mark a connection as fast-tracked new action was implemented "fasttrack-connection" for firewall filter and mangle. Currently, only IPv4 TCP and UDP connections can be fast-tracked and to maintain connection tracking entries some random packets will still be sent to a slow path. This must be taken into consideration when designing firewalls with enabled "fasttrack".
FastTrack handler also supports source and destination NAT, so special exceptions for NATed connections are not required.
Warning |
---|
Traffic that belongs to a fast-tracked connection travels in FastPath, which means that it will not be visible by other router L3 facilities (firewall, queues, IPsec, IP accounting, VRF assignment, etc). Fasttrack lookups route before routing marks have been set, so it works only with the main routing table. |
The easiest way to start using this feature on home routers is to enable "fasttrack" for all established, related connections:
Code Block | ||||
---|---|---|---|---|
| ||||
/ip firewall filter
add chain=forward action=fasttrack-connection connection-state=established,related \
comment="fasttrack established/related"
add chain=forward action=accept connection-state=established,related \
comment="accept established/related" |
Notice that the first rule marks established/related connections as fast-tracked, the second rule is still required to accept packets belonging to those connections. The reason for this is that, as was mentioned earlier, some random packets from fast-tracked connections are still sent the slow pathway and only UDP and TCP are fast-tracked, but we still want to accept packets for other protocols.
After adding the "FastTrack" rule special dummy rule appeared at the top of the list. This is not an actual rule, it is for visual information showing that some of the traffic is traveling FastPath and will not reach other firewall rules.
Note |
---|
FastTrack can process packets only in the main routing table so it is the system administrator duty to not FastTrack connections that are going through non-main routing table (thus connections that are processed with mangle action=mark-routing rules). Otherwise packets might be misrouted though the main routing table. |
These rules appear as soon as there is at least one fast-tracked connection tracking entry and will disappear after the last fast-tracked connection times out in the connection table.
Note |
---|
The connection is FastTracked until a connection is closed, timed out or the router is rebooted. |
Configuration example: excluding specific host, from being Fast-Tracked
Code Block | ||
---|---|---|
| ||
/ip firewall filter
add action=accept chain=forward connection-state=established,related src-address=192.168.88.111
add action=accept chain=forward connection-state=established,related dst-address=192.168.88.111
add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=no
add action=accept chain=forward connection-state=established,related |
In this example, we exclude host 192.168.88.111, from being Fast-tracked, by first accepting it with the firewall rule, both for source and destination. The idea is - not allowing the traffic to reach the FastTrack action.
Note: the "exclusion" rules, must be placed before fasttrack filters, order is important
...
.
Requirements
IPv4 FastTrack is active if the following conditions are met:
- no mesh, metarouter interface configuration;
- sniffer, torch, and traffic generator are not running;
- "/tool mac-scan" is not actively used;
- "/tool ip-scan" is not actively used;
- FastPath and Route cache is are enabled under IP/Settings (route cache condition does not apply to RouterOS v7 or newer);
- bridge FastPath is enabled if a connection is going over the bridge interface;
Packet flow for the visually impaired
...