Page History

...

More advanced firewall setups, or complicated tasks, such as traffic prioritization, routing policies, where it is necessary to utilize more than one RouterOS facility, requires require knowledge: How these facilities work together? What happens when and why?

...

• overall diagram;
• detailed bridging, routing, and MPLS flow diagram;
• a diagram that shows what facilities and in what order is included in prerouting, input, forward, output, and postrouting.

Overall Packetflow Diagram

...

There are 4 boxes in the center of the diagram: Bridging, Routing, Mpls decisions, and local router processes. So for example, if the packet needs to be routed over the router, a packet will flow as illustrated in the image below. Without looking deeper into each facility, the packet enters in-interface, the router determines that it is IP traffic and needs to be routed, the packet goes through all routing processes and exits out-interface.

Let's take a look at another example that will illustrate what happens if the packet's destination is a router. For example, the in-interface receives ICMP (ping) packet, its destination is the router itself, so the packet will go for local-in processing. After the packet is processed ICMP (ping) reply is generated inside the router (local-out processing) and will be sent out over the out-interface.

...

• physical in-interface - the starting point of the packet received by the router;
• logical in-interface - the starting point of the decapsulated packet (from tunnels, IPsec, etc);
• local in - the last point of a packet destined to router itself;
• interface HTB (Hierarchical Token Bucket) - interface queue;
• physical out-interface - last point of the packet before it is actually sent out;
• logical out-interface - last point of the packet before encapsulation (to tunnels, IPsec, etc);
• local out - the starting point of a packet generated by the router;

...

• routing decision - go through routes in the routing table to find a match for the destination IP address of the packet. When a match is found - the packet will be processed further, in case of no match - the packet will be discarded.;
• mpls decision - what to do with the packet based on MPLS forwarding tables;
• bridging decision - bridge goes through the MAC address table to find a match for the destination MAC address of the packet. When a match is found - the packet will be processed further, in case of no match - multiple copies of the packet will be created and packets will be flooded (sent out via all bridge ports). A single packet copy will also reach a bridge input chain as the bridge interface itself is one of the many destinations.
• use-ip-firewall - whether a a 'use-ip-firewall' option is enabled in bridge settings;
• ipsec-policy - whether a packet matches any of configured IPsec policies;

...

• The PREROUTING chain: Rules in this chain apply to packets as they just arrive on the network interface. This chain is present in the nat, mangle and raw tables.
• The INPUT chain: Rules in this chain apply to packets just before they’re given to a local process. This chain is present in the mangle and filter tables.
• The OUTPUT chain: The rules here apply to packets just after they’ve been produced by a process. This chain is present in the raw, mangle, nat, and filter tables.
• The FORWARD chain: The rules here apply to any packets that are routed through the current host. This chain is only present in the mangle and filter tables.
• The POSTROUTING chain: The rules in this chain apply to packets as they just leave the network interface. This chain is present in the nat and mangle tables.

Each of the prerouting, input, forward, output, and postrouting blocks contain even more facilities, which are illustrated in the third part of the packet flow diagram:

...

• Hotspot-in - allow to capture traffic which otherwise would be discarded by connection tracking - this way our Hotspot feature is able to provide connectivity even if networks settings are an incomplete mess ;
• RAW Prerouting - RAW table prerouting chain;
• Connection tracking - packet is processed by connection tracking;
• Mangle Prerouting - Mangle prerouting chain;
• Mangle Input - Mangle input chain;
• Filter Input - Firewall filter input chain;
• HTB Global - Queue tree;
• Simple Queues - ;
• TTL - indicates an exact place where Time To Live (TTL) of the routed packet is reduced by 1 if TTL becomes 0, a packet will be discarded;
• Mangle Forward - Mangle forward chain;
• Filter Forward - Mangle forward chain;
• Accounting - Authentication, Authorization, and Accounting feature processing;
• RAW Output - RAW table output chain;
• Mangle Output - Mangle output chain;
• Filter Output - Firewall filter output chain;
• Routing Adjustment - this is a workaround that allows to set-up policy routing in mangle chain output (routing-mark) ;
• Mangle Postrouting - Mangle postrouting chain;
• Src Nat - Network Address Translation srcnat chain;
• Dst Nat - Network Address Translation dstnat chain;
• Hotspot-out - undo all that was done by hotspot-in for the packets that are going back to the client;

...

Section

Column width 400px We already learned that packet goes into in-interface, the router determines that it is an IP packet and need to be routed, and here starts the complicated process:  Packet  The packet enters prerouting processing: check if there is a hotspot and modify the packet for hotspot use process packet through RAW prerouting chain; send the packet through connection tracking; process packet through Mangle prerouting chain;  process packet through NATs dst-nat chain; Run packet through routing table to make routing decision; Packet The packet enters forward process; check TTL value; process packet through Mangle forward chain; process packet through Filter forward chain; send the packet to accounting processes; A packet enters postrouting process; process packet through Mangle postrouting chain; process packet through NATs src-nat chain; if there is a hotspot undo any modifications made in hotspot-in; process packet through queue tree (HTB Global); process packet through simple queues; Check if there is IPsec and then process through IPsec policies; Column

...

...

...

Below is discussed a general bridging process in RouterOS. Most of the packets will always follow the same processing path, but in certain configurations (e.g. with enabled VLAN filtering, horizon, STP, DHCP, or IGMP snooping) some packets can be treated differently. Please visit the bridging manual for more specific information.

...

...

Bridge Output

...

...

The hardware offloading, however, does not restrict a device to only hardware limited features, rather it is possible to take advantage of the hardware and software processing at the same time. This does require a profound understanding of how packets travel through the switch chip and when exactly they are passed to the main CPU.

...

We will further discuss a packet flow when a bridge hardware offloading is enabled and a packet is forwarded between two switched ports on a single switch chip. This is the most common and also the simplest example:

...

• a packet's destination MAC address match with a local MAC address, e.g. when a packet is destined to a local bridge interface;
• a packet might get flooded to all switch ports including the switch-cpu port, e.g. when broadcast, multicast, or unknown unicast traffic is received;
• a switch might have learned that some hosts can only be reached through the CPU (switch-cpu port learning is discussed in the next section), e.g. when a bridge contains HW and non-HW offloaded interfaces, such as wireless, EoIP, and even Ethernet interfaces;
• a packet is intentionally copied to the switch-cpu, e.g. for a packet inspection;
• a packet is triggered by the switch configuration and should be processed in software, e.g. a DHCP or IGMP snooping.

...

...

This process takes place when a packet exits the RouterOS software processing and it is received on the switch-cpu port. Again, there are two paths the packet can take. One where hardware offloading and switching is not even used (e.g. a standalone interface for routing or a bridged interface but with deliberately disabled HW offloading), so the packet is simply sent out through the physical out-interface. Another path is taken when hardware offloading is active on the out-interface. This will cause the packet to pass through the switching decision. Just like any other switch port, the switch will learn the source MAC addresses from packets that are received on the switch-cpu port. This does come in handy when a bridge contains HW and non-HW offloaded interfaces, so the switch can learn which frames should be forwarded to the CPU. See the packet walkthrough when an out-interface is hardware offloaded:

...

So far we looked at examples when in or out interfaces are actual physical interfaces (Ethernet, wireless), but how packets will flow if the router receives tunnel encapsulated packetpackets?

Let's assume that there is an IPIP packet coming in the router. Since it is a regular ipv4 packet it will be processed through all routing-related facilities ( until "J" in the diagram). Then the router will look if the packet needs to be decapsulated., in this case, it is an IPIP packet so "yes" send the packet to decapsulation. After that packet will go another loop through all the facilities but this time as a decapsulated IPv4 packet.

...

...

...

FastPath is an interface driver extension, that allows a driver to talk directly to specific RouterOS facilities and skipping all others.

Packet The packet can be forwarded by a fast path handler only if at least the source interface supports a fast path. For complete fast-forwarding, destination interface support is also required.

Currently, RouterOS has the following FastPath handlers:

...

• firewall rules are not configured;
• firewall address lists are not configured;
• simple and queue trees with parent=global are not configured;
• no mesh, metarouter interface configuration;
• sniffer, torch, and traffic generator is not running;
• connection tracking is not active;
• ip IP accounting is disabled;
• VRFs are not configured (/ip route vrf is empty);
• Hotspot A hotspot is not used (/ip hotspot has no interfaces);
• IPSec policies are not configured;
• /tool mac-scan is not actively used;
• /tool ip-scan is not actively used.

...

Traffic Generator and MPLS automatically use FastPath if the interface support supports this feature. Currently, MPLS fast-path applies only to MPLS switched traffic (frames that enter router as MPLS and must leave router as MPLS) - MPLS ingress and egress (including VPLS tunnel endpoints that do VPLS encap/decap) will operate as before.

...

• there are no bridge firewall rules;
• use-ip-firewall is disabled;
• no mesh, MetaRouter interface configuration;
• sniffer, torch, and traffic generator is not running.

...

EoIP, Gre, IPIP, and L2TP interfaces have per-interface setting allow-fast-path. Allowing fast path on these interfaces have has a side effect of bypassing firewall, connection tracking, simple queues, queue tree with parent=global, IP accounting, IPsec, hotspot universal client, vrf assignment for encapsulated packets that go trough fast path. Also, packet fragments cannot be received in FastPath.

...

Only interface queue that guarantees FastPath is only-hardware-queue. If you need an interface queue other than hardware then the packet will not go fully FastPath, but there is not a big impact on performance, as "interface queue" is the last step in the packet flow.

Packet The packet may go Half-FastPath by switching from FastPath to SlowPath, but not the other way around. So, for example, if the receiving interface has FastPath support, but the out interface does not, then the router will process the packet by FastPath handlers as far as it can and then proceed with SlowPath. If the receiving interface does not support FastPath but the out interface does, the packet will be processed by SlowPath all the way through the router.

...

Fasttrack can be decoded as Fast Path + Connection Tracking. It allows to mark marking connections as "fasttrackedfast-tracked", after marking packets that belong to fastracked fast-tracked connection will be sent fastpath fast-path way. Connection The connection table entry for such a connection now will have fasttracked a fast-tracked flag.

To mark a connection as fasttracked fast-tracked new action was implemented "fasttrack-connection" for firewall filter and mangle. Currently, only IPv4 TCP and UDP connections can be fasttracked fast-tracked and to maintain connection tracking entries some random packets will still be sent to a slow path. This must be taken into consideration when designing firewalls with enabled "fasttrack".

FastTrack handler also supports source and destination NAT, so special exceptions for NATed connections are not required.

Easiest The easiest way to start using this feature on home routers is to enable "fasttrack" for all established, related connections:

/ip firewall filter 
add chain=forward action=fasttrack-connection connection-state=established,related \
  comment="fasttrack established/related"
add chain=forward action=accept connection-state=established,related \
  comment="accept established/related"

Notice that the first rule marks established/related connections as fasttrackedfast-tracked, the second rule is still required to accept packets belonging to those connections. The reason for this is that, as was mentioned earlier, some random packets from fasttracked fast-tracked connections are still sent the slow path way and only UDP and TCP are fasttrackedfast-tracked, but we still want to accept packets for other protocols.

After adding "fasttrack" rule special dummy rule appeared at the top of the list. This is not an actual rule, it is for visual information showing that some of the traffic is traveling FastPath and will not reach other firewall rules.

These rules appear as soon as there is at least one "fasttracked" fast-tracked connection tracking entry and will disappear after the last "fasttracked" fast-tracked connection times out in the connection table.

...