Introduction
Layer 3 Hardware Offloading (L3HW, otherwise known as IP switching or HW routing) allows to offload some router features onto the switch chip. This allows reaching wire speeds when routing packets, which simply would not be possible with the CPU.
Switch Configuration
To enable Layer 3 Hardware Offloading, set l3-hw-offloading=yes for the switch:
/interface/ethernet/switch set 0 l3-hw-offloading=yes
Switch Port Configuration
Layer 3 Hardware Offloading can be configured for each physical switch port. For example:
/interface/ethernet/switch/port set sfp-sfpplus1 l3-hw-offloading=yes
Note that l3hw settings for switch and ports are different:
- Setting
l3-hw-offloading=nofor the switch completely disables offloading - all packets will be routed by CPU. - However, setting
l3-hw-offloading=nofor a switch port only disables hardware routing from/to this particular port. Moreover, the port can still participate in Fastrack connection offloading.
To enable full hardware routing, enable l3hw on all switch ports:
/interface/ethernet/switch set 0 l3-hw-offloading=yes /interface/ethernet/switch/port set [find] l3-hw-offloading=yes
To make all packets go through the CPU first, and offload only the Fasttrack connections, disable l3hw on all ports but keep it enabled on the switch chip itself:
/interface/ethernet/switch set 0 l3-hw-offloading=yes /interface/ethernet/switch/port set [find] l3-hw-offloading=no
Packets get routed by the hardware only if both source and destination ports have l3-hw-offloading=yes. If at least one of them has l3-hw-offloading=no, packets will go through the CPU/Firewall while offloading only the Fasttrack connections.
The next example enables hardware routing on all ports but the upstream port (sfp-sfpplus16). Packets going to/from sfp-sfpplus16 will enter the CPU and, therefore, subject to Firewall/NAT processing.
/interface/ethernet/switch set 0 l3-hw-offloading=yes /interface/ethernet/switch/port set [find] l3-hw-offloading=yes /interface/ethernet/switch/port set sfp-sfpplus16 l3-hw-offloading=no
The existing connections may be unaffected by the l3-hw-offloading setting change.
L3HW Settings
The L3HW Settings menu has been introduced in RouterOS version 7.6.
Sub-menu: /interface ethernet switch l3hw-settings
Property | Description |
|---|---|
| fasttrack-hw (yes | no; Default: yes (if supported)) | Enables or disables FastTrack HW Offloading. Keep it enabled unless HW TCAM memory reservation is required, e.g., for dynamic switch ACL rules creation. Not all switch chips support FastTrack HW Offloading (see hw-supports-fasttrack). |
| ipv6-hw (yes | no; Default: no) | Enables or disables IPv6 Hardware Offloading. Since IPv6 routes occupy a lot of HW memory, enable it only if IPv6 traffic speed is significant enough to benefit from hardware routing. |
| icmp-reply-on-error (yes | no; Default: yes) | Since the hardware cannot send ICMP messages, the packet must be redirected to the CPU to send an ICMP reply in case of an error (e.g., "Time Exceeded", "Fragmentation required", etc.). Enabling icmp-reply-on-error helps with network diagnostics but may open potential vulnerabilities for DDoS attacks. Disabling icmp-reply-on-error silently drops the packets on the hardware level in case of an error. |
Read-Only Properties
Property | Description |
|---|---|
| hw-supports-fasttrack (yes | no) | Indicates if the hardware (switch chip) supports FastTrack HW Offloading. |
Interface Lists
It is impossible to use interface lists directly to control l3-hw-offloading because an interface list may contain virtual interfaces (such as VLAN) while the l3-hw-offloading setting must be applied to physical switch ports only. For example, if there are two VLAN interfaces (vlan20 and vlan30) running on the same switch port (trunk port), it is impossible to enable hardware routing on vlan20 but keep it disabled on vlan30.
However, an interface list may be used as a port selector. The following example demonstrates how to enable hardware routing on LAN ports (ports that belong to the "LAN" interface list) and disable it on WAN ports:
:foreach i in=[/interface/list/member/find where list=LAN] do={
/interface/ethernet/switch/port set [/interface/list/member/get $i interface] l3-hw-offloading=yes
}
:foreach i in=[/interface/list/member/find where list=WAN] do={
/interface/ethernet/switch/port set [/interface/list/member/get $i interface] l3-hw-offloading=no
}
Please take into account that since interface lists are not used directly in the hardware routing control, modifying the interface list also does not automatically reflect into l3hw changes. For instance, adding a switch port to the "LAN" interface list does not automatically enable l3-hw-offloading on that. The user has to rerun the above script to apply the changes.
MTU
The hardware supports up to 8 MTU profiles, meaning that the user can set up to 8 different MTU values for interfaces: the default 1500 + seven custom ones.
l3-hw-offloading while changing the MTU/L2MTU values on the interfaces./interface/ethernet/switch set 0 l3-hw-offloading=no /interface set sfp-sfpplus1 mtu=9000 l2mtu=9022 /interface set sfp-sfpplus2 mtu=9000 l2mtu=9022 /interface set sfp-sfpplus3 mtu=10000 l2mtu=10022 /interface/ethernet/switch set 0 l3-hw-offloading=yes
Layer 2 Dependency
Layer 3 hardware processing lies on top of Layer 2 hardware processing. Therefore, L3HW offloading requires L2HW offloading on the underlying interfaces. The latter is enabled by default, but there are some exceptions. For example, CRS3xx devices support only one hardware bridge. If there are multiple bridges, others are processed by the CPU and are not subject to L3HW.
Another example is ACL rules. If a rule redirects traffic to the CPU for software processing, then hardware routing (L3HW) is not triggered:
/interface/ethernet/switch/rule/add switch=switch1 ports=ether1 redirect-to-cpu=yes
To make sure that Layer 3 is in sync with Layer 2 on both the software and hardware sides, we recommend disabling L3HW while configuring Layer 2 features. The recommendation applies to the following configuration:
- adding/removing/enabling/disabling bridge;
- adding/removing switch ports to/from the bridge;
- bonding switch ports / removing bond;
- changing VLAN settings;
- changing MTU/L2MTU on switch ports;
- changing ethernet (MAC) addresses.
In short, disable l3-hw-offloading while making changes under /interface/bridge/ and /interface/vlan/:
/interface/ethernet/switch set 0 l3-hw-offloading=no /interface/bridge # put bridge configuration changes here /interface/vlan # define/change VLAN interfaces /interface/ethernet/switch set 0 l3-hw-offloading=yes
MAC telnet and RoMON
There is a limitation for MAC telnet and RoMON when L3HW offloading is enabled on 98DX8xxx, 98DX4xxx or 98DX325x switch chips. Packets from these protocols are dropped and do not reach the CPU, thus access to the device will fail.
If MAC telnet or RoMON are desired in combination with L3HW, certain ACL rules can be created to force these packets to the CPU.
For example, if MAC telnet access on sfp-sfpplus1 and sfp-sfpplus2 is needed, you will need to add this ACL rule. It is possible to select even more interfaces with the ports setting.
/interface ethernet switch rule add dst-port=20561 ports=sfp-sfpplus1,sfp-sfpplus2 protocol=udp redirect-to-cpu=yes switch=switch1
For example, if RoMON access on sfp-sfpplus2 is needed, you will need to add this ACL rule.
/interface ethernet switch rule add mac-protocol=0x88BF ports=sfp-sfpplus2 redirect-to-cpu=yes switch=switch1
Inter-VLAN Routing
Since L3HW depends on L2HW, and L2HW is the one that does VLAN processing, Inter-VLAN hardware routing requires a hardware bridge underneath. Even if a particular VLAN has only one tagged port member, the latter must be a bridge member. Do not assign a VLAN interface directly on a switch port! Otherwise, L3HW offloading fails and the traffic will get processed by the CPU:
/interface/vlan add interface=ether2 name=vlan20 vlan-id=20
Assign VLAN interface to the bridge instead. This way, VLAN configuration gets offloaded to the hardware, and, with L3HW enabled, the traffic is subject to inter-VLAN hardware routing.
/interface/ethernet/switch set 0 l3-hw-offloading=no /interface/bridge/port add bridge=bridge interface=ether2 /interface/bridge/vlan add bridge=bridge tagged=bridge,ether2 vlan-ids=20 /interface/vlan add interface=bridge name=vlan20 vlan-id=20 /ip/address add address=192.0.2.1/24 interface=vlan20 /interface/bridge set bridge vlan-filtering=yes /interface/ethernet/switch set 0 l3-hw-offloading=yes
/interface/bridge/vlan/ entry.L3HW MAC Address Range Limitation (DX2000/DX3000 series only)
Marvell Prestera DX2000 and DX3000 switch chips have a hardware limitation that allows configuring only the last (least significant) octet of the MAC address for each interface. The other five (most significant) octets are configurated globally and, therefore, must be equal for all interfaces (switch ports, bridge, VLANs). In other words, the MAC addresses must be in the format "XX:XX:XX:XX:XX:??", where:
- "XX:XX:XX:XX:XX" part is common for all interfaces.
- "??" is a variable part.
This requirement applies only to Layer 3 (routing). Layer 2 (bridging) does not use the switch's ethernet addresses. Moreover, it does not apply to bridge ports because they use the bridge's MAC address.
The requirement for common five octets applies to:
- Standalone switch ports (not bridge members) with hardware routing enabled (
l3-hw-offloading=yes). - Bridge itself.
- VLAN interfaces (those are using bridge's MAC address by default).
Route Configuration
Suppressing HW Offload
By default, all the routes are participating to be hardware candidate routes. To further fine-tune which traffic to offload, there is an option for each route to disable/enable suppress-hw-offload.
For example, if we know that majority of traffic flows to the network where servers are located, we can enable offloading only to that specific destination:
/ip/route set [find where static && dst-address!="192.168.3.0/24"] suppress-hw-offload=yes
Now only the route to 192.168.3.0/24 has H-flag, indicating that it will be the only one eligible to be selected for HW offloading:
[admin@MikroTik] > /ip/route print where static Flags: A - ACTIVE; s - STATIC, y - COPY; H - HW-OFFLOADED Columns: DST-ADDRESS, GATEWAY, DISTANCE # DST-ADDRESS GATEWAY D 0 As 0.0.0.0/0 172.16.2.1 1 1 As 10.0.0.0/8 10.155.121.254 1 2 AsH 192.168.3.0/24 172.16.2.1 1
H-flag does not indicate that route is actually HW offloaded, it indicates only that route can be selected to be HW offloaded.
Routing Filters
For dynamic routing protocols like OSFP and BGP, it is possible to suppress HW offloading using routing filters. For example, to suppress HW offloading on all OSFP instance routes, use "suppress-hw-offload yes" property:
/routing/ospf/instance set [find name=instance1] in-filter-chain=ospf-input /routing/filter/rule add chain="ospf-input" rule="set suppress-hw-offload yes; accept"
Offloading Fasttrack Connections
Firewall filter rules have hw-offload option for Fasttrack, allowing fine-tuning connection offloading. Since the hardware memory for Fasttrack connections is very limited, we can choose what type of connections to offload and, therefore, benefit from near-the-wire-speed traffic. The next example offloads only TCP connections while UDP packets are routed via the CPU and do not occupy HW memory:
/ip/firewall/filter add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=yes protocol=tcp add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=no add action=accept chain=forward connection-state=established,related
Stateless Hardware Firewall
While connection tracking and stateful firewalling can be performed only by the CPU, the hardware can perform stateless firewalling via switch rules (ACL). The next example prevents (on a hardware level) accessing a MySQL server from the ether1, and redirects to the CPU/Firewall packets from ether2 and ether3:
/interface ethernet switch rule add switch=switch1 dst-address=10.0.1.2/32 dst-port=3306 ports=ether1 new-dst-ports="" add switch=switch1 dst-address=10.0.1.2/32 dst-port=3306 ports=ether2,ether3 redirect-to-cpu=yes
Switch Rules (ACL) vs. Fasttrack HW Offloading
Some firewall rules may be implemented both via switch rules (ACL) and CPU Firewall Filter + Fasttrack HW Offloading. Both options grant near-the-wire-speed performance. So the question is which one to use?
First, not all devices support Fasttrack HW Offloading. And without HW offloading, Firewall Filter uses only software routing, which is dramatically slower than its hardware counterpart. Second, even if Fasttrack HW Offloading is an option, a rule of thumb is:
Always use Switch Rules (ACL), if possible.
Switch rules share the hardware memory with Fastrack connections. However, hardware resources are allocated for each Fasttrack connection while a single ACL rule can match multiple connections. For example, if you have a guest WiFi network connected to sfp-sfpplus1 VLAN 10 and you don't want it to access your internal network, simply create an ACL rule:
/interface/ethernet/switch/rule add switch=switch1 ports=sfp-sfpplus1 vlan-id=10 dst-address=10.0.0.0/8 new-dst-ports=""
The matched packets will be dropped on the hardware level. It is much better than letting all guest packets to the CPU for Firewall filtering.
Of course, ACL rules cannot match everything. For instance, ACL rules cannot filter connection states: accept established, drop others. That is where Fasttrack HW Offloading gets into action - redirect the packets to the CPU by default for firewall filtering, then offload the established Fasttrack connections. However, disabling l3-hw-offloading for the entire switch port is not the only option.
Define ACL rules with redirect-to-cpu=yes instead of setting l3-hw-offloading=no of the switch port for narrowing down the traffic that goes to the CPU.
Configuration Examples
Inter-VLAN Routing with Upstream Port Behind Firewall/NAT
This example demonstrates how to benefit from near-to-wire-speed inter-VLAN routing while keeping Firewall and NAT running on the upstream port. Moreover, Fasttrack connections to the upstream port get offloaded to hardware as well, boosting the traffic speed close to wire-level. Inter-VLAN traffic is fully routed by the hardware, not entering the CPU/Firewall, and, therefore, not occupying the hardware memory of Fasttrack connections.
We use the CRS317-1G-16S+ model with the following setup:
- sfp1-sfp4 - bridged ports, VLAN ID 20, untagged
- sfp5-sfp8 - bridged ports, VLAN ID 30, untagged
- sfp16 - the upstream port
- ether1 - management port
Setup interface lists for easy access:
/interface list add name=LAN add name=WAN add name=MGMT /interface list member add interface=sfp-sfpplus1 list=LAN add interface=sfp-sfpplus2 list=LAN add interface=sfp-sfpplus3 list=LAN add interface=sfp-sfpplus4 list=LAN add interface=sfp-sfpplus5 list=LAN add interface=sfp-sfpplus6 list=LAN add interface=sfp-sfpplus7 list=LAN add interface=sfp-sfpplus8 list=LAN add interface=sfp-sfpplus16 list=WAN add interface=ether1 list=MGMT
/interface bridge add name=bridge vlan-filtering=yes /interface bridge port add bridge=bridge interface=sfp-sfpplus1 pvid=20 add bridge=bridge interface=sfp-sfpplus2 pvid=20 add bridge=bridge interface=sfp-sfpplus3 pvid=20 add bridge=bridge interface=sfp-sfpplus4 pvid=20 add bridge=bridge interface=sfp-sfpplus5 pvid=30 add bridge=bridge interface=sfp-sfpplus6 pvid=30 add bridge=bridge interface=sfp-sfpplus7 pvid=30 add bridge=bridge interface=sfp-sfpplus8 pvid=30 /interface bridge vlan add bridge=bridge tagged=bridge untagged=sfp-sfpplus1,sfp-sfpplus2,sfp-sfpplus3,sfp-sfpplus4 vlan-ids=20 add bridge=bridge tagged=bridge untagged=sfp-sfpplus5,sfp-sfpplus6,sfp-sfpplus7,sfp-sfpplus8 vlan-ids=30
Routing requires dedicated VLAN interfaces. For standard L2 VLAN bridging (without inter-VLAN routing), the next step can be omitted.
/interface vlan add interface=bridge name=vlan20 vlan-id=20 add interface=bridge name=vlan30 vlan-id=30 /ip address add address=192.168.20.17/24 interface=vlan20 network=192.168.20.0 add address=192.168.30.17/24 interface=vlan30 network=192.168.30.0
Configure management and upstream ports, a basic firewall, NAT, and enable hardware offloading of Fasttrack connections:
/ip address add address=192.168.88.1/24 interface=ether1 add address=10.0.0.17/24 interface=sfp-sfpplus16 /ip route add gateway=10.0.0.1 /ip firewall filter add action=fasttrack-connection chain=forward connection-state=established,related hw-offload=yes add action=accept chain=forward connection-state=established,related /ip firewall nat add action=masquerade chain=srcnat out-interface-list=WAN
At this moment, all routing still is performed by the CPU. Enable hardware routing on the switch chip:
# Enable full hardware routing on LAN ports
:foreach i in=[/interface/list/member/find where list=LAN] do={
/interface/ethernet/switch/port set [/interface/list/member/get $i interface] l3-hw-offloading=yes
}
# Disable full hardware routing on WAN or Management ports
:foreach i in=[/interface/list/member/find where list=WAN or list=MGMT] do={
/interface/ethernet/switch/port set [/interface/list/member/get $i interface] l3-hw-offloading=no
}
# Activate Layer 3 Hardware Offloading on the switch chip
/interface/ethernet/switch/set 0 l3-hw-offloading=yes
Results:
- Within the same VLAN (e.g., sfp1-sfp4), traffic is forwarded by the hardware on Layer 2 (L2HW).
- Inter-VLAN traffic (e.g. sfp1-sfp5) is routed by the hardware on Layer 3 (L3HW).
- Traffic from/to WAN port gets processed by the CPU/Firewall first. Then Fasttrack connections get offloaded to the hardware (Hardware-Accelerated L4 Stateful Firewall). NAT applies both on CPU- and HW-processed packets.
- Traffic to the management port is protected by the Firewall.
Typical Misconfiguration
Below are typical user errors of configuring Layer 3 Hardware Offloading.
VLAN interface on a switch port
/interface vlan add name=vlan10 vlan-id=10 interface=sfp-sfpplus1
VLAN interface must be set on the bridge due to Layer 2 Dependency. Otherwise, L3HW will not work. The correct configuration is:
/interface bridge port add bridge=bridge1 interface=sfp-sfpplus1 pvid=10 /interface bridge vlan add bridge=bridge1 tagged=bridge1,sfp-sfpplus1 vlan-ids=10 /interface vlan add name=vlan10 vlan-id=10 interface=bridge1
Not adding the bridge interface to /in/br/vlan
For Inter-VLAN routing, the bridge interface itself needs to be added to the tagged members of the given VLANs. In the next example, Inter-VLAN routing works between VLAN 10 and 11, but packets are NOT routed to VLAN 20.
/interface bridge vlan add bridge=bridge1 vlan-ids=10 tagged=bridge1,sfp-sfpplus1 add bridge=bridge1 vlan-ids=11 tagged=bridge1 untagged=sfp-sfpplus2,sfp-sfpplus3 add bridge=bridge1 vlan-ids=20 tagged=sfp-sfpplus1 untagged=sfp-sfpplus4,sfp-sfpplus5
The above example does not always mean an error. Sometimes, you may want the device to act as a simple L2 switch in some/all VLANs. Just make sure you set such behavior on purpose, not due to a mistake.
Creating multiple bridges
The devices support only one hardware bridge. If there are multiple bridges created, only one gets hardware offloading. While for L2 that means software forwarding for other bridges, in the case of L3HW, multiple bridges may lead to undefined behavior.
Instead of creating multiple bridges, create one and segregate L2 networks with VLAN filtering.
Using ports that do not belong to the switch
Some devices have two switch chips or the management port directly connected to the CPU. For example, CRS312-4C+8XG has an ether9 port connected to a separate switch chip. Trying to add this port to a bridge or involve it in the L3HW setup leads to unexpected results. Leave the management port for management!
[admin@crs312] /interface/ethernet/switch> print Columns: NAME, TYPE, L3-HW-OFFLOADING # NAME TYPE L3-HW-OFFLOADING 0 switch1 Marvell-98DX8212 yes 1 switch2 Atheros-8227 no [admin@crs312] /interface/ethernet/switch> port print Columns: NAME, SWITCH, L3-HW-OFFLOADING, STORM-RATE # NAME SWITCH L3-HW-OFFLOADING STORM-RATE 0 ether9 switch2 1 ether1 switch1 yes 100 2 ether2 switch1 yes 100 3 ether3 switch1 yes 100 4 ether4 switch1 yes 100 5 ether5 switch1 yes 100 6 ether6 switch1 yes 100 7 ether7 switch1 yes 100 8 ether8 switch1 yes 100 9 combo1 switch1 yes 100 10 combo2 switch1 yes 100 11 combo3 switch1 yes 100 12 combo4 switch1 yes 100 13 switch1-cpu switch1 100 14 switch2-cpu switch2
Relying on Fasttrack HW Offloading too much
Since Fasttrack HW Offloading offers near-the-wire-speed performance at zero configuration overhead, the users tempt to use it as the default solution. However, the number of HW Fasttrack connections is very limited, leaving the other traffic for the CPU. Try using the hardware routing as much as possible, reduce the CPU traffic to the minimum via switch ACL rules, and then fine-tune which Fasttrack connections to offload with firewall filter rules.
L3HW Feature Support
- HW - the feature is supported and offloaded to the hardware.
- CPU - the feature is supported but performed by software (CPU)
- N/A - the feature is not available together with L3HW. Layer 3 hardware offloading must be completely disabled (switch
l3-hw-offloading=no) to make this feature work. - FW - the feature requires
l3-hw-offloading=nofor a given switch port. On the switch level,l3-hw-offloading=yes.
| Feature | Support | Comments | Release |
|---|---|---|---|
| IPv4 Unicast Routing | HW | 7.1 | |
| IPv6 Unicast Routing | HW | /interface/ethernet/switch/l3hw-settings/set ipv6-hw=yes | 7.6 |
| IPv4 Multicast Routing | CPU | ||
| IPv6 Multicast Routing | CPU | ||
| ECMP | HW | Multipath routing | 7.1 |
| Blackholes | HW | /ip/route add dst-address=10.0.99.0/24 blackhole | 7.1 |
| gateway=<interface_name> | CPU/HW | /ip/route add dst-address=10.0.0.0/24 gateway=ether1 This works only for directly connected networks. Since HW does not know how to send ARP requests, | 7.1 |
| BRIDGE | HW | IP Routing from/to hardware-offloaded bridge interface. | 7.1 |
| VLAN | HW | Routing between VLAN interfaces that are created on hardware-offloaded bridge interface with vlan-filtering. | 7.1 |
| Bonding | HW | /interface/bonding | 7.1 |
| IPv4 Firewall | FW | Users must choose either HW-accelerated routing or firewall. Firewall rules get processed by the CPU. Fasttrack connections get offloaded to HW. | 7.1 |
| IPv4 NAT | FW | NAT rules applied to the offloaded Fasttrack connections get processed by HW too. | 7.1 |
| MLAG | N/A | ||
| VRF | N/A | Only the main routing table gets offloaded. | |
| VRRP | N/A | ||
| VXLAN | CPU | ||
| MTU | HW | The hardware supports up to 8 MTU profiles. | 7.1 |
| QinQ and tag-stacking | CPU | Stacked VLAN interfaces will lose HW offloading, while other VLANs created directly on the bridge interface can still use HW offloading. |
Only the devices listed in the table below support L3 HW Offloading.
L3HW Device Support
Only the devices listed in the table below support L3 HW Offloading.
CRS3xx: Switch DX3000 and DX2000 Series
The devices below are based on Marvell 98DX224S, 98DX226S, or 98DX3236 switch chip models. These devices do not support Fasttrack or NAT connection offloading.
The 98DX3255 and 98DX3257 models are exceptions, which have a feature set of the DX8000 rather than the DX3000 series.
| Model | Switch Chip | Release | IPv4 Route Prefixes1 | IPv6 Route Prefixes2 | Nexthops | ECMP paths per prefix3 |
|---|---|---|---|---|---|---|
| CRS305-1G-4S+ | 98DX3236 | 7.1 | 13312 | 3328 | 4K | 8 |
| CRS310-1G-5S-4S+ | 98DX226S | 7.1 | 13312 | 3328 | 4K | 8 |
| CRS318-1Fi-15Fr-2S | 98DX224S | 7.1 | 13312 | 3328 | 4K | 8 |
| CRS318-16P-2S+ | 98DX226S | 7.1 | 13312 | 3328 | 4K | 8 |
| CRS326-24G-2S+ | 98DX3236 | 7.1 | 13312 | 3328 | 4K | 8 |
| CRS328-24P-4S+ | 98DX3236 | 7.1 | 13312 | 3328 | 4K | 8 |
| CRS328-4C-20S-4S+ | 98DX3236 | 7.1 | 13312 | 3328 | 4K | 8 |
1 Since the total amount of routes that can be offloaded is limited, prefixes with higher netmask are preferred to be forwarded by hardware (e.g., /32, /30, /29, etc.), any other prefixes that do not fit in the HW table will be processed by the CPU. Directly connected hosts are offloaded as /32 (IPv4) or /128 (IPv6) route prefixes. The number of hosts is also limited by max-neighbor-entries in IP Settings / IPv6 Settings.
2 IPv4 and IPv6 routing tables share the same hardware memory.
3 If a route has more paths than the hardware ECMP limit (X), only the first X paths get offloaded.
CRS3xx, CRS5xx: Switch DX8000 and DX4000 Series
The devices below are based on Marvell 98DX8xxx, 98DX4xxx switch chips, or 98DX325x model.
| Model | Switch Chip | Release | IPv4 Routes 1 | IPv4 Hosts 7 | IPv6 Routes8 | IPv6 Hosts7 | Nexthops | Fasttrack connections 2,3,4 | NAT entries 2,5 |
|---|---|---|---|---|---|---|---|---|---|
| CRS317-1G-16S+ | 98DX8216 | 7.1 | 120K - 240K | 64K | 30K - 40K | 32K | 8K | 4.5K | 4K |
| CRS309-1G-8S+ | 98DX8208 | 7.1 | 16K - 36K | 16K | 4K - 6K | 8K | 8K | 4.5K | 4K |
| CRS312-4C+8XG | 98DX8212 | 7.1 | 16K - 36K | 16K | 4K - 6K | 8K | 8K | 2.25K | 2.25K |
| CRS326-24S+2Q+ | 98DX8332 | 7.1 | 16K - 36K | 16K | 4K - 6K | 8K | 8K | 2.25K | 2.25K |
| CRS354-48G-4S+2Q+ | 98DX3257 6 | 7.1 | 16K - 36K | 16K | 4K - 6K | 8K | 8K | 2.25K | 2.25K |
| CRS504-4XQ | 98DX4310 | 7.1 | 60K - 120K | 64K | 15K - 20K | 32K | 8K | 4.5K | 4K |
| CRS518-16XS-2XQ | 98DX8525 | 7.3 | 60K - 120K | 64K | 15K - 20K | 32K | 8K | 4.5K | 4K |
1 Depends on the complexity of the routing table. Whole-byte IP prefixes (/8, /16, /24, etc.) occupy less HW space than others (e.g., /22). Starting with RouterOS v7.3, when the Routing HW table gets full, only routes with longer subnet prefixes are offloaded (/30, /29, /28, etc.) while the CPU processes the shorter prefixes. In RouterOS v7.2 and before, Routing HW memory overflow led to undefined behavior. Users can fine-tune what routes to offload via routing filters (for dynamic routes) or suppressing hardware offload of static routes. IPv4 and IPv6 routing tables share the same hardware memory.
2 When the HW limit of Fasttrack or NAT entries is reached, other connections will fall back to the CPU. MikroTik's smart connection offload algorithm ensures that the connections with the most traffic are offloaded to the hardware.
3 Fasttrack connections share the same HW memory with ACL rules. Depending on the complexity, one ACL rule may occupy the memory of 3-6 Fasttrack connections.
4 MPLS shares the HW memory with Fasttrack connections. Moreover, enabling MPLS requires the allocation of the entire memory region, which could otherwise store up to 768 (0.75K) Fasttrack connections. The same applies to Bridge Port Extender. However, MPLS and BPE may use the same memory region, so enabling them both doesn't double the limitation of Fasttrack connections.
5 If a Fasttrack connection requires Network Address Translation, a hardware NAT entry is created. The hardware supports both SRCNAT and DSTNAT.
6 The switch chip has a feature set of the DX8000 series.
7 DX4000/DX8000 switch chips store directly connected hosts, IPv4 /32, and IPv6 /128 route entries in the FDB table rather than the routing table. The HW memory is shared between regular FDB L2 entries (MAC), IPv4, and IPv6 addresses. The number of hosts is also limited by max-neighbor-entries in IP Settings / IPv6 Settings.
8 IPv4 and IPv6 routing tables share the same hardware memory.
CCR2000
| Model | Switch Chip | Release | IPv4 Routes | IPv4 Hosts | IPv6 Routes | IPv6 Hosts | Nexthops | Fasttrack connections | NAT entries |
|---|---|---|---|---|---|---|---|---|---|
| CCR2116-12G-4S+ | 98DX3255 1 | 7.1 | 16K - 36K | 16K | 4K - 6K | 8K | 8K | 2.25K | 2.25K |
| CCR2216-1G-12XS-2XQ | 98DX8525 | 7.1 | 60K - 120K | 64K | 15K - 20K | 32K | 8k | 4.5K | 4K |
1 The switch chip has a feature set of the DX8000 series.