...
QoS works by identifying the type of traffic flowing through the switch and assigning it a priority level based on its requirements. The switch can then use this information to alter packet headers and prioritize the flow of traffic, ensuring that higher-priority traffic is given preferential treatment over lower-priority traffic.
The current implementation is for QoS Phase 1 - QoS Marking (introduced in RouterOS v7.10).
Planned QoS implementation phases:
- QoS Marking. QoS profile matching by ingress packet headers, then egress header alternation according to the assigned QoS profiles (introduced in RouterOS v7.10).
- QoS Enforcement. Avoid or resolve congestion based on the assigned QoS profiles and traffic shaping (introduced in RouterOS v7.13 for 98DX224S, 98DX226S, and 98DX3236 switch chips, and extended to all capable switch chips starting from RouterOS v7.15).
- QoS Policy. Assign QoS profiles via ACL rules (introduced in RouterOS v7.15).
- Extra QoS Features: WRED (Weighted Random Early Detection), ECN notification and processing (introduced in RouterOS v7.15 to capable switch chips).
- Traffic shaping.
QoS Terminology
These terms will be used throughout the article.
- .
QoS Terminology
These terms will be used throughout the article.
- QoS - Quality of Service.
- ACL -
- QoS - Quality of Service.
- ACL - Access Control List, a set of switch rules used to filter network traffic based on specified criteria.AQM - Active Queue Management.
- DSCP - Differentiated Services Code Point, a 6-bit field in the IP header used to prioritize network traffic.ECN - Explicit Congestion Notification.
- PCP - Priority Code Point, a 3-bit field in the VLAN header used to prioritize traffic within a VLAN.
- WRED - Weighted Random Early Detection.
/in/eth/sw/
a shortcut for/interface/ethernet/switch/
. The shortcut works in CLI, too.
...
Basic Configuration Example
In this example, we define just one QoS level - VoIP (IP Telephony) on top of the standard "Best Effort" class. Let's imagine that we have a CRS326-24G-2S+ device where:
- all ports are bridged and using vlan-filtering;
- sfp-sfpplus1 is a VLAN trunk connected to another switch;
- ether1-ether9 are dedicated ports for IP phones;
- ether10-ether24 are standard ports for host connection;
First, we need to define QoS profiles. Defined dscp
and pcp
values will be used in forwarded packets on egress:
Code Block | ||
---|---|---|
| ||
/interface ethernet switch qos profile
add dscp=46 name=voip pcp=5 |
Port-based QoS profile assignment on dedicated ports for IP phones applies to ingress traffic. Other Ethernet ports will use the default qos-profile
(where dscp=0
and pcp=0
):
Code Block | ||
---|---|---|
| ||
/interface ethernet switch port
set ether1 qos-profile=voip
set ether2 qos-profile=voip
set ether3 qos-profile=voip
set ether4 qos-profile=voip
set ether5 qos-profile=voip
set ether6 qos-profile=voip
set ether7 qos-profile=voip
set ether8 qos-profile=voip
set ether9 qos-profile=voip |
The trunk port receives both types of QoS traffic. We need to create VLAN priority mapping with the QoS profile and enable qos-trust-l2
to differentiate them:
Code Block | ||
---|---|---|
| ||
/interface ethernet switch qos map vlan
add pcp=5 qos-profile=voip
/interface ethernet switch port
set sfp-sfpplus1 qos-trust-l2=trust |
Finally, enable QoS hardware offloading for the above settings to start working:
Code Block | ||
---|---|---|
| ||
/interface ethernet switch
set switch1 qos-hw-offloading=yes |
It is possible to verify the port QoS profile and Layer2, and Layer3 trust settings with print qos
command:
Code Block | ||
---|---|---|
| ||
[admin@MikroTik] /interface/ethernet/switch/port print qos
Columns: NAME, SWITCH, QOS-PROFILE, QOS-MAP, QOS-TRUST-L2, QOS-TRUST-L3
# NAME SWITCH QOS-PROFILE QOS-MAP QOS-TRUST-L2 QOS-TRUST-L3
0 ether1 switch1 voip default ignore ignore
1 ether2 switch1 voip default ignore ignore
2 ether3 switch1 voip default ignore ignore
3 ether4 switch1 voip default ignore ignore
4 ether5 switch1 voip default ignore ignore
5 ether6 switch1 voip default ignore ignore
6 ether7 switch1 voip default ignore ignore
7 ether8 switch1 voip default ignore ignore
8 ether9 switch1 voip default ignore ignore
9 ether10 switch1 default default ignore ignore
10 ether11 switch1 default default ignore ignore
11 ether12 switch1 default default ignore ignore
12 ether13 switch1 default default ignore ignore
13 ether14 switch1 default default ignore |
1 Due to hardware limitations, some switch chip models may break traffic flow while accessing QoS port/queue usage data.
2 The device gathers max queue fill statistics instead of displaying the current usage values. Use the reset-counters command to reset those stats.
Applications and Usage Examples
Basic Configuration Example
In this example, we define just one QoS level - VoIP (IP Telephony) on top of the standard "Best Effort" class. Let's imagine that we have a CRS326-24G-2S+ device where:
- all ports are bridged and using vlan-filtering;
- sfp-sfpplus1 is a VLAN trunk connected to another switch;
- ether1-ether9 are dedicated ports for IP phones;
- ether10-ether24 are standard ports for host connection;
First, we need to define QoS profiles. Defined dscp
and pcp
values will be used in forwarded packets on egress:
Code Block | ||
---|---|---|
| ||
/interface ethernet switch qos profile
add dscp=46 name=voip pcp=5 traffic-class=5 |
Port-based QoS profile assignment on dedicated ports for IP phones applies to ingress traffic. Other Ethernet ports will use the default qos-profile
(where dscp=0
and pcp=0
):
Code Block | ||
---|---|---|
| ||
/interface ethernet switch qos port
set ether1 profile=voip
set ether2 profile=voip
set ether3 profile=voip
set ether4 profile=voip
set ether5 profile=voip
set ether6 profile=voip
set ether7 profile=voip
set ether8 profile=voip
set ether9 profile=voip |
Info |
---|
Starting from RouterOS v7.13, QoS port settings moved from |
The trunk port receives both types of QoS traffic. We need to create VLAN priority mapping with the QoS profile and enable qos-trust-l2
to differentiate them:
Code Block | ||
---|---|---|
| ||
/interface ethernet switch qos map vlan
add pcp=5 profile=voip
/interface ethernet switch port
set sfp-sfpplus1 trust-l2=trust |
Finally, enable QoS hardware offloading for the above settings to start working:
Code Block | ||
---|---|---|
| ||
/interface ethernet switch
set switch1 qos-hw-offloading=yes |
It is possible to verify the port QoS settings with print
command:
Code Block | ||
---|---|---|
| ||
[admin@MikroTik] /interface/ethernet/switch/qos/port print Columns: NAME, SWITCH, PROFILE, MAP, TRUST-L2, TRUST-L3 # NAME SWITCH PROFILE MAP TRUST-L2 TRUST-L3 TX-MANAGER 0 ether1 switch1 voip default ignore ignore default 1 ether2 switch1 voip default ignore ignore default 2 ether3 switch1 voip default ignore ignore default 3 ether4 14 ether15 switch1 default voip default ignore ignore default 4 ether5 15 ether16 switch1 default voip default ignore ignore default ignore 5 ether6 16 ether17 switch1 voip switch1 default ignore ignore default 6 ether7 switch1 voip ignore default ignore ignore default 17 7 ether8 ether18 switch1 voip default ignore ignore default 8 ether9 switch1 voip ignore default ignore ignore default 9 ether10 switch1 default default ignore ignore default 10 ether11 switch1 default default ignore ignore default 11 ether12 switch1 default default ignore ignore default 12 ether13 switch1 default default ignore ignore default 13 ether14 switch1 default default ignore ignore default 14 ether15 switch1 default default ignore ignore default 15 ether16 switch1 default default ignore ignore default 16 ether17 switch1 default default ignore ignore default 17 ether18 switch1 default default ignore ignore default 18 ether19 switch1 default default ignore ignore default 19 ether20 switch1 default default ignore ignore default 20 ether21 switch1 default default ignore ignore default 21 ether22 switch1 default default ignore ignore default 22 ether23 switch1 default default ignore ignore default 23 ether24 switch1 default default ignore ignore default 24 sfp-sfpplus1 switch1 default default trust ignore default 25 sfp-sfpplus2 switch1 default default ignore ignore default 26 switch1-cpu switch1 |
Now incoming packets on ports ether1-ether9 are marked with a Priority Code Point (PCP) value of 5 and a Differentiated Services Code Point (DSCP) value of 46, and incoming packets on ports ether10-ether24 are marked with PCP and DSCP values of 0. When packets are incoming to sfp-sfpplus1 port, any packets with a PCP value of 5 or higher will retain their PCP value of 5 and DSCP value of 46, while all other packets will be marked with PCP and DSCP values of 0.
Dante
Starting from RouterOS v7.15, all MikroTik QoS-Capable devices comply with Dante.
Dante hardware use the following DSCP / Diffserv priority values for traffic prioritization.
Dante Priority | Usage | DSCP Label | DSCP Value |
---|---|---|---|
High | Time critical PTP events | CS7 | 56 |
Medium | Audio, PTP | EF | 46 |
Low | (reserved) | CS1 | 8 |
None | Other traffic | BE | 0 |
The example assumes that the switch is using its default configuration, which includes a default "bridge" interface and all Ethernet interfaces added as bridge ports, and any of these interfaces could be used for Dante.
First, create QoS Profiles to match Dante traffic classes, there is already a pre-existing "default" profile that corresponds to Dante's None priority.
Code Block | ||
---|---|---|
| ||
/interface/ethernet/switch/qos/profile
add name=dante-ptp dscp=56 pcp=7 traffic-class=7
add name=dante-audio dscp=46 pcp=5 traffic-class=5
add name=dante-low dscp=8 pcp=1 traffic-class=0 |
Then, create a QoS mapping to match QoS profiles based on DSCP values.
Code Block | ||
---|---|---|
| ||
/interface/ethernet/switch/qos/map/ip
add dscp=56 profile=dante-ptp
add dscp=46 profile=dante-audio
add dscp=8 profile=dante-low |
Configure hardware queues to enforce QoS on Dante traffic.
Code Block | ||
---|---|---|
| ||
/interface/ethernet/switch/qos/tx-manager/queue
set [find where traffic-class>=2] schedule=strict-priority
set [find where traffic-class<2] schedule=low-priority-group weight=1 |
Dante's High and Medium priority traffic is scheduled in strict order. The devices transmits time critical PTP packets until queue7 gets empty, then proceeds with audio (queue5). Low and other traffic gets transmitted only when PTP and audio queues are empty. Since Dante does not define priority order between Low and Other traffic (usually, CS1 has lower priority than Best Effort), and the Low traffic class is reserved for future use anyway, we treat both traffic types equally by putting both into the same group with the same weight. Feel free to change the CS1/BE traffic scheduling according to the requirements if some Dante hardware in your network use the low-priority traffic class.
The next step is enable trust mode for incomming Layer3 packets (IP DSCP field):
Code Block | ||
---|---|---|
| ||
/interface/ethernet/switch/qos/port
set [find] trust-l3=keep |
Finally, enable QoS hardware offloading for the above settings to start working:
Code Block | ||
---|---|---|
| ||
/interface ethernet switch
set switch1 qos-hw-offloading=yes |
When using Dante in multicast mode, it is beneficial to enable IGMP snooping on the switch. This feature directs traffic only to ports with subscribed devices, preventing unnecessary flooding. Additionally, enabling an IGMP querier (if not already enabled on another device in the same LAN), adjusting query intervals, and activating fast-leave can further optimize multicast performance.
Code Block | ||
---|---|---|
| ||
/interface/bridge
set [find name=bridge] igmp-snooping=yes multicast-querier=yes query-interval=60s
/interface/bridge/port
set [find] fast-leave=yes |
QoS Marking
Understanding Map ranges
In order to avoid defining all possible PCP and DSCP mappings, RouterOS allows setting the minimal PCP and DSCP values for QoS Profile mapping.
In the following example, PCP values 0-2 use the default QoS profile, 3-4 - streaming, 5 - voip, and 6-7 - control.
Code Block | ||
---|---|---|
| ||
/interface ethernet switch qos map vlan
add pcp=3 profile=streaming
add pcp=5 profile=voip
add pcp=6 profile=control |
Since the pcp
parameter identifies the minimum value, all packets with a higher PCP value match too. If such behavior is undesired, add mapping for higher values. The next example sets voip profile for pcp=5 only. Packets with PCP values 6 or 7 are reset back to the default profile.
Code Block | ||
---|---|---|
| ||
/interface ethernet switch qos map vlan
add pcp=5 profile=voip
add pcp=6 profile=default |
Understanding Port, Profile, and Map relation
Each switch port has Layer2 and Layer3 trust settings that will change how ingress packets are classified into QoS profiles and what PCP and DSCP values will be used. Below are tables that describe all possible options:
qos-trust-l2 | qos-trust-l3 | Behavior |
---|---|---|
ignore | ignore | The port is considered untrusted. Both headers are ignored, and the port's profile is forced to all ingress packets. This is the default setting. |
ignore | trust | Trust the Layer 3 header. Use the DSCP field from the IP header of ingress packets for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the lookup fails (no QoS profiles are mapped to the given DSCP value), the default QoS profile is used (not the switch port's QoS profile). The switch port's profile field is used only for non-IP traffic. |
ignore | keep | Trust the Layer 3 header. Use the DSCP field from the IP header of ingress packets for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the lookup fails, the default QoS profile is used. The switch port's profile field is used only for non-IP traffic. If the forwarded/routed packet is VLAN-tagged, its PCP value is set from the selected QoS profile. However, the original DSCP value of the packet is kept intact. |
trust | ignore | Trust the Layer 2 header, but ignore L3. If an ingress packet is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the lookup fails (no QoS profiles are mapped to the given PCP value), the default QoS profile is used. The switch port's profile field is used only for untagged traffic. |
trust | trust | Trust both headers, but Layer 3 has higher precedence. In the case of an IP packet, use the DSCP field for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the DSCP-to-QoS lookup fails, use the default profile. If the packet is not an IP packet but is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the VLAN-to-QoS lookup fails, use the default QoS profile. Non-IP untagged packets use the switch port's profile. |
trust | keep | The same as trust+trust, but the original DSCP value is preserved in forwarded/routed packets. |
keep | ignore | Trust the Layer 2 header but ignore L3. If an ingress packet is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the lookup fails (no QoS profiles are mapped to the given PCP value), the default QoS profile is used. The switch port's profile field is used only for untagged traffic. If the packet is VLAN-tagged on both ingress and egress, the original PCP value is kept. |
keep | trust | Trust both headers, but Layer 3 has higher precedence. In the case of an IP packet, use the DSCP field for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the DSCP-to-QoS lookup fails, use the default profile. If the packet is not an IP packet but is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the VLAN-to-QoS lookup fails, use the default QoS profile. Non-IP untagged packets use the switch port's profile. If the packet is VLAN-tagged on both ingress and egress, the original PCP value is kept. The DSCP value in forwarded/routed packets is set from the selected QoS profile. |
keep | keep | Trust both headers, but Layer 3 has higher precedence. In the case of an IP packet, use the DSCP field for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the DSCP-to-QoS lookup fails, use the default profile. If the packet is not an IP packet but is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the VLAN-to-QoS lookup fails, use the default QoS profile. Non-IP untagged packets use the switch port's profile. Keep both the original PCP and/or DSCP values intact in cases of VLAN-tagged and/or IP packets, respectively. |
...
1 applies only when ingress traffic is untagged, but the egress needs to be VLAN-tagged.
...
Starting from RouterOS v7.15, it is possible to assign QoS profiles via Switch Rules (ACL).
...
The following example assigns a QoS profile based on the source MAC address.
Code Block | ||
---|---|---|
| ||
/interface ethernet switch rule
add new-qos-profile=stream ports=ether1,ether2 src-mac-address=00:01:02:00:00:00/FF:FF:FF:00:00:00 switch=switch1
add new-qos-profile=voip ports=ether1,ether2 src-mac-address=04:05:06:00:00:00/FF:FF:FF:00:00:00 switch=switch1 |
QoS Enforcement
Hardware Queues
Each switch port has eight hardware transmission (tx) queues (queue0..queue7). Each queue corresponds to a traffic class (tc0..tc7) set by a QoS profile. Each ingress packet gets assigned to a QoS profile, which, in turn, determines the traffic class for tx queue selection on the egress port.
Hardware queues are of variable size - set by the Transmission Manager. Moreover, multiple ports and/or queues can share resources with each other (so-called Shared Buffers). For example, a device with 25 ports has memory (buffers) to queue 1200 packets in total. If we split the resources equally, each port gets 48 exclusive buffers with a maximum of 6 packets per queue (48/8) - which is usually insufficient to absorb even a short burst of traffic. However, choosing to share 50% of the buffers leaves each port with 24 exclusive buffers (3 per queue), but at the same time, a single queue can grow up to 603 buffers (3 exclusive + 600 shared).
RouterOS allows enabling/disabling the shared pool for each queue individually - for example, to prevent low-priority traffic from consuming the entire hardware memory. In addition, port buffer limits may prevent a single low-speed port from consuming the entire shared pool. See QoS Settings and Transmission Manager for details.
Info |
---|
The default, best-effort (PCP=0, DSCP=0) traffic class is 1, while the lowest priority (PCP=1) has traffic class 0. |
Hardware Resources
The hardware (switch chips) has limited resources (memory). There are two main hardware resources that are relevant to QoS:
- Packet descriptors - contain packet control information (target port, header alternation, etc).
- Data buffers - memory chunks containing the actual payload. Buffer size depends on the switch chip model. Usually - 256 bytes.
One packet descriptor may use multiple buffers (depending on the payload size); buffers may be shared by multiple descriptors - in cases of multicast/broadcast. If the hardware does not have enough free descriptors or buffers, the packet gets dropped (tail-drop).
Hardware resources can be limited per destination type (multicast/unicast), per port, and per each tx queue. If any limits are reached, no more packets can be enqueued for transmission, and further packets get dropped.
RouterOS obscures low-level hardware information, allowing to set resource limits either in terms of packets or a percentage of the total amount. RouterOS automatically calculates the required hardware descriptor and buffer count based on the user-specified packet limit and port's MTU. Moreover, RouterOS comes with preconfigured hardware resources, so there is no need to do a manual configuration in common QoS environments.
Warning |
---|
Changing any hardware resource allocation parameter in runtime results in a temporary device halt when no packets can be enqueued nor transmitted. Temporary packet loss is expected while the device is forwarding traffic. |
Resource Saving
Since reallocating hardware resources in runtime is not an option, RouterOS cannot automatically free queue buffers reserved for inactive ports. Those buffers remain unused. However, if the user clearly knows that the specific ports will never be used (e.g., stay physically disconnected), the respective queue resources can be manually freed by introducing the "offline" tx-manager with minimum resources:
Code Block | ||
---|---|---|
| ||
/interface/ethernet/switch/qos/tx-manager/
add name=offline comment="use this for always disconnected ports"
/interface/ethernet/switch/qos/tx-manager/queue/
set [find where tx-manager=offline] queue-buffers=1 use-shared-buffers=no
/interface/ethernet/switch/qos/port
set [find where !running] tx-manager=offline |
Traffic Prioritization
The hardware provides two types of traffic transmission prioritization:
- Strict Priority - traffic from higher queues is always transmitted first;
- Weighted Priority Groups - multiple queues participate in packet transmission scheduling at the same time.
Strict priority queues are straightforward. If the highest priority queue (Q7) has packets, those are transmitted first. When Q7 is empty, packets from Q6 get transmitted, and so on. The packets from the lowest priority queue (Q0) are transmitted only if all other queues are empty.
...
Weighted Priority Groups schedule traffic for transmission from multiple queues (group members) in a weighted round-robin manner. A queue's weight sets the number of packets transmitted from the queue in each round. For example, if Q2, Q1, and Q0 are the group members, and their weights are 3, 2, and 1, respectively, the scheduler transmits 3 packets from Q2, 2 - from Q1, and 1 - from Q0. The actual Tx order is "Q2, Q1, Q0, Q2, Q1, Q2" - for even fairer scheduling.
There are two hardware groups: low-priority-group
and high-priority-group
. There is a strict priority ordering between the two groups: the low-priority-group is transmitting only when all queues in the high-priority-group are empty. However, it is possible to use only one group for all queues.
The default (built-in) RouterOS queue setup is listed below. Q3-Q5 share the bandwidth within the high-priority group, where packets are transmitted while Q6 and Q7 are empty. Q0-Q2 are the members of the low-priority-group, where packets are transmitted while Q3-Q7 are empty.
Code Block | ||
---|---|---|
| ||
[admin@MikroTik] /interface/ethernet/switch/qos/tx-manager/queue> print
Columns: TX-MANAGER, TRAFFIC-CLASS, SCHEDULE, WEIGHT, QUEUE-BUFFERS, USE-SHARED-BUFFERS
# TX-MANAGER TRAFFIC-CLASS SCHEDULE WEIGHT QUEUE-BUFFERS USE-SHARED-BUFFERS
0 default 0 low-priority-group 1 auto no
1 default 1 low-priority-group 2 auto yes
2 default 2 low-priority-group 3 auto yes
3 default 3 high-priority-group 3 auto yes
4 default 4 high-priority-group 4 auto yes
5 default 5 high-priority-group 5 auto yes
6 default 6 strict-priority auto yes
7 default 7 strict-priority auto yes |
Info |
---|
It is recommended that all group members are adjacent to each other. |
Active Queue Management (AQM)
...
WRED is a per-queue congestion control mechanism that signals congestion events to the end-points by dropping packets. WRED relies on the existence of rate throttling mechanisms in the end-points that react to packet loss, such as TCP/IP. WRED uses a randomized packet drop algorithm in an attempt to anticipate congestion events and respond to them by throttling traffic rates before the congestion actually happens. The randomness property of WRED prevents throughput collapse related to the global synchronization of TCP flows.
WRED can be enabled/disabled per each queue in each Tx Manager. Disable WRED for lossless traffic! Also, there is no reason to enable WRED on high-speed ports where congestion should not happen in the first place.
The behavior is controlled via WRED margins. WRED margin is the distance to the queue/pool buffer limit (cap) - where a random packet drop begins. A different margin can be applied to queues that use or don't use shared buffers. A queue that uses a shared pool may set a bigger WRED margin due to a higher overall cap (queue buffers + shared pool). RouterOS automatically chooses the actual WRED margin values according to queue or shared pool capacities. The user may shift the margins in one way or another via QoS Settings.
For example, if queue1-packet-cap=96, and WRED margin is 32 (assuming use-shared-buffers=no), then:
- first 64 packets are always enqueued (96 - 32 = 64).
- WRED kicks in starting from 65th packet; the chance of 65th packet to be dropped is 1/32 or roughly 3%; the formula: (65 - 64) / 32;
- the drop chance of 72th packet is 25%: (72 - 64) / 32 = 8/32 = 1/4;
- half of the newly enquining packets are dropped when the queue fill level reaches 80 packets: (80 - 64) / 32 = 16/32 = 1/2;
- 75% of the packets are dropped at the fill level 88: (88 - 64) / 32 = 24/32 = 3/4.
When WRED is enabled (wred=yes), the queue cap is rounded to the applied WRED margin. In the above example, setting wred-queue-margin=64 raises queue1-packet-cap to 128. That, in turn, may lower the resources available to other queues, such as shared buffers. A much safer option is to raise wred-shared-queue-margin which may reduce the shared buffers available for the affected queue but not the shared pool itself. For example, if: "wred-shared-queue-margin=256, use-shared-buffers=yes, wred=yes, shared-pool-index=0, queue2-packet-cap=30, and shared-pool0-packet-cap=900", queue2 can use up to 768 buffers (30+900=930, rounded down by the scale of 256), and WRED starting at 512, while the other ports/queues still may use the remaining 162 buffers (30+900-768) of the shared pool.
Choosing a WRED margin value is a tradeoff between congestion anticipation and burst absorption. Setting a higher WRED margin may lead to earlier traffic rate throttling and, therefore, resolve congestion. On the other hand, a high margin leads to packet drops in limited traffic bursts that could be absorbed by the queue buffers and transformed losslessly if WRED didn't kick in. For instance, initiating a remote database connection usually starts with heavier traffic ("packet burst") at the initialization phase; then, the traffic rate drops down to a "reasonable" level. Any packet drop during the initialization phase leads to nothing but a slower database connection due to the need for retransmission. Hence, lowering the WRED margin or entirely disabling WRED on such traffic is advised. The opposite case is video streaming. Early congestion detection helps select a comfortable streaming rate without losing too much bandwidth on retransmission or/and "overshooting" by sacrificing the quality level by too much.
Info |
---|
Use Switch Rules (ACL) or other QoS Marking techniques to differentiate traffic and put packets into queues with desired WRED settings. |
The following script only applies WRED to TCP/IP traffic by redirecting it to queue2. UDP and other packets are left in queue1 - since their end-points usually cannot respond to early drops. Queue1 and queue2 are scheduled equally - without prioritizing one queue over another.
Code Block | ||
---|---|---|
| ||
/interface/ethernet/switch/qos/profile
add name=tcp-wred traffic-class=2 pcp=0 dscp=0
# move TCP traffic to queue2
/interface/ethernet/switch/rule
add new-qos-profile=tcp-wred ports=ether1,ether2,ether3,ether4 protocol=tcp switch=switch1
# set the same scheduling priority (weight) between queue1 and queue2
# apply WRED only to queue2 - TCP traffic
/interface/ethernet/switch/qos/tx-manager/queue/
set [find where traffic-class=1] weight=2 schedule=low-priority-group use-shared-buffers=yes shared-pool-index=0 wred=no
set [find where traffic-class=2] weight=2 schedule=low-priority-group use-shared-buffers=yes shared-pool-index=0 wred=yes |
...
Some switch chips can perform ECN marking of IP packets on the hardware level, according to RFC 3168. Hardware ECN marking is based on the WRED mechanism, but instead of dropping IP packets, they are marked with CE (Congestion Experienced, binary 11) in the ECN field (two least significant bits in IPv4/TOS or IPv6/TrafficClass octet). Only ECN-Capable IP packets may be marked - those with the ECN field value of ECT(1) or ECT(0) (binary 01 or 10, respectively). Not ECN-Capable Transport packets (ECN=00) never get marked. If a packet already has the CE mark (ECN=11), it never gets cleared, even if the device does not experience congestion.
Set ecn=yes in Tx Manager to enable ECN marking. The per-queue ECN setting is unavailable due to hardware limitations. ECN and WRED share the same queue fill threshold: wred-shared-queue-margin (see QoS Settings).
Warning |
---|
ECN marking mechanism requires the respective Tx queues to use shared buffers (use-shared-buffers=yes). |
The packet receives the CE mark if all conditions below are met:
- The packet is either IPv4 or IPv6.
- The ECN field value in IP header is either ECT(1) or ECT(0).
- Egress port's Tx Manager has ecn=yes.
- The assigned Tx Queue uses shared buffers (use-shared-buffers=yes).
- The Tx Queue detects congestion via WRED margins.
Property Reference
Switch settings
...
Switch QoS settings (in addition to the existing ones).
...
Port settings
...
Info |
---|
Starting from RouterOS v7.13, QoS port settings moved from |
Switch port QoS settings. Assigns a QoS profile to ingress packets on the given port. The assigned profile can be changed via match rules if the port is considered trusted.
By default, ports are untrusted and receive the default QoS profile (Best-Effort, PCP=0, DSCP=0), where priority fields are cleared from the egress packets.
...
Whenever to trust the Layer 2 headers of the incoming packets (802.1p PCP field):
- ignore - ignore L2 header; use the port's profile value for all incoming packets;
- trust - use PCP field of VLAN-tagged packets for QoS profile lookup in map. Untagged packets use the port's profile value. Forwarded VLAN or priority-tagged packets receive the PCP value from the selected QoS profile (overwriting the original value).
- keep - trust but keep the original PCP value in forwarded packets.
...
Whenever to trust the Layer 3 headers of the incoming packets (IP DSCP field):
- ignore - ignore L3 header; use either L2 header or the port's profile (depends on trust-l2).
- trust - use DSCP field of IP packets for QoS profile lookup in map. Forwarded/routed IP packets receive the DSCP value from the selected QoS profile (overwriting the original value).
- keep - trust but keep the original DSCP value in forwarded/routed packets.
...
The name of the Transmission Manager that is responsible for enqueuing and transmitting packets from the given port (see /in/eth/sw/qos/tx-manager
).
Info |
---|
L3 trust mode has higher precedence than L2 unless trust-l3=ignore or the packet does not have an IP header. |
Info |
---|
Forwarded/routed packets obtain priority field values (PCP, DSCP) from the selected QoS profile, overwriting the original values unless the respective trust mode is set to keep. |
Commands.
...
Port Stats
Code Block | ||||
---|---|---|---|---|
| ||||
[admin@Mikrotik] /interface/ethernet/switch/qos/port> print stats where name=ether2
name: ether2
tx-packet: 2 887
tx-byte: 3 938 897
drop-packet: 1 799
drop-byte: 2 526 144
tx-queue0-packet: 50
tx-queue1-packet: 1 871
tx-queue3-packet: 774
tx-queue5-packet: 192
tx-queue0-byte: 3 924
tx-queue1-byte: 2 468 585
tx-queue3-byte: 1 174 932
tx-queue5-byte: 291 456
drop-queue1-packet: 1 799
drop-queue1-byte: 2 526 144 |
...
tx-queue0-packet .. tx-queue7-packet
...
tx-queue0-byte .. tx-queue7-byte
...
drop-queue0-packet .. drop-queue7-packet
...
drop-queue0-byte .. drop-queue7-byte
...
Port Resources/Usage
Warning |
---|
Due to hardware limitations, some switch chip models may break traffic flow while accessing QoS port |
Code Block | ||||
---|---|---|---|---|
| ||||
[admin@crs326] /interface/ethernet/switch/qos/port> print usage where name=ether2 18 ether19 switch1 default default ignore ignore name: ether2 19 ether20 switch1 default default ignore ignore packet-cap: 136 20 ether21 switch1 default default ignore packet-use: ignore 5 21 ether22 switch1 default byte-cap: 35 840 default ignore ignore byte-use: 9 472 22 ether23 queue0-packet-cap: switch1 130 default queue0-packet-use: default ignore 1 queue1-packet-cap: ignore 5 23 ether24 queue1-packet-use: switch1 default 4 queue3-packet-cap: default ignore 65 queue3-packet-use: ignore 2 24 sfp-sfpplus1 switch1 queue0-byte-cap: 24 576 default default queue0-byte-use:trust 256 ignore queue1-byte-cap: 7 680 25 sfp-sfpplus2 switch1 default queue1-byte-use: default 6 144 ignore queue3-byte-cap: 14ignore 080 26 queue3-byte-use: 3 072 |
...
switch1-cpu switch1 |
Now incoming packets on ports ether1-ether9 are marked with a Priority Code Point (PCP) value of 5 and a Differentiated Services Code Point (DSCP) value of 46, and incoming packets on ports ether10-ether24 are marked with PCP and DSCP values of 0. When packets are incoming to sfp-sfpplus1 port, any packets with a PCP value of 5 or higher will retain their PCP value of 5 and DSCP value of 46, while all other packets will be marked with PCP and DSCP values of 0.
Understanding Map ranges
To avoid the need to define all possible PCP and DSCP mappings, RouterOS allows setting the minimal PCP and DSCP values for QoS Profile mapping.
In the following example, PCP values 0-2 use the default QoS profile, 3-4 - streaming, 5 - voip, and 6-7 - control.
Code Block | ||
---|---|---|
| ||
/interface ethernet switch qos map vlan
add pcp=3 qos-profile=streaming
add pcp=5 qos-profile=voip
add pcp=6 qos-profile=control |
Since the pcp
parameter identifies the minimum value, all packets with a higher PCP value match too. If such behavior is undesired, add mapping for higher values. The next example sets voip profile for pcp=5
only. Packets with PCP values 6 or 7 are reset back to the default profile.
Code Block | ||
---|---|---|
| ||
/interface ethernet switch qos map vlan
add pcp=5 qos-profile=voip
add pcp=6 qos-profile=default |
Understanding Port, Profile and Map relation
Each switch port has Layer2 and Layer3 trust settings, that will change how ingress packets are classified into QoS profiles and what PCP and DSCP values will be used. Below are tables that describe all possible options:
qos-trust-l2 | qos-trust-l3 | Behavior |
---|---|---|
ignore | ignore | The port is considered untrusted. Both headers are ignored, and the port's qos-profile is forced to all ingress packets. This is the default setting. |
ignore | trust | Trust the Layer 3 header. Use the DSCP field from the IP header of ingress packets for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the lookup fails (no QoS profiles are mapped to the given DSCP value), the default profile is used (not the switch port's qos-profile). The switch port's qos-profile field is used only for non-IP traffic. |
ignore | keep | Trust the Layer 3 header. Use the DSCP field from the IP header of ingress packets for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the lookup fails, the default profile is used. The switch port's qos-profile field is used only for non-IP traffic. If the forwarded/routed packet is VLAN-tagged, its PCP value is set from the selected QoS profile. However, the original DSCP value of the packet is kept intact. |
trust | ignore | Trust the Layer 2 header but ignore L3. If an ingress packet is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the lookup fails (no QoS profiles are mapped to the given PCP value), the default profile is used. The switch port's qos-profile field is used only for untagged traffic. |
trust | trust | Trust both headers, but Layer 3 has higher precedence. In the case of an IP packet, use the DSCP field for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the DSCP-to-QoS lookup fails, use the default profile. If the packet is not an IP packet but is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the VLAN-to-QoS lookup fails, use the default profile. Non-IP untagged packets use the switch port's qos-profile. |
trust | keep | The same as trust+trust, but the original DSCP value is preserved in forwarded/routed packets. |
keep | ignore | Trust the Layer 2 header but ignore L3. If an ingress packet is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the lookup fails (no QoS profiles are mapped to the given PCP value), the default profile is used. The switch port's qos-profile field is used only for untagged traffic. If the packet is VLAN-tagged on both ingress and egress, the original PCP value is kept. |
keep | trust | Trust both headers, but Layer 3 has higher precedence. In the case of an IP packet, use the DSCP field for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the DSCP-to-QoS lookup fails, use the default profile. If the packet is not an IP packet but is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the VLAN-to-QoS lookup fails, use the default profile. Non-IP untagged packets use the switch port's qos-profile. If the packet is VLAN-tagged on both ingress and egress, the original PCP value is kept. The DSCP value in forwarded/routed packets is set from the selected QoS profile. |
keep | keep | Trust both headers, but Layer 3 has higher precedence. In the case of an IP packet, use the DSCP field for QoS profile lookup (see /in/eth/sw/qos/map/ip ). If the DSCP-to-QoS lookup fails, use the default profile. If the packet is not an IP packet but is VLAN-tagged, use the PCP field from the VLAN header for QoS profile lookup (see /in/eth/sw/qos/map/vlan ). If the VLAN-to-QoS lookup fails, use the default profile. Non-IP untagged packets use the switch port's qos-profile. Keep both the original PCP and/or DSCP values intact in cases of VLAN-tagged and/or IP packets, respectively. |
Port settings | The selected QoS profile and the source for PCP / DSCP field values in forwarded/routed packets | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
qos-trust-l2 | qos-trust-l3 | VLAN-Tagged IP | Untagged IP | VLAN-Tagged Non-IP | Untagged Non-IP | ||||||||
QoS Profile | PCP | DSCP | QoS Profile | PCP 1 | DSCP | QoS Profile | PCP | DSCP | QoS Profile | PCP 1 | DSCP | ||
ignore | ignore | qos-profile | qos-profile | qos-profile | qos-profile | qos-profile | qos-profile | qos-profile | qos-profile | - | qos-profile | qos-profile | - |
ignore | trust | map/ip | map/ip | map/ip | map/ip | map/ip | map/ip | qos-profile | qos-profile | - | qos-profile | qos-profile | - |
ignore | keep | map/ip | map/ip | original | map/ip | map/ip | original | qos-profile | qos-profile | - | qos-profile | qos-profile | - |
trust | ignore | map/vlan | map/vlan | map/vlan | qos-profile | qos-profile | qos-profile | map/vlan | map/vlan | - | qos-profile | qos-profile | - |
trust | trust | map/ip | map/ip | map/ip | map/ip | map/ip | map/ip | map/vlan | map/vlan | - | qos-profile | qos-profile | - |
trust | keep | map/ip | map/ip | original | map/ip | map/ip | original | map/vlan | map/vlan | - | qos-profile | qos-profile | - |
keep | ignore | map/vlan | original | map/vlan | qos-profile | qos-profile | qos-profile | map/vlan | original | - | qos-profile | qos-profile | - |
keep | trust | map/ip | original | map/ip | map/ip | qos-profile | map/ip | map/vlan | original | - | qos-profile | qos-profile | - |
keep | keep | map/ip | original | original | map/ip | qos-profile | original | map/vlan | original | - | qos-profile | qos-profile | - |
1 applies only when ingress traffic is untagged, but the egress needs to be VLAN-tagged.
Property Reference
Switch settings
Sub-menu:/interface/ethernet/switch
Switch QoS settings (in addition to the existing ones).
Property | Description |
---|---|
qos-hw-offloading (yes | no; Default: no) | Allows enabling QoS for the given switch chip (if the latter supports QoS). |
Port settings
Sub-menu:/interface/ethernet/switch/port
Switch port settings (in addition to the existing ones). Assigns a QoS profile to ingress packets on the given port. The assigned profile can be changed via match rules if the port is considered trusted.
By default, ports are untrusted and receive the default QoS profile (Best-Effort, PCP=0, DSCP=0), where priority fields are cleared from the egress packets.
Property | Description |
---|---|
qos-map (name; Default: default) | Allows user-defined priority-to-profile mapping in the case of a trusted port or host (see /in/eth/sw/qos/map ). |
qos-profile (name; Default: default) | The name of the QoS profile to assign to the ingress packets by default (see /in/eth/sw/qos/profile ). |
qos-trust-l2 (ignore | trust | keep; Default: ignore) | Whenever to trust the Layer 2 headers of the incoming packets (802.1p PCP field):
|
qos-trust-l3 (ignore | trust | keep; Default: ignore) | Whenever to trust the Layer 3 headers of the incoming packets (IP DSCP field):
|
Info |
---|
L3 trust mode has higher precedence than L2 unless |
Info |
---|
Forwarded/routed packets obtain priority field values (PCP, DSCP) from the selected QoS profile, overwriting the original values, unless the respective trust mode is set to keep. |
Commands (in addition to the existing ones).
Command | Description |
---|---|
print qos | Prints all QoS-related information in a human-friendly format. |
QoS Settings
Sub-menu:/interface/ethernet/switch/qos
Almost the entire QoS HW configuration is located under /in/eth/sw/qos
. Such an approach allows storing all QoS-related configuration items in one place, easy monitoring and exporting (/in/eth/sw/qos/export
).
All QoS entries have two major flags:
- H - Hardware-offloaded.
- I - Inactive.
QoS Profile
Sub-menu:/interface/ethernet/switch/qos/profile
QoS profiles determine priority field values (PCP, DSCP) for the forwarded/routed packets. Congestion avoidance/resolution is based on QoS profiles. Each packet gets a QoS profile assigned based on the ingress switch port QoS settings (see /in/eth/sw/port
).
Property | Description |
---|---|
dscp (integer: 0..63; Default: 0) | IPv4/IPv6 DSCP field value for the egress packets assigned to the QoS profile. |
name (string; Default: ) | The user-defined name of the QoS profile. |
pcp (integer: 0..7; Default: 0) | VLAN priority value (IEEE 802.1q PCP - Priority Code Point). Used only if the egress packets assigned to the QoS profile are VLAN-tagged (have the 802.1q header). The value can be further altered via the QoS Egress Map. |
QoS Mapping
Sub-menu:/interface/ethernet/switch/qos/map
Priority-to-profile mapping table(-s) for trusted packets. All switch chips have one built-in map - default. In addition, some models allow the user to define custom mapping tables and assign different maps to various switch ports via the qos-map property:
- devices based on Marvell Prestera 98DX224S, 98DX226S, or 98DX3236 switch chip models support only one map - default.
- devices based on Marvell Prestera 98DX8xxx, 98DX4xxx switch chips, or 98DX325x model devices support up to 12 maps (the default + 11 user-defined).
Property | Description |
---|---|
name (string; Default: ) | The user-defined name of the mapping table. |
VLAN Map
Sub-menu:/interface/ethernet/switch/qos/map/vlan
Matches VLAN priorities (802.1p PCP field) to QoS profiles. By default, all values are matched to the default QoS profile.
Property | Description |
---|---|
qos-map (name; Default: default) | The name of the mapping table. |
qos-profile (name; Default: ) | The name of the QoS profile to assign to the matched packets. |
pcp (integer: 0..7; Default: 0) | MinimumVLAN priority (PCP) value for the lookup. |
DSCP Map
Sub-menu:/interface/ethernet/switch/qos/map/ip
Matches DSCP values to QoS profiles.
Property | Description |
---|---|
dscp (integer: 0..63; Default: 0) | Minimum DSCP value for the lookup. |
qos-map (name; Default: default) | The name of the mapping table. If not set, the standard (built-in) mapping table gets altered. |
qos-profile (name; Default: ) | The name of the QoS profile to assign to the matched packets |
1 Port's packet/byte usage can exceed the capacity if Shared Buffers are enabled.
2 Only the queues in use are printed.
QoS Menu
...
Almost the entire QoS HW configuration is located under /in/eth/sw/qos
. Such an approach allows storing all QoS-related configuration items in one place, easy monitoring and exporting (/in/eth/sw/qos/export
).
QoS entries have two major flags:
- H - Hardware-offloaded.
- I - Inactive.
...
QoS Monitor
...
Code Block | ||||
---|---|---|---|---|
| ||||
[admin@CCR2216] /interface/ethernet/switch/qos monitor
total-packet-cap: 32768
total-packet-use: 0
total-buffer-cap: 32768
total-buffer-use: 0
multicast-packet-cap: 3276
multicast-packet-use: 0
multicast-buffer-cap: 3276
multicast-buffer-use: 0
shared-pool0-packet-cap: 13107
shared-pool0-packet-use: 0
shared-pool0-buffer-cap: 13107
shared-pool0-buffer-use: 0 |
Monitors hardware QoS resources.
...
QoS profiles determine priority field values (PCP, DSCP) for the forwarded/routed packets. Congestion avoidance/resolution is based on QoS profiles. Each packet gets a QoS profile assigned based on the ingress switch port QoS settings (see /in/eth/sw/port
).
...
Priority-to-profile mapping table(-s) for trusted packets. All switch chips have one built-in map - default. In addition, some models allow the user to define custom mapping tables and assign different maps to various switch ports via the qos-map property:
- devices based on Marvell Prestera 98DX224S, 98DX226S, or 98DX3236 switch chip models support only one map - default.
- devices based on Marvell Prestera 98DX8xxx, 98DX4xxx switch chips, or 98DX325x model devices support up to 12 maps (the default + 11 user-defined).
...
VLAN Map
...
Matches VLAN priorities (802.1p PCP/DEI fields) to QoS profiles. By default, all values are matched to the default QoS profile.
...
DSCP Map
...
Matches DSCP values to QoS profiles.
...
Sub-menu:
/interface/ethernet/switch/qos/tx-manager
Transmission (Tx) Manager controls packet enqueuing for transmission and packet tx order. Different switch ports can be assigned to different Tx managers. The maximum number of hardware Tx managers depends on the switch chip model (usually - 8).
...
Transmission Queue Scheduler
Sub-menu:
/interface/ethernet/switch/qos/tx-manager/queue
Each port has eight Tx queues. The assigned Tx Manager controls packet enqueuing and schedules transmission orders. Each queue can have either strict priority (where packets with the highest traffic class are always transmitted first) or grouped together for a weighted round-robin tx schedule.
Creating a Tx Manager automatically creates all eight respective queue schedulers.
Warning |
---|
Changing any properties of Tx manager or queues completely halts traffic enqueueing and transmission during the offload process. Temporary packet loss is expected while the device is forwarding traffic. |
Property | Description |
---|---|
tx-manager (name; read-only) | The linked Tx Manager |
traffic-class (integer: 0..7; read-only) | The traffic class (tc0..tc7) and the respective port queue (queue0..queue7) that the scheduler controls. |
schedule (strict-priority | high-priority-group | low-priority-group ) |
|
weight (integer: 0..255; Default: 1) | The weight value for the traffic class if it is a member of a schedule group. The field is not used in the case of strict priority schedule. |
queue-buffers (integer; Default: auto) | Per-queue buffer pool. The maximum number of packets that can be assigned to the queue (per each assigned port). |
use-shared-buffers (yes | no) | Allow the queue to use the shared buffer pool when queue-buffers are full. If the queue is full and the shared buffers are disabled, the packet gets dropped. If the shared buffers are enabled, the queue may use up to shared-packet-cap or shared-poolX-packet-cap (see QoS Settings for details) packets from the shared pool. |
shared-pool-index (integer; Default: 0) | The shared pool index for the queue to use. Relevant only if use-shared-buffers=yes and the device supports multiple shared pools. |
. |