Prerequisites for MPLS
"Loopback" IP address
Although not a strict requirement, it is advisable to configure routers participating in the MPLS network with "loopback" IP addresses (not attached to any real network interface) to be used by LDP to establish sessions.
This serves 2 purposes:
- as there is only one LDP session between any 2 routers, no matter how many links connect them, the loopback IP address ensures that the LDP session is not affected by interface state or address changes
- use of loopback address as LDP transport address ensures proper penultimate hop popping behavior when multiple labels are attached to the packet as in the case of VPLS
In RouterOS "loopback" IP address can be configured by creating a dummy bridge interface without any ports and adding the address to it. For example:
/interface bridge add name=lo /ip address add address=255.255.255.1/32 interface=lo
IP connectivity
As LDP distributes labels for active routes, the essential requirement is properly configured IP routing. LDP by default distributes labels for active IGP routes (that is - connected, static, and routing protocol learned routes, except BGP).
For instructions on how to set up properly IGP refer to appropriate documentation sections:
LDP supports ECMP routes.
You should be able to reach any loopback address from any location of your network before continuing with the LDP configuration. Connectivity can be verified with the ping tool running from loopback address to loopback address:
[admin@rack1_b33_CCR1036] /ip/address> /ping 255.255.255.3 src-address=255.255.255.1
SEQ HOST SIZE TTL TIME STATUS
0 255.255.255.3 56 64 247us
1 255.255.255.3 56 64 226us
2 255.255.255.3 56 64 486us
sent=3 received=3 packet-loss=0% min-rtt=226us avg-rtt=319us max-rtt=486us
Configuring LDP
Example Setup Overview
Let's consider that we have already existing four routers setup, with working IP connectivity.
(lo:255.255.255.1) (lo:255.255.255.2) (lo:255.255.255.3) (lo:255.255.255.4) |---------R1-----(192.168.1.0/24)-----R2-----(192.168.2.0/24)-----R3-----(192.168.3.0/24)-----R4---------|
You should be able to run a trace from R1 to R4:
[admin@rR1] /ip/address> /tool traceroute 255.255.255.4 src-address=255.255.255.1 Columns: ADDRESS, LOSS, SENT, LAST, AVG, BEST, WORST, STD-DEV # ADDRESS LOSS SENT LAST AVG BEST WORST STD-DEV 1 192.168.1.2 0% 2 0.2ms 0.2 0.2 0.2 0 2 192.168.2.2 0% 2 0.2ms 0.2 0.2 0.2 0 3 255.255.255.4 0% 2 0.2ms 0.2 0.2 0.2 0
In order to distribute labels for routes, LDP should get enabled. On R1 this is done by commands (interface ether3 is facing network 192.168.1.0/24):
/mpls ldp set enabled=yes transport-address=9.9.9.1 lsr-id=9.9.9.1 /mpls ldp interface add interface=ether3
Note that the transport address gets set to 9.9.9.1. This makes the router originate LDP session connections with this address and also advertise this address as a transport address to LDP neighbors.
Other routers are configured in a similar way - LDP is enabled on interfaces that connect routers and not enabled on interfaces that connect customer networks. For example, on R5:
[admin@R5] > /ip address print Flags: X - disabled, I - invalid, D - dynamic # ADDRESS NETWORK BROADCAST INTERFACE 0 4.4.4.5/24 4.4.4.0 4.4.4.255 ether1 1 5.5.5.5/24 5.5.5.0 5.5.5.255 ether2 2 9.9.9.5/32 9.9.9.5 9.9.9.5 lobridge [admin@R5] > /mpls ldp interface print Flags: I - invalid, X - disabled # INTERFACE HELLO-INTERVAL HOLD-TIME 0 ether1 5s 15s 1 ether2 5s 15s
After LDP sessions are established, on R5 there are 2 LDP neighbors:
[admin@R5] > /mpls ldp neighbor print
Flags: X - disabled, D - dynamic, O - operational, T - sending-targeted-hello, V - vpls
# TRANSPORT LOCAL-TRANSPORT PEER SEND-TARGETED ADDRESSES
0 DO 9.9.9.4 9.9.9.5 9.9.9.4:0 no 3.3.3.4
5.5.5.4
9.9.9.4
1 DO 9.9.9.3 9.9.9.5 9.9.9.3:0 no 2.2.2.3
3.3.3.3
4.4.4.3
9.9.9.3
/mpls local-bindings shows labels that this router has assigned to routes and peers it has distributed the label to. It shows that R5 has distributed labels for all its routes to both of its neighbors - R3 and R4
[admin@R5] > /mpls local-bindings print
Flags: X - disabled, A - advertised, D - dynamic, L - local-route, G - gateway-route, e - egress
# DST-ADDRESS LABEL PEERS
0 ADLe 4.4.4.0/24 impl-null 9.9.9.4:0
9.9.9.3:0
1 ADLe 9.9.9.5/32 impl-null 9.9.9.4:0
9.9.9.3:0
2 ADG 9.9.9.4/32 17 9.9.9.4:0
9.9.9.3:0
3 ADLe 5.5.5.0/24 impl-null 9.9.9.4:0
9.9.9.3:0
4 ADG 1.1.1.0/24 18 9.9.9.4:0
9.9.9.3:0
5 ADG 2.2.2.0/24 19 9.9.9.4:0
9.9.9.3:0
6 ADG 9.9.9.1/32 20 9.9.9.4:0
9.9.9.3:0
7 ADG 9.9.9.2/32 21 9.9.9.4:0
9.9.9.3:0
8 ADG 9.9.9.3/32 22 9.9.9.4:0
9.9.9.3:0
9 ADG 3.3.3.0/24 23 9.9.9.4:0
9.9.9.3:0
/mpls remote-bindings shows labels that are allocated for routes by neighboring routers and advertised to this router:
[admin@R5] > /mpls remote-bindings print Flags: X - disabled, A - active, D - dynamic # DST-ADDRESS NEXTHOP LABEL PEER 0 D 4.4.4.0/24 16 9.9.9.4:0 1 AD 3.3.3.0/24 5.5.5.4 impl-null 9.9.9.4:0 2 D 9.9.9.5/32 17 9.9.9.4:0 3 AD 9.9.9.4/32 5.5.5.4 impl-null 9.9.9.4:0 4 D 5.5.5.0/24 impl-null 9.9.9.4:0 5 D 1.1.1.0/24 18 9.9.9.4:0 6 D 2.2.2.0/24 19 9.9.9.4:0 7 D 9.9.9.1/32 20 9.9.9.4:0 8 D 9.9.9.2/32 21 9.9.9.4:0 9 D 9.9.9.3/32 22 9.9.9.4:0 10 AD 1.1.1.0/24 4.4.4.3 16 9.9.9.3:0 11 AD 2.2.2.0/24 4.4.4.3 impl-null 9.9.9.3:0 12 D 4.4.4.0/24 impl-null 9.9.9.3:0 13 D 3.3.3.0/24 impl-null 9.9.9.3:0 14 AD 9.9.9.1/32 4.4.4.3 17 9.9.9.3:0 15 AD 9.9.9.3/32 4.4.4.3 impl-null 9.9.9.3:0 16 D 9.9.9.4/32 18 9.9.9.3:0 17 D 5.5.5.0/24 19 9.9.9.3:0 18 AD 9.9.9.2/32 4.4.4.3 20 9.9.9.3:0 19 D 9.9.9.5/32 21 9.9.9.3:0
Here we can observe that R5 has received label bindings for all routes from both its neighbors - R3 and R4, but only the ones for whom particular neighbor is next hop are active. For example:
[admin@R5] > /ip route print Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit # DST-ADDRESS PREF-SRC G GATEWAY DISTANCE INTERFACE ... 5 ADo 9.9.9.1/32 r 4.4.4.3 110 ether1 ...
[admin@R5] > /mpls remote-bindings print Flags: X - disabled, A - active, D - dynamic # DST-ADDRESS NEXTHOP LABEL PEER ... 7 D 9.9.9.1/32 20 9.9.9.4:0 ... 14 AD 9.9.9.1/32 4.4.4.3 17 9.9.9.3:0 ...
From the above we see that R3, which is next hop for network 9.9.9.1/32 from R5 perspective, has assigned label 17 for traffic going to 9.9.9.1/32. This implies that when R5 will be routing traffic to this network, will impose label 17.
Label switching rules can be seen in /mpls forwarding-table. For example, on R3 it looks like this:
[admin@R3] > /mpls forwarding-table print # IN-LABEL OUT-LABELS DESTINATION INTERFACE NEXTHOP ... 2 17 17 9.9.9.1/32 ether1 2.2.2.2 ...
This rule says that R3 receiving packet with label 17 will change it to label 17 assigned by R2 for network 9.9.9.1/32 (R2 is next hop for 9.9.9.1/32 from R3 perspective):
[admin@R2] > /mpls local-bindings print
Flags: X - disabled, A - advertised, D - dynamic, L - local-route, G - gateway-route, e - egress
# DST-ADDRESS LABEL PEERS
...
3 ADG 9.9.9.1/32 17 9.9.9.1:0
9.9.9.3:0
...
R2 MPLS forwarding table tells:
[admin@R2] > /mpls forwarding-table print # IN-LABEL OUT-LABELS DESTINATION INTERFACE NEXTHOP ... 1 17 9.9.9.1/32 ether1 1.1.1.1 ...
Notice, that forwarding rule does not have any out-labels. The reason for this is that R2 is doing penultimate hop popping for this network. R1 does not assign any real label for 9.9.9.1/32 network, because it is known that R1 is egress point for 9.9.9.1/32 network (router is egress point for networks that are directly connected to it, because next hop for traffic is not MPLS router), therefore it advertises "implicit null" label for this route:
[admin@R2] > /mpls remote-bindings print Flags: X - disabled, A - active, D - dynamic # DST-ADDRESS NEXTHOP LABEL PEER ... 13 AD 9.9.9.1/32 1.1.1.1 impl-null 9.9.9.1:0 ...
This tells R2 to forward traffic for 9.9.9.1/32 to R1 unlabelled, which is exactly what R2 mpls forwarding-table entry tells. Penultimate hop popping ensures that routers do not have to do unnecessary label lookup when it is known in advance that router will have to route packet.
Using traceroute in MPLS networks
RFC4950 introduces extensions to ICMP protocol for MPLS. The basic idea is that some ICMP messages may carry MPLS "label stack object" (list of labels that were on packet when it caused particular ICMP message). ICMP messages of interest for MPLS are Time Exceeded and Need Fragment.
MPLS label carries not only label value, but also TTL field. When imposing label on IP packet, MPLS TTL is set to value in IP header, when last label is removed from IP packet, IP TTL is set to value in MPLS TTL. Therefore MPLS switching network can be diagnosed by means of traceroute tool that supports MPLS extension.
For example, traceroute from R5 to R1 looks like this:
[admin@R5] > /tool traceroute 9.9.9.1 src-address=9.9.9.5
ADDRESS STATUS
1 4.4.4.3 15ms 5ms 5ms
mpls-label=17
2 2.2.2.2 5ms 3ms 6ms
mpls-label=17
3 9.9.9.1 6ms 3ms 3ms
Traceroute results show MPLS labels on packet when it produced ICMP Time Exceeded. The above means: when R3 received packet with MPLS TTL 1, it had label 17 on. This matches label advertised by R3 for 9.9.9.1/32. The same way R2 observed label 17 on packet on next traceroute iteration - R3 switched label 17 to label 17, as explaned above. R1 received packet without labels - R2 did penultimate hop popping as explaned above.
Drawbacks of using traceroute in MPLS network
Label switching ICMP errors
One of drawbacks of using traceroute in MPLS networks is the way MPLS handles produced ICMP errors. In IP networks ICMP errors are simply routed back to source of packet that caused the error. In MPLS network it is possible that router that produces error message does not even have route to source of IP packet (for example in case of assymetric label switching paths or some kind of MPLS tunneling, e.g. to transport MPLS VPN traffic).
Due to this produced ICMP errors are not routed to the source of packet that caused the error, but switched further along label switching path, assuming that when label switching path endpoint will receive ICMP error, it will know how to properly route it back to source.
This causes the situation, that traceroute in MPLS network can not be used the same way as in IP network - to determine failure point in the network. If label switched path is broken anywhere in the middle, no ICMP replies will come back, because they will not make it to the far endpoint of label switching path.
Penultimate hop popping and traceroute source address
Thorough understanding of pen ultimate hop behaviour and routing is necessary to understand and avoid problems that penultimate hop popping causes to traceroute.
In the example setup, regular traceroute from R5 to R1 would yield the following results:
[admin@R5] > /tool traceroute 9.9.9.1
ADDRESS STATUS
1 0.0.0.0 timeout timeout timeout
2 2.2.2.2 37ms 4ms 4ms
mpls-label=17
3 9.9.9.1 4ms 2ms 11ms
compared to:
[admin@R5] > /tool traceroute 9.9.9.1 src-address=9.9.9.5
ADDRESS STATUS
1 4.4.4.3 15ms 5ms 5ms
mpls-label=17
2 2.2.2.2 5ms 3ms 6ms
mpls-label=17
3 9.9.9.1 6ms 3ms 3ms
The reason why first traceroute does not get response from R3 is that by default traceroute on R5 uses source address 4.4.4.5 for its probes, because it is preferred source for route over which next hop to 9.9.9.1/32 is reachable:
[admin@R5] > /ip route print Flags: X - disabled, A - active, D - dynamic, C - connect, S - static, r - rip, b - bgp, o - ospf, m - mme, B - blackhole, U - unreachable, P - prohibit # DST-ADDRESS PREF-SRC G GATEWAY DISTANCE INTERFACE ... 3 ADC 4.4.4.0/24 4.4.4.5 0 ether1 ... 5 ADo 9.9.9.1/32 r 4.4.4.3 110 ether1 ...
When first traceroute probe is transmitted (source: 4.4.4.5, destination 9.9.9.1), R3 drops it and produces ICMP error message (source 4.4.4.3 destination 4.4.4.5) that is switched all the way to R1. R1 then sends ICMP error back - it gets switched along label switching path to 4.4.4.5.
R2 is penultimate hop popping router for network 4.4.4.0/24, because 4.4.4.0/24 is directly connected to R3. Therefore R2 removes last label and sends ICMP error to R3 unlabelled:
[admin@R2] > /mpls forwarding-table print # IN-LABEL OUT-LABELS DESTINATION INTERFACE NEXTHOP ... 3 19 4.4.4.0/24 ether2 2.2.2.3 ...
R3 drops received IP packet, because it receives packet with its own address as source address. ICMP errors produced by following probes come back correctly, because R3 receives unlabelled packets with source addresses 2.2.2.2 and 9.9.9.1, which are acceptable to route.
Command:
[admin@R5] > /tool traceroute 9.9.9.1 src-address=9.9.9.5 ...
produces expected results, because the source address of traceroute probes is 9.9.9.5. When ICMP errors are travelling back from R1 to R5, penultimate hop popping for 9.9.9.5/32 network happens at R3, therefore it never gets to route packet with its own address as source address.
Property Reference
LDP Instance
MikroTik RouterOS implements Label Distribution Protocol (RFC 3036, RFC 5036, and RFC 7552) for IPv4 and IPv6 address families. LDP is a protocol that performs the set of procedures and exchange messages by which Label Switched Routers (LSRs) establish Label Switched Paths (LSPs) through a network by mapping network-layer routing information directly to data-link layer switched paths.
Sub-menu: /mpls
Properties
| Property | Description |
|---|---|
| afi (ip | ipv6; Default: ) | For which address family routes to do label mapping. |
| comments (string; Default: ) | Short description of the entry |
| disabled (yes | no; Default: no) | |
| distribute-for-default (yes | no; Default: no) | Defines whether to map label for the default route. |
| hop-limit (integer[0..255]; Default: ) | Max hop limit used for loop detection. Works in combination with the loop-detect property. |
| loop-detect (yes | no; Default: ) | Defines whether to run LSP loop detection. Will not work correctly if not enabled on all LSRs. Should be used only on non-TTL networks such as ATMs. |
| lsr-id | Unique label switching router's ID. |
| path-vector-limit | Max path vector limit used for loop detection. Works in combination with the loop-detect property. |
| transport-addresses | Specifies LDP session connections origin addresses and also advertises these addresses as transport addresses to LDP neighbors. |
| use-explicit-null (yes | no; Default: no) | Whether to distribute explicit-null label bindings. |
| vrf (name; Default: main) | Name of the VRF table this instance will operate on. |
Interface
Sub-menu: /mpls ldp interface
| Property | Description |
|---|---|
| afi (ip | ipv6; Default: ) | Address families that can be used by LDP transport. |
| accept-dynamic-neighbors (yes | no; Default:) | Defines whether to discover neighbors dynamically or use only statically configured in LDP neighbors menu |
| comments (string; Default: ) | Short description of the entry |
| disabled (yes | no; Default: no) | |
| hello-interval (string; Default: ) | The interval between hello packets that the router sends out on specified interface/s. The default value is 5s. |
| hold-time (string; Default: ) | Specifies the interval after which a neighbor discovered on the interface is declared as not reachable. The default value is 15s. |
| interface (string; Default: ) | Name of the interface or interface list where LDP will be listening. |
| transport-addresses (List of IPs; Default: ) | Used transport addresses if differs from LDP Instance settings. |
Neighbors
Sub-menu: /mpls ldp neighbor
This
Accept Filter
Sub-menu: /mpls ldp accept-filter
This
Advertise Filter
Sub-menu: /mpls ldp advertise-filter
This
Local Mapping
Sub-menu: /mpls local-mapping
This sub-menu shows labels bound to the routes locally in the router. In this menu also static mappings can be configured if there is no intention to use LDP dynamically.
Properties
| Property | Description |
|---|---|
| comments (string; Default: ) | Short description of the entry |
| disabled (yes | no; Default: no) | |
| dst-address (IP/Mask; Default: ) | Destination prefix the label is assigned to. |
| label (integer[0..1048576] | alert | expl-null | expl-null6 | impl-null | none; Default: ) | Label number assigned to destination. |
| vrf (name; Default: main) | Name of the VRF table this mapping belongs to. |
Read-only Properties
| Property | Description |
|---|---|
| adv-path () | |
| inactive (yes | no) | Whether binding is active and can be selected as a candidate for forwarding. |
| dynamic (yes | no) | Whether entry was dynamically added |
| egress (yes | no) | |
| gateway (yes | no) | Whether the destination is reachable through the gateway. |
| local (yes | no) | Whether the destination is locally reachable on the router |
| peers (IP:label_space) | IP address and label space of the peer to which this entry was advertised. |
Remote Mapping
Sub-menu: /mpls remote-mapping
Sub-menu shows label bindings for routes received from other routers. Static mapping can be configured if there is no intention to use LDP dynamically. This table is used to build Forwarding Table
Properties
| Property | Description |
|---|---|
| comments (string; Default: ) | Short description of the entry |
| disabled (yes | no; Default: no) | |
| dst-address (IP/Mask; Default: ) | Destination prefix the label is assigned to. |
| label (integer[0..1048576] | alert | expl-null | expl-null6 | impl-null | none; Default: ) | Label number assigned to destination. |
| nexthop (IP; Default:) | |
| vrf (name; Default: main) | Name of the VRF table this mapping belongs to. |
Read-only Properties
| Property | Description |
|---|---|
| inactive (yes | no) | Whether binding is active and can be selected as a candidate for forwarding. |
| dynamic (yes | no) | Whether entry was dynamically added |
| path (string) |