Basic MPLS Configuration and Label Swapping Explained

Topology

For step by step MPLS configuration, please check MPLS Configuration post

Below is PEs configuration for this topology, we will use as reference while checking the MPLS label swapping and packet captures:

PE1 Configuration:

ip vrf Customer_A
 rd 65000:1
 route-target export 65000:1
 route-target import 65000:1
!
ip vrf Customer_B
 rd 65000:2
 route-target export 65000:2
 route-target import 65000:2
!
interface Loopback1
 ip address 10.10.11.1 255.255.255.255
!
interface GigabitEthernet0/0
 ip address 172.16.11.1 255.255.255.252
 mpls ip
!
interface GigabitEthernet1/0
 ip vrf forwarding Customer_B
 ip address 192.168.60.1 255.255.255.0
 ip ospf 2 area 0
!
interface GigabitEthernet2/0
 ip vrf forwarding Customer_A
 ip address 192.168.10.1 255.255.255.0
 ip ospf 3 area 0
!
router ospf 2 vrf Customer_B
 router-id 192.168.60.1
 log-adjacency-changes
 redistribute bgp 65000 subnets
!
router ospf 3 vrf Customer_A
 log-adjacency-changes
 redistribute bgp 65000 subnets
!
router ospf 100
 router-id 10.10.11.1
 log-adjacency-changes
 network 10.10.11.1 0.0.0.0 area 0
 network 172.16.11.0 0.0.0.3 area 0
!
router bgp 65000
 no synchronization
 bgp log-neighbor-changes
 neighbor 10.10.22.2 remote-as 65000
 neighbor 10.10.22.2 update-source Loopback1
 no auto-summary
 !
 address-family vpnv4
  neighbor 10.10.22.2 activate
  neighbor 10.10.22.2 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf Customer_B
  redistribute ospf 2 vrf Customer_B
  no synchronization
 exit-address-family
 !
 address-family ipv4 vrf Customer_A
  redistribute ospf 3 vrf Customer_A
  no synchronization
 exit-address-familyCode language: JavaScript (javascript)

PE2 Configuration:

ip vrf Customer_A
 rd 65000:1
 route-target export 65000:1
 route-target import 65000:1
!
ip vrf Customer_B
 rd 65000:2
 route-target export 65000:2
 route-target import 65000:2

interface Loopback1
 ip address 10.10.22.2 255.255.255.255
!
interface GigabitEthernet0/0
 ip address 172.16.22.2 255.255.255.252
 mpls ip
!
interface GigabitEthernet1/0
 ip vrf forwarding Customer_B
 ip address 192.168.70.1 255.255.255.0
 ip ospf 2 area 0
 negotiation auto
!
interface GigabitEthernet2/0
 ip vrf forwarding Customer_A
 ip address 192.168.20.1 255.255.255.0
 ip ospf 33 area 0
 negotiation auto
!
router ospf 2 vrf Customer_B
 router-id 192.168.70.1
 log-adjacency-changes
 redistribute bgp 65000 subnets
!
router ospf 33 vrf Customer_A
 log-adjacency-changes
 redistribute bgp 65000 subnets
!
router ospf 100
 router-id 10.10.22.2
 log-adjacency-changes
 network 10.10.22.2 0.0.0.0 area 0
 network 172.16.22.0 0.0.0.3 area 0
!
router ospf 3 vrf Customer_B
 log-adjacency-changes
!
router bgp 65000
 no synchronization
 bgp log-neighbor-changes
 neighbor 10.10.11.1 remote-as 65000
 neighbor 10.10.11.1 update-source Loopback1
 no auto-summary
 !
 address-family vpnv4
  neighbor 10.10.11.1 activate
  neighbor 10.10.11.1 send-community extended
 exit-address-family
 !
 address-family ipv4 vrf Customer_B
  redistribute ospf 2 vrf Customer_B
  no synchronization
 exit-address-family
 !
 address-family ipv4 vrf Customer_A
  redistribute ospf 33 vrf Customer_A
  no synchronization
 exit-address-family
!Code language: JavaScript (javascript)

and P core routers just have basic MPLS ldp configured for them,

OK, let’s zoom in on CustomerA and see how CE1-CustA (192.168.10.10) is able to communicate with CE2-CustA (192.168.20.10):

Starting from CE1-CustA, we see it’s learning about destination 192.168.20.10 via OSPF from its neighbor PE1:

CE1-CustA#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.10.0/24 is directly connected, GigabitEthernet0/0
O E2 192.168.20.0/24 [110/1] via 192.168.10.1, 00:16:46, GigabitEthernet0/0

CE1-CustA#show ip ospf neighbor

Neighbor ID     Pri   State           Dead Time   Address         Interface
192.168.10.1      1   FULL/BDR        00:00:38    192.168.10.1    GigabitEthernet0/0

Now, a bit more interesting part:

When we check PE1, we see that 192.168.20.0/24 is learnt via iBGP from peer 10.10.22.2 (which is the loopback of PE2):

PE1#show ip route vrf Customer_A

Routing Table: Customer_A
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

C    192.168.10.0/24 is directly connected, GigabitEthernet2/0
B    192.168.20.0/24 [200/0] via 10.10.22.2, 00:30:58Code language: PHP (php)

Verifying how PE1 actually got this BGP prefix from wireshark capture:

We see the BGP update from PE2 (10.10.22.2) including the MPLS vpnv4 NLRI for prefix 192.168.20.0/24:

In the BGP prefix, we notice the label (VPN label or inner label), basically PE2 saying to PE2 saying to its BGP peer PE1: “to reach to 192.168.20.0/24 for CustomerA (65000:1) ,it must send with label 21”.

In other words, label make sense only for PE2 (for the CustomerA destination prefix 192.168.20.0/24),

After PE1 receive this BGP update, let’s verify directly from PE1 CLI:

We see for destination 192.168.20.0/24, the Next hop 10.10.22.2 (PE2) and label is 21:

PE1#show ip bgp vpnv4 vrf Customer_A 192.168.20.0
BGP routing table entry for 65000:1:192.168.20.0/24, version 14
Paths: (1 available, best #1, table Customer_A)
  Not advertised to any peer
  Local
    10.10.22.2 (metric 4) from 10.10.22.2 (10.10.22.2)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:65000:1 OSPF DOMAIN ID:0x0005:0x000000210200
        OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:192.168.20.1:0
      mpls labels in/out nolabel/21Code language: PHP (php)

Now, let’s assume PE1 will just add this single label (21) and send packet out to P1,

let’s verify routing table of the core router P1, it actually has not aware of the customer prefix 192.168.20.10, this is expected, but worth to mention:

P1#show ip route 192.168.20.10
% Network not in table

P1#show ip route
Codes: C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route

Gateway of last resort is not set

     172.16.0.0/30 is subnetted, 3 subnets
O       172.16.22.0 [110/2] via 172.16.12.2, 01:49:14, GigabitEthernet1/0
C       172.16.12.0 is directly connected, GigabitEthernet1/0
C       172.16.11.0 is directly connected, GigabitEthernet0/0
     10.0.0.0/32 is subnetted, 4 subnets
O       10.10.11.1 [110/2] via 172.16.11.1, 01:20:05, GigabitEthernet0/0
O       10.10.22.2 [110/3] via 172.16.12.2, 01:49:04, GigabitEthernet1/0
C       10.10.111.1 is directly connected, Loopback1
O       10.10.222.2 [110/2] via 172.16.12.2, 01:49:14, GigabitEthernet1/0Code language: PHP (php)

Now, the idea is that PE1 knows about the customer destination prefix from BGP vpnv4 update (sent by PE2) and PE2 indicating in the BGP update that in order to reach the Customer_A prefix 192.168.20.0/24, send it to me with label 21. This is why we say that Inner label (aka vpn label) is dictated by BGP.

but, PE1 needs to reach PE2 10.10.22.2 (where the label 21 makes sense), that where the core of MPLS concept comes in, the label switching (aka label swapping).

Basically, traffic won’t be routed in MPLS core, rather it’s based on Label swapping. From above, we can conclude that vpn (inner label) is send by PE2, so, this label cannot be used for swapping in the MPLS core router (P1 and P2).

Another label is needed for swapping in MPLS core, if we check the packet capture for LDP (Label Distributuon Protocol):

For the prefix 10.10.22.2 (which is loopback for PE2), P1 is sending LDP mapping to PE1 indicating that in order to reach to this prefix, use MPLS label 23 (This is the outer Label).

So, now PE1 is able to construct 2 labels:

This recursion we are describing above can be clearly seen from the CEF command:

PE1#show ip cef vrf Customer_A 192.168.20.10 detail
192.168.20.0/24, epoch 0
  recursive via 10.10.22.2 label 21
    nexthop 172.16.11.2 GigabitEthernet0/0 label 18Code language: PHP (php)

So, for PE1 (Customer_A vrf) to reach destination 192.168.20.10, it got this prefix via BGP with NH 10.10.22.2 and label 21, and to reach this Next-hop, packet will be sent to 172.16.11.2 (which is P1) and add a label 18 (Outer label).

That’s too much talking, let’s see it in Wireshark, starting an ICMP ping between CE1-CustA (192.168.10.10) and CE2_CustA (192.168.20.10):

We can the stack of label (Outer label 18 and inner label 21):

Then, this packet will reach Core router P1, and the only job needs to be done by P1 is to swap the outer label.

If we check the mpls forwarding table: if we received label 18, it will be swapped with label 17 and sent to Next 172.16.12.2 which is P2:

P1#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     Pop Label     10.10.222.2/32    0             Gi1/0      172.16.12.2
17     Pop Label     172.16.22.0/30    0             Gi1/0      172.16.12.2
18     17            10.10.22.2/32     84886         Gi1/0      172.16.12.2
19     Pop Label     10.10.11.1/32     74892         Gi0/0      172.16.11.1Code language: PHP (php)

Quick question, how does P1 knows the outgoing label for the prefix 10.10.22.2/32?

The same way PE1 got the outer label information from P1 via LDP, P1 got its outgoing label from P2 via LDP. We can verify by capturing in the link between P1 and P2:

We see label 17 in the LDP packet from P2:

So, P1 will swap Outer label (18 to 17) and will send it to P2:

Now, on P2, since this is the last hop before PE2 (where the vpn inner label makes sense), there is no need to swap anymore, and outer label will be poped by P2 and sent to PE2 with a single label (the VPN label):

P2#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     Pop Label     10.10.111.1/32    0             Gi0/0      172.16.12.1
17     Pop Label     10.10.22.2/32     81847         Gi1/0      172.16.22.2
18     Pop Label     172.16.11.0/30    0             Gi0/0      172.16.12.1
19     19            10.10.11.1/32     2196          Gi0/0      172.16.12.1Code language: PHP (php)

An interesting note, in the LDP packet sent from PE2 to P2 for prefix 10.10.22.2, since this prefix is on PE2, I would expect PE2 to send LDP label mapping message for this prefix with Null value.

I was a bit confused when saw it’s label 3, then verified in RFC (RFC3032) documents that Label value 3 is reserved value and it means “Implicit NULL Label”:

So, from this LDP packet from PE2, P2 knows that for prefix 10.10.22.2, the LDP neighbor PE2 is where this prefix resides and dictate that it needs to popup label for it.

Indeed, if we check traffic on link from P2 to PE2, we only see a single tag: 21 (vpn label)

At this points, the packet is received by PE2:

PE2#show mpls forwarding-table
Local  Outgoing      Prefix            Bytes Label   Outgoing   Next Hop
Label  Label or VC   or Tunnel Id      Switched      interface
16     Pop Label     10.10.222.2/32    0             Gi0/0      172.16.22.1
17     Pop Label     172.16.12.0/30    0             Gi0/0      172.16.22.1
18     16            10.10.111.1/32    0             Gi0/0      172.16.22.1
19     19            10.10.11.1/32     0             Gi0/0      172.16.22.1
20     18            172.16.11.0/30    0             Gi0/0      172.16.22.1
21     No Label      192.168.20.0/24[V]   \
                                       1710          aggregate/Customer_A
22     No Label      192.168.70.0/24[V]   \
                                       0             aggregate/Customer_BCode language: PHP (php)

PE2 knows about VPN label 21 for Customer_A and will remove the label (No Label) and will send it to CE2-CustA:

PE2#show ip route vrf Customer_A 192.168.20.10
Routing entry for 192.168.20.0/24
  Known via "connected", distance 0, metric 0 (connected, via interface)
  Redistributing via bgp 65000
  Advertised by bgp 65000
  Routing Descriptor Blocks:
  * directly connected, via GigabitEthernet2/0
      Route metric is 0, traffic share count is 1