VXLAN with Multicast Control Plane -Configuration and Packet Flow [EVE-ng Lab]
Topology:
I- Build the underlay and set up Multicast
1- Configure IP address connection, OSPF, and activate Pim Sparse-mode:
- Leaf-1:
LEAF-1(config)# feature ospf
LEAF-1(config)# feature pim
LEAF-1(config)# interface Ethernet1/2
LEAF-1(config-if)# description to Spine-1
LEAF-1(config-if)# no switchport
LEAF-1(config-if)# mtu 9216
LEAF-1(config-if)# ip address 10.0.0.1/30
LEAF-1(config-if)# ip ospf network point-to-point
LEAF-1(config-if)# ip router ospf 1 area 0.0.0.0
LEAF-1(config-if)# no shutdown
LEAF-1(config-if)# ip pim sparse-mode
LEAF-1(config-if)# interface Ethernet1/3
LEAF-1(config-if)# description to Spine-2
LEAF-1(config-if)# no switchport
LEAF-1(config-if)# mtu 9216
LEAF-1(config-if)# ip address 10.0.0.5/30
LEAF-1(config-if)# ip ospf network point-to-point
LEAF-1(config-if)# ip router ospf 1 area 0.0.0.0
LEAF-1(config-if)# no shutdown
LEAF-1(config-if)# ip pim sparse-mode
LEAF-1(config)# interface loopback0
LEAF-1(config-if)# description Loopback
LEAF-1(config-if)# ip address 192.168.0.
3/32
LEAF-1(config-if)# ip router ospf 1 area 0.0.0.0
LEAF-1(config-if)# ip pim sparse-mode
LEAF-1(config-if)# exit
LEAF-1(config)# router ospf 1
LEAF-1(config-router)# router-id 192.168.0.3
Code language: PHP (php)- Leaf-2:
LEAF-2(config)# feature ospf
LEAF-2(config)# feature pim
LEAF-2(config)# interface Ethernet1/3
LEAF-2(config-if)# description to Spine-1
LEAF-2(config-if)# no switchport
LEAF-2(config-if)# mtu 9216
LEAF-2(config-if)# ip address 10.0.0.9/30
LEAF-2(config-if)# ip ospf network point-to-point
LEAF-2(config-if)# ip router ospf 1 area 0.0.0.0
LEAF-2(config-if)# no shutdown
LEAF-2(config-if)# ip pim sparse-mode
LEAF-2(config-if)# interface Ethernet1/4
LEAF-2(config-if)# description to Spine-2
LEAF-2(config-if)# no switchport
LEAF-2(config-if)# mtu 9216
LEAF-2(config-if)# ip address 10.0.0.13/30
LEAF-2(config-if)# ip ospf network point-to-point
LEAF-2(config-if)# ip router ospf 1 area 0.0.0.0
LEAF-2(config-if)# no shutdown
LEAF-2(config-if)# ip pim sparse-mode
LEAF-2(config)# interface loopback0
LEAF-2(config-if)# description Loopback
LEAF-2(config-if)# ip address 192.168.0.4/32
LEAF-2(config-if)# ip router ospf 1 area 0.0.0.0
LEAF-2(config-if)# ip pim sparse-mode
LEAF-2(config)# router ospf 1
LEAF-2(config-router)# router-id 192.168.0.4
Code language: PHP (php)- Spine-1:
SPINE-1(config)# feature ospf
SPINE-1(config)# feature pim
SPINE-1(config)# interface Ethernet1/1
SPINE-1(config-if)# description to leaf-1
SPINE-1(config-if)# no switchport
SPINE-1(config-if)# mtu 9216
SPINE-1(config-if)# ip address 10.0.0.2/30
SPINE-1(config-if)# ip ospf network point-to-point
SPINE-1(config-if)# ip router ospf 1 area 0.0.0.0
SPINE-1(config-if)# no shutdown
SPINE-1(config-if)# ip pim sparse-mode
SPINE-1(config-if)# interface Ethernet1/2
SPINE-1(config-if)# description to leaf-2
SPINE-1(config-if)# no switchport
SPINE-1(config-if)# mtu 9216
SPINE-1(config-if)# ip address 10.0.0.10/30
SPINE-1(config-if)# ip ospf network point-to-point
SPINE-1(config)# interface loopback0
SPINE-1(config-if)# description Loopback
SPINE-1(config-if)# ip address 192.168.0.1/32
SPINE-1(config-if)# ip router ospf 1 area 0.0.0.0
SPINE-1(config-if)# ip pim sparse-mode
SPINE-1(config-if)# ip router ospf 1 area 0.0.0.0
SPINE-1(config-if)# no shutdown
SPINE-1(config-if)# ip pim sparse-modeCode language: PHP (php)- Spine-2:
SPINE-2(config)# feature ospf
SPINE-2(config)# feature pim
SPINE-2(config)# interface Ethernet1/1
SPINE-2(config-if)# description to leaf-1
SPINE-2(config-if)# no switchport
SPINE-2(config-if)# mtu 9216
SPINE-2(config-if)# ip address 10.0.0.6/30
SPINE-2(config-if)# ip ospf network point-to-point
SPINE-2(config-if)# ip router ospf 1 area 0.0.0.0
SPINE-2(config-if)# no shutdown
SPINE-2(config-if)# ip pim sparse-mode
SPINE-2(config-if)# !
SPINE-2(config-if)# interface Ethernet1/2
SPINE-2(config-if)# description to leaf-2
SPINE-2(config-if)# no switchport
SPINE-2(config-if)# mtu 9216
SPINE-2(config-if)# ip address 10.0.0.14/30
SPINE-2(config-if)# ip ospf network point-to-point
SPINE-2(config-if)# ip router ospf 1 area 0.0.0.0
SPINE-2(config-if)# no shutdown
SPINE-2(config-if)# ip pim sparse-mode
SPINE-2(config)# interface loopback0
SPINE-2(config-if)# description Loopback
SPINE-2(config-if)# ip address 192.168.0.2/32
SPINE-2(config-if)# ip router ospf 1 area 0.0.0.0
SPINE-2(config-if)# ip pim sparse-mode
SPINE-2(config)# router ospf 1
SPINE-2(config-router)# router-id 192.168.0.2
Code language: PHP (php)2- Verification of OSPF and PIM neighborship:
- Leaf-1:
LEAF-1# show ip ospf neighbors
OSPF Process ID 1 VRF default
Total number of neighbors: 2
Neighbor ID Pri State Up Time Address Interface
192.168.0.1 1 FULL/ - 00:07:27 10.0.0.2 Eth1/2
192.168.0.2 1 FULL/ - 00:02:43 10.0.0.6 Eth1/3
LEAF-1# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD
Priority Capable State
10.0.0.2 Ethernet1/2 00:10:16 00:01:25 1 yes n/a
10.0.0.6 Ethernet1/3 00:06:16 00:01:40 1 yes n/a- Leaf-2:
LEAF-2# show ip ospf neighbors
OSPF Process ID 1 VRF default
Total number of neighbors: 2
Neighbor ID Pri State Up Time Address Interface
192.168.0.1 1 FULL/ - 00:09:36 10.0.0.10 Eth1/3
192.168.0.2 1 FULL/ - 00:04:54 10.0.0.14 Eth1/4
LEAF-2# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD
Priority Capable State
10.0.0.10 Ethernet1/3 00:10:46 00:01:28 1 yes n/a
10.0.0.14 Ethernet1/4 00:08:17 00:01:31 1 yes n/a
LEAF-2#
Spine-1:
SPINE-1# show ip ospf neighbors
OSPF Process ID 1 VRF default
Total number of neighbors: 2
Neighbor ID Pri State Up Time Address Interface
192.168.0.3 1 FULL/ - 00:10:40 10.0.0.1 Eth1/1
192.168.0.4 1 FULL/ - 00:10:40 10.0.0.9 Eth1/2
SPINE-1# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD
Priority Capable State
10.0.0.1 Ethernet1/1 00:13:27 00:01:42 1 yes n/a
10.0.0.9 Ethernet1/2 00:11:51 00:01:34 1 yes n/a
SPINE-1#
Spine-2:
SPINE-2# show ip ospf neighbors
OSPF Process ID 1 VRF default
Total number of neighbors: 2
Neighbor ID Pri State Up Time Address Interface
192.168.0.3 1 FULL/ - 00:06:37 10.0.0.5 Eth1/1
192.168.0.4 1 FULL/ - 00:06:38 10.0.0.13 Eth1/2
SPINE-2# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD
Priority Capable State
10.0.0.5 Ethernet1/1 00:10:06 00:01:21 1 yes n/a
10.0.0.13 Ethernet1/2 00:10:02 00:01:31 1 yes n/a
SPINE-2#
Once, we have OSPF reachability for the Loopback interfaces and PIM neighbors are UP.
Next, we will configure the PIM Rendez-vous point as Spine-1 and we will configure it for the group range 239.1.1.0/25 (which will cover the group we will use for Bridge domain).
3- Configure PIM RP:
On Leaf-1 and Leaf-2:
LEAF-1(config)# ip pim rp-address 192.168.0.1 group-list 239.1.1.0/25
LEAF-1(config)# ip pim ssm range 232.0.0.0/8Code language: PHP (php)II- Configure VXLAN (Overlay):
On Leaf-1 and Leaf-2:
- Enable features:
we need to enable certain features specifically for:
- feature interface-vlan: allow the configuration of SVI (interface vlan)
- feature nv overlay: Enables the VXLAN feature.
- feature vn-segment-vlan-based: Configures the global mode for all VXLAN bridge domains.
LEAF-1(config)# feature interface-vlan
LEAF-1(config)# feature nv overlay
LEAF-1(config)# feature vn-segment-vlan-basedCode language: PHP (php)- Create VLAN for endpoints and assign VNID ((vn-segement id) to the VLAN :
- The VLAN represent the local representation for the bridge domain on the Leaf.
- The VXLAN VNI is the global value for the VXLAN on every switch
- Virtual Network Identifier (VNI) identify a VXLAN segment.
! create vlan
LEAF-1(config)# vlan 140
LEAF-1(config)# interface eth1/1
LEAF-1(config-if)# switchport access vlan 140
! assign vlan to vn-segment
LEAF-1(config-vlan)# vlan 140
LEAF-1(config-vlan)# name L2-VNI-140-Tenant1
LEAF-1(config-vlan)# vn-segment 50140Code language: PHP (php)- Create interface Vlan as gateway for endpoints (On Leaf-1 and Leaf-2):
LEAF-1(config)# interface Vlan140
LEAF-1(config-if)# no shutdown
LEAF-1(config-if)# no ip redirects
LEAF-1(config-if)# ip address 172.21.140.1/24Code language: PHP (php)- Configure NVE interface (VTEP):
- The NVE (Network Virtualization Edge) interface is a logical interface that will perform encapsulation and de-encapsulation of VXLAN packets.
- The NVE interface will use loopback1 interface as source interface, which means the loopback address will be used as outer source IP address for the VXLAN packets.
- The NVE is mapped to the VNI and the multicast group.
On Leaf-1:
LEAF-1(config)# interface nve1
LEAF-1(config-if-nve)# no shutdown
LEAF-1(config-if-nve)# source-interface loopback1
LEAF-1(config-if-nve)# member vni 50140
LEAF-1(config-if-nve-vni)# mcast-group 239.1.1.0
LEAF-1(config)# interface loopback1
LEAF-1(config-if)# description VXLAN VTEP
LEAF-1(config-if)# ip pim sparse-mode
LEAF-1(config-if)# ip address 192.168.0.13/32
LEAF-1(config-if)# ip router ospf 1 area 0.0.0.0Code language: PHP (php)on Leaf-2:
LEAF-2(config)# interface nve1
LEAF-2(config-if-nve)# no shutdown
LEAF-2(config-if-nve)# source-interface loopback1
LEAF-2(config-if-nve)# member vni 50140
LEAF-2(config-if-nve-vni)# mcast-group 239.1.1.0
LEAF-2(config)# interface loopback1
LEAF-2(config-if)# description VXLAN VTEP
LEAF-2(config-if)# ip pim sparse-mode
LEAF-2(config-if)# ip address 192.168.0.14/32
LEAF-2(config-if)# ip router ospf 1 area 0.0.0.0Code language: PHP (php)Verifications and Packet flow:
- show nve vni: will display all NVE and its corresponding VNIs, multicast-group
learning mode (Dataplane) and BD which is the VLAN mapped to the VNI.
LEAF-01# show nve vni
Codes: CP - Control Plane DP - Data Plane
UC - Unconfigured SA - Suppress ARP
SU - Suppress Unknown Unicast
Xconn - Crossconnect
MS-IR - Multisite Ingress Replication
Interface VNI Multicast-group State Mode Type [BD/VRF] Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1 50140 239.1.1.0 Up DP L2 [140]Let’s check the packet flow for communication between “Linux” and “VPC2” that will cause the Leafs to learn other peer via Data plane:
1- Linux server wants to communicate with VPC2, Linux send an ARP request to discover the MAC of VPC2.
2- Once ARP is received on Leaf-1, it will flood the packet in the VLAN and also will encapsulate the ARP request in VXLAN and send it to multicast group which is configured for the VNI.
(The VLAN 140 is mapped to VNI 50140 (which also configured with mcast group 239.1.1.0, NVE1 is member of VNI 50140)
So, the ARP packet is encapsulated into VXLAN with Outer destination IP: 239.1.1.0, source IP: Lo1 of leaf-1, VNID 50140 and sent toward the RP.
3- The multicast PIM RP (spine-1 in our example) will receive the packet and it will forward a copy to every VTEP that has joined the multicast group (mcast group already configured and mapped to the VNI on leaf-2).
4- Leaf-2 receives the VXLAN packet (including original ARP request) and de-encapsulate it. It will first learn the MAC address of “Linux” pointing to newly learned VTEP of leaf-1. Leaf-2 will forward the ARP request to “VPC2”.
5- The VPC2 will respond with ARP reply. Leaf-2 will receive it and check it’s MAC address table and see that MAC of destination “Linux” is learned via the interface nve1 (learned via DP from the last ARP request).
➤ So, Leaf-2 will encapsulate the ARP reply in VXLAN and send it as unicast toward Leaf-1.
6- Once Leaf-1 receive the packet, de-encapsulate it, learn the MAC of “VPC2” and mapped it to nve1 interface.
At this point, both Leaf-1 and Leaf-2 learn via Data plane each peer VTEP and also the MAC which had communication.
It can be verified with the following commands:
- Show nve peers:
show nve peers will display the learned VTEP peers and learning type, in our case DP.
LEAF-01# show nve peers
Interface Peer-IP State LearnType Uptime Route
r-Mac
--------- -------------------------------------- ----- --------- -------- -----
------------
nve1 192.168.0.14 Up DP 4d00h n/a- Show mac address:
In the show mac-address table, we can see the MAC address table of the remote Endpoint associated with the Peer VTEP.
when a packet comes to Leaf-01 destined to the MAC 0050.0000.0800, it will be encapsulated in VXLAN and unicasted to the peer address which is already learned via Dataplane (with multicast).
LEAF-01# show mac address-table
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
* 140 0050.0000.0700 dynamic 0 F F Eth1/7
* 140 0050.0000.0800 dynamic 0 F F nve1(192.168.0.14)
* 140 5001.0000.1b08 static - F F Vlan140
G - 0000.2222.3333 static - F F sup-eth1(R)
G - 5001.0000.1b08 static - F F sup-eth1(R)
G 140 5001.0000.1b08 static - F F sup-eth1(R)