VXLAN with Multicast Control Plane -Configuration and Packet Flow [EVE-ng Lab]

Topology:

I- Build the underlay and set up Multicast

1- Configure IP address connection, OSPF, and activate Pim Sparse-mode:

• Leaf-1:

LEAF-1(config)# feature ospf
LEAF-1(config)# feature pim

LEAF-1(config)# interface Ethernet1/2
LEAF-1(config-if)#  description to Spine-1
LEAF-1(config-if)#  no switchport
LEAF-1(config-if)#  mtu 9216
LEAF-1(config-if)#  ip address 10.0.0.1/30
LEAF-1(config-if)#  ip ospf network point-to-point
LEAF-1(config-if)#  ip router ospf 1 area 0.0.0.0
LEAF-1(config-if)#  no shutdown
LEAF-1(config-if)#  ip pim sparse-mode

LEAF-1(config-if)# interface Ethernet1/3
LEAF-1(config-if)#  description to Spine-2
LEAF-1(config-if)#  no switchport
LEAF-1(config-if)#  mtu 9216
LEAF-1(config-if)#  ip address 10.0.0.5/30
LEAF-1(config-if)#  ip ospf network point-to-point
LEAF-1(config-if)#  ip router ospf 1 area 0.0.0.0
LEAF-1(config-if)#  no shutdown
LEAF-1(config-if)#  ip pim sparse-mode



LEAF-1(config)# interface loopback0
LEAF-1(config-if)#  description Loopback
LEAF-1(config-if)#  ip address 192.168.0.
3/32
LEAF-1(config-if)#  ip router ospf 1 area 0.0.0.0
LEAF-1(config-if)#  ip pim sparse-mode


LEAF-1(config-if)# exit
LEAF-1(config)# router ospf 1
LEAF-1(config-router)#  router-id 192.168.0.3
Code language: PHP (php)

• Leaf-2:

LEAF-2(config)# feature ospf
LEAF-2(config)# feature pim

LEAF-2(config)# interface Ethernet1/3
LEAF-2(config-if)#  description to Spine-1
LEAF-2(config-if)#  no switchport
LEAF-2(config-if)#  mtu 9216
LEAF-2(config-if)#  ip address 10.0.0.9/30
LEAF-2(config-if)#  ip ospf network point-to-point
LEAF-2(config-if)#  ip router ospf 1 area 0.0.0.0
LEAF-2(config-if)#  no shutdown
LEAF-2(config-if)#  ip pim sparse-mode


LEAF-2(config-if)# interface Ethernet1/4
LEAF-2(config-if)#  description to Spine-2
LEAF-2(config-if)#  no switchport
LEAF-2(config-if)#  mtu 9216
LEAF-2(config-if)#  ip address 10.0.0.13/30
LEAF-2(config-if)#  ip ospf network point-to-point
LEAF-2(config-if)#  ip router ospf 1 area 0.0.0.0
LEAF-2(config-if)#  no shutdown
LEAF-2(config-if)#  ip pim sparse-mode

LEAF-2(config)# interface loopback0
LEAF-2(config-if)#  description Loopback
LEAF-2(config-if)#  ip address 192.168.0.4/32
LEAF-2(config-if)#  ip router ospf 1 area 0.0.0.0
LEAF-2(config-if)#  ip pim sparse-mode

LEAF-2(config)# router ospf 1
LEAF-2(config-router)#  router-id 192.168.0.4

Code language: PHP (php)

• Spine-1:

SPINE-1(config)# feature ospf
SPINE-1(config)# feature pim

SPINE-1(config)# interface Ethernet1/1
SPINE-1(config-if)#  description to leaf-1
SPINE-1(config-if)#  no switchport
SPINE-1(config-if)#  mtu 9216
SPINE-1(config-if)#  ip address 10.0.0.2/30
SPINE-1(config-if)#  ip ospf network point-to-point
SPINE-1(config-if)#  ip router ospf 1 area 0.0.0.0
SPINE-1(config-if)#  no shutdown
SPINE-1(config-if)#  ip pim sparse-mode


SPINE-1(config-if)# interface Ethernet1/2
SPINE-1(config-if)#  description to leaf-2
SPINE-1(config-if)#  no switchport
SPINE-1(config-if)#  mtu 9216
SPINE-1(config-if)#  ip address 10.0.0.10/30
SPINE-1(config-if)#  ip ospf network point-to-point

SPINE-1(config)# interface loopback0
SPINE-1(config-if)#  description Loopback
SPINE-1(config-if)#  ip address 192.168.0.1/32
SPINE-1(config-if)#  ip router ospf 1 area 0.0.0.0
SPINE-1(config-if)#  ip pim sparse-mode

SPINE-1(config-if)#  ip router ospf 1 area 0.0.0.0
SPINE-1(config-if)#  no shutdown
SPINE-1(config-if)#  ip pim sparse-modeCode language: PHP (php)

• Spine-2:

SPINE-2(config)# feature ospf
SPINE-2(config)# feature pim

SPINE-2(config)# interface Ethernet1/1
SPINE-2(config-if)#  description to leaf-1
SPINE-2(config-if)#  no switchport
SPINE-2(config-if)#  mtu 9216
SPINE-2(config-if)#  ip address 10.0.0.6/30
SPINE-2(config-if)#  ip ospf network point-to-point
SPINE-2(config-if)#  ip router ospf 1 area 0.0.0.0
SPINE-2(config-if)#  no shutdown
SPINE-2(config-if)#  ip pim sparse-mode
SPINE-2(config-if)# !

SPINE-2(config-if)# interface Ethernet1/2
SPINE-2(config-if)#  description to leaf-2
SPINE-2(config-if)#  no switchport
SPINE-2(config-if)#  mtu 9216
SPINE-2(config-if)#  ip address 10.0.0.14/30
SPINE-2(config-if)#  ip ospf network point-to-point
SPINE-2(config-if)#  ip router ospf 1 area 0.0.0.0
SPINE-2(config-if)#  no shutdown
SPINE-2(config-if)#  ip pim sparse-mode


SPINE-2(config)# interface loopback0
SPINE-2(config-if)#  description Loopback
SPINE-2(config-if)#  ip address 192.168.0.2/32
SPINE-2(config-if)#  ip router ospf 1 area 0.0.0.0
SPINE-2(config-if)#  ip pim sparse-mode

SPINE-2(config)# router ospf 1
SPINE-2(config-router)#  router-id 192.168.0.2
Code language: PHP (php)

2- Verification of OSPF and PIM neighborship:

• Leaf-1:

LEAF-1#  show ip ospf neighbors
 OSPF Process ID 1 VRF default
 Total number of neighbors: 2
 Neighbor ID     Pri State            Up Time  Address         Interface
 192.168.0.1       1 FULL/ -          00:07:27 10.0.0.2        Eth1/2
 192.168.0.2       1 FULL/ -          00:02:43 10.0.0.6        Eth1/3


LEAF-1# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor        Interface            Uptime    Expires   DR       Bidir-  BFD
                                                         Priority Capable State
10.0.0.2        Ethernet1/2          00:10:16  00:01:25  1        yes     n/a
10.0.0.6        Ethernet1/3          00:06:16  00:01:40  1        yes     n/a

• Leaf-2:

LEAF-2# show ip ospf  neighbors
 OSPF Process ID 1 VRF default
 Total number of neighbors: 2
 Neighbor ID     Pri State            Up Time  Address         Interface
 192.168.0.1       1 FULL/ -          00:09:36 10.0.0.10       Eth1/3
 192.168.0.2       1 FULL/ -          00:04:54 10.0.0.14       Eth1/4


LEAF-2# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor        Interface            Uptime    Expires   DR       Bidir-  BFD
                                                         Priority Capable State
10.0.0.10       Ethernet1/3          00:10:46  00:01:28  1        yes     n/a
10.0.0.14       Ethernet1/4          00:08:17  00:01:31  1        yes     n/a
LEAF-2#

Spine-1:

SPINE-1# show ip ospf  neighbors
 OSPF Process ID 1 VRF default
 Total number of neighbors: 2
 Neighbor ID     Pri State            Up Time  Address         Interface
 192.168.0.3       1 FULL/ -          00:10:40 10.0.0.1        Eth1/1
 192.168.0.4       1 FULL/ -          00:10:40 10.0.0.9        Eth1/2


SPINE-1# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor        Interface            Uptime    Expires   DR       Bidir-  BFD
                                                         Priority Capable State
10.0.0.1        Ethernet1/1          00:13:27  00:01:42  1        yes     n/a
10.0.0.9        Ethernet1/2          00:11:51  00:01:34  1        yes     n/a
SPINE-1#

Spine-2:

SPINE-2# show ip ospf  neighbors
 OSPF Process ID 1 VRF default
 Total number of neighbors: 2
 Neighbor ID     Pri State            Up Time  Address         Interface
 192.168.0.3       1 FULL/ -          00:06:37 10.0.0.5        Eth1/1
 192.168.0.4       1 FULL/ -          00:06:38 10.0.0.13       Eth1/2


SPINE-2# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor        Interface            Uptime    Expires   DR       Bidir-  BFD
                                                         Priority Capable State
10.0.0.5        Ethernet1/1          00:10:06  00:01:21  1        yes     n/a
10.0.0.13       Ethernet1/2          00:10:02  00:01:31  1        yes     n/a
SPINE-2#

Once, we have OSPF reachability for the Loopback interfaces and PIM neighbors are UP.

Next, we will configure the PIM Rendez-vous point as Spine-1 and we will configure it for the group range 239.1.1.0/25 (which will cover the group we will use for Bridge domain).

3- Configure PIM RP:

On Leaf-1 and Leaf-2:

LEAF-1(config)# ip pim rp-address 192.168.0.1 group-list 239.1.1.0/25
LEAF-1(config)# ip pim ssm range 232.0.0.0/8Code language: PHP (php)

II- Configure VXLAN (Overlay):

On Leaf-1 and Leaf-2:

• Enable features:

we need to enable certain features specifically for:

• feature interface-vlan: allow the configuration of SVI (interface vlan)
• feature nv overlay: Enables the VXLAN feature.
• feature vn-segment-vlan-based: Configures the global mode for all VXLAN bridge domains.

LEAF-1(config)# feature interface-vlan
LEAF-1(config)# feature nv overlay
LEAF-1(config)# feature vn-segment-vlan-basedCode language: PHP (php)

• Create VLAN for endpoints and assign VNID ((vn-segement id) to the VLAN :

The VLAN represent the local representation for the bridge domain on the Leaf.
The VXLAN VNI is the global value for the VXLAN on every switch
Virtual Network Identifier (VNI) identify a VXLAN segment.

! create vlan
LEAF-1(config)# vlan 140

LEAF-1(config)# interface eth1/1
LEAF-1(config-if)# switchport access vlan 140

! assign vlan to vn-segment
LEAF-1(config-vlan)# vlan 140
LEAF-1(config-vlan)#  name L2-VNI-140-Tenant1
LEAF-1(config-vlan)#  vn-segment 50140Code language: PHP (php)

• Create interface Vlan as gateway for endpoints (On Leaf-1 and Leaf-2):

LEAF-1(config)# interface Vlan140
LEAF-1(config-if)#  no shutdown
LEAF-1(config-if)#  no ip redirects
LEAF-1(config-if)#  ip address 172.21.140.1/24Code language: PHP (php)

• Configure NVE interface (VTEP):

The NVE (Network Virtualization Edge) interface is a logical interface that will perform encapsulation and de-encapsulation of VXLAN packets.

The NVE interface will use loopback1 interface as source interface, which means the loopback address will be used as outer source IP address for the VXLAN packets.

The NVE is mapped to the VNI and the multicast group.

On Leaf-1:

LEAF-1(config)# interface nve1
LEAF-1(config-if-nve)#  no shutdown
LEAF-1(config-if-nve)#  source-interface loopback1
LEAF-1(config-if-nve)#  member vni 50140
LEAF-1(config-if-nve-vni)#  mcast-group 239.1.1.0


LEAF-1(config)# interface loopback1
LEAF-1(config-if)#  description VXLAN VTEP
LEAF-1(config-if)# ip pim sparse-mode
LEAF-1(config-if)# ip address 192.168.0.13/32
LEAF-1(config-if)# ip router ospf 1 area 0.0.0.0Code language: PHP (php)

on Leaf-2:

LEAF-2(config)# interface nve1
LEAF-2(config-if-nve)#  no shutdown
LEAF-2(config-if-nve)#  source-interface loopback1
LEAF-2(config-if-nve)#  member vni 50140
LEAF-2(config-if-nve-vni)#  mcast-group 239.1.1.0


LEAF-2(config)# interface loopback1
LEAF-2(config-if)#  description VXLAN VTEP
LEAF-2(config-if)# ip pim sparse-mode
LEAF-2(config-if)# ip address 192.168.0.14/32
LEAF-2(config-if)#  ip router ospf 1 area 0.0.0.0Code language: PHP (php)

Verifications and Packet flow:

• show nve vni: will display all NVE and its corresponding VNIs, multicast-group

learning mode (Dataplane) and BD which is the VLAN mapped to the VNI.

LEAF-01# show nve vni
Codes: CP - Control Plane        DP - Data Plane
       UC - Unconfigured         SA - Suppress ARP
       SU - Suppress Unknown Unicast
       Xconn - Crossconnect
       MS-IR - Multisite Ingress Replication

Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      50140    239.1.1.0         Up    DP   L2 [140]

Let’s check the packet flow for communication between “Linux” and “VPC2” that will cause the Leafs to learn other peer via Data plane:

1- Linux server wants to communicate with VPC2, Linux send an ARP request to discover the MAC of VPC2.

2- Once ARP is received on Leaf-1, it will flood the packet in the VLAN and also will encapsulate the ARP request in VXLAN and send it to multicast group which is configured for the VNI.

(The VLAN 140 is mapped to VNI 50140 (which also configured with mcast group 239.1.1.0, NVE1 is member of VNI 50140)

So, the ARP packet is encapsulated into VXLAN with Outer destination IP: 239.1.1.0, source IP: Lo1 of leaf-1, VNID 50140 and sent toward the RP.

3- The multicast PIM RP (spine-1 in our example) will receive the packet and it will forward a copy to every VTEP that has joined the multicast group (mcast group already configured and mapped to the VNI on leaf-2).

4- Leaf-2 receives the VXLAN packet (including original ARP request) and de-encapsulate it. It will first learn the MAC address of “Linux” pointing to newly learned VTEP of leaf-1. Leaf-2 will forward the ARP request to “VPC2”.

5- The VPC2 will respond with ARP reply. Leaf-2 will receive it and check it’s MAC address table and see that MAC of destination “Linux” is learned via the interface nve1 (learned via DP from the last ARP request).

➤ So, Leaf-2 will encapsulate the ARP reply in VXLAN and send it as unicast toward Leaf-1.

6- Once Leaf-1 receive the packet, de-encapsulate it, learn the MAC of “VPC2” and mapped it to nve1 interface.

At this point, both Leaf-1 and Leaf-2 learn via Data plane each peer VTEP and also the MAC which had communication.

It can be verified with the following commands:

• Show nve peers:

show nve peers will display the learned VTEP peers and learning type, in our case DP.

LEAF-01# show nve peers
Interface Peer-IP                                 State LearnType Uptime   Route
r-Mac
--------- --------------------------------------  ----- --------- -------- -----
------------
nve1      192.168.0.14                            Up    DP        4d00h    n/a

• Show mac address:

In the show mac-address table, we can see the MAC address table of the remote Endpoint associated with the Peer VTEP.

when a packet comes to Leaf-01 destined to the MAC 0050.0000.0800, it will be encapsulated in VXLAN and unicasted to the peer address which is already learned via Dataplane (with multicast).

LEAF-01# show mac address-table
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*  140     0050.0000.0700   dynamic  0         F      F    Eth1/7
*  140     0050.0000.0800   dynamic  0         F      F    nve1(192.168.0.14)
*  140     5001.0000.1b08   static   -         F      F    Vlan140
G    -     0000.2222.3333   static   -         F      F    sup-eth1(R)
G    -     5001.0000.1b08   static   -         F      F    sup-eth1(R)
G  140     5001.0000.1b08   static   -         F      F    sup-eth1(R)