ACI L2 Multicast Configuration and Troubleshooting
- Basically, L2 Multicast in ACI assumes that PIM isn’t enabled under the BD (If PIM enabled, even for Multicast traffic within the same BD, it will be routed)
L2 Multicast configuration
- There is no much configuration to be done for L2 multicast in ACI, since it will leverage the BD GiPo which is already built in via Fabric ISIS FTAG tree.
Mainly, you need to configure IGMP policy (default can be used), IGMP snooping policy and enable IGMP Querier:
- Configure IGMP Snooping Policy
Navigate to Policies > Protocols > IGMP snooping, right-click and select Create IGMP Snoop Policy
If BD is used as the IGMP Querier, then you need to enable checkbox “Enable querier”:
Under the Bridge domain, apply the previously created IGMP policy:
Don’t forget to enable querier under BD subnet (GW) in case the BD will be used as querier (this is not needed if you have an external querier)
Verifications and troubleshooting
- ELAM Capture on the leaf where source is connected:
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L3 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L3 Type : IPv4
IP Version : 4
DSCP : 0
IP Packet Length : 1498 ( = IP header(28 bytes) + IP payload )
Don't Fragment Bit : set
TTL : 32
IP Protocol Number : UDP
IP CheckSum : 45009( 0xAFD1 )
Destination IP : 239.251.1.1
Source IP : 172.16.31.2
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L4 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L4 Type : UDP
Source Port : 47372( 0xB90C )
Destination Port : 5001( 0x1389 )
TCP/UDP CheckSum : 0x895D( 0x895D ) In the ELAM report, if we look for FTAG, we will see an ID from 0-15:
module-1(DBG-elam-insel6)# ereport | grep FTAG
FTAG : 10( 0xA ) In our case, the multicast flow was sent via the FTAG tree 10,
- Verify the FTAG tree:
To find the FTAG, we have multiple ways, mainly three:
1- You can verify from Leaf CLI:
Leaf-2# show isis internal mcast routes ftag | grep "FTAG ID: 10" -A 5
FTAG ID: 10 [Enabled] Cost:( 1/ 6/ 0)
----------------------------------
Root port: Ethernet1/50.216
OIF List:
Ethernet1/49.215Code language: PHP (php)
The Root for this FTAG Tree (10) is Spine-2, you can verify with LLDP:
leaf2# show lldp neighbors
Capability codes:
(R) Router, (B) Bridge, (T) Telephone, (C) DOCSIS Cable Device
(W) WLAN Access Point, (P) Repeater, (S) Station, (O) Other
Device ID Local Intf Hold-time Capability Port ID
spine1 Eth1/49 120 BR Eth1/2
spine2 Eth1/50 120 BR Eth1/2
Total entries displayed: 12Code language: PHP (php)
2- You can also use the following FTAG tree viewer script (available on GitHub from the following link):
It’s easy to run and help with troubleshooting BUM traffic or L2 multicast traffic, it will draw on the FTAG when you run the script on APIC and specifying parameter including pod id and the FTAG id, below an example for FTAG 10 in Pod-1:
apic1# python ftag-tree.py --pod 1 --ftag 10
################################################################################
# Pod 1 FTAG 10
# Root spine-202
# active nodes: 6, inactive nodes: 0
################################################################################
spine-202
+- 1/1 -------- 1/50 leaf-101
+- 1/2 -------- 1/50 leaf-102
| +- 1/49 -------- 1/2 spine-201
| +- 1/25 ...... (EXT) Eth1/25 spine3
| +- 1/26 ...... (EXT) Eth1/25 spine4
|
+- 1/3 -------- 1/50 leaf-103
+- 1/4 -------- 1/50 leaf-104
+- 1/5 xxxxxxxxxxx ? -?
+- 1/25 ...... (EXT) Eth1/26 spine3
+- 1/26 ...... (EXT) Eth1/26 spine4Code language: PHP (php)
3- But, I think the easiest way is to check the FTAG tree without script is via APIC GUI, you can verify from the first node where traffic is hit and follow the FTAG tree ID:
Example below, we check the FTAG 10 for leaf-2 where the ELAM was triggered and we can see the Root port pointing to Spine-2:
So, the leaf receiving the multicast traffic from source will encapsulate it in VXLAN and send to Spine-2 as per hashed FTAG calculation with resulted with ID 10, the VXLAN packet have the following header:
The Inner IP header:
- Source Outer: Source IP
- Destination Outer IP: destination multicast group traffic is being sent to
The outer IP header:
- Outer Source IP: PTEP of the leaf from traffic is generated (could be vPC VIP is case of vPC)
- Outer Destination IP: Bridge domain multicast Gipo (Multicast group subnet + FTAG ID)
We can verify via ELAM on the spine:
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L3 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L3 Type : IPv4
DSCP : 0
Don't Fragment Bit : 0x0
TTL : 30
IP Protocol Number : UDP
Destination IP : 225.0.138.186
Source IP : 10.0.128.68
------------------------------------------------------------------------------------------------------------------------------------------------------
Inner L3 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L3 Type : IPv4
DSCP : 0
Don't Fragment Bit : 0x1
TTL : 32
IP Protocol Number : UDP
Destination IP : 239.251.1.1
Source IP : 172.16.31.2
------------------------------------------------------------------------------------------------------------------------------------------------------
Outer L4 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L4 Type : iVxLAN
Don't Learn Bit : 0
Src Policy Applied Bit : 0
Dst Policy Applied Bit : 0
sclass (src pcTag) : 0xc009
VRF or BD VNID : 15335397( 0xE9FFE5 )
------------------------------------------------------------------------------------------------------------------------------------------------------
Inner L4 Header
------------------------------------------------------------------------------------------------------------------------------------------------------
L4 Type : UDP
Source Port : 58993
Destination Port : 5001 Code language: PHP (php)
We see Destination outer IP: 225.0.138.186, which is BD GiPo subnet + FTAG id (10):
# fv.BDDef
bdDn : uni/tn-bameur_MC/BD-App_BD
isSvc : no
OptimizeWanBandwidth : no
accEncap : unknown
arpFlood : yes
bcastP : 225.0.138.176Code language: PHP (php)
Also, as mentioned, you can follow from GUI, from the following Root Port unspecified means that Spine-2 is the root for this FTAG tree:
Lastly, with IGMP snooping and Querier enabled along with L3 unknown Multicast Flooding set to Optimized flow, traffic will be sent only to receiver ports:
This option will optimize flooding based IGMP snooping groups
