ACI Micro EPGs forwarding
Contents
Basic Micro-EPG configuration
leaf1# show system internal epm endpoint ip 172.16.31.2
MAC : 0050.5**0.c**b ::: Num IPs : 1
IP# 0 : 172.16.31.2 ::: IP# 0 flags : peer-aged|sclass| ::: l3-sw-hit: No ::: sclass : 49154
Vlan id : 2 ::: Vlan vnid : 8547 ::: VRF name : bameur_MC:VRF_App
BD vnid : 16383955 ::: VRF vnid : 2850820
Phy If : 0x16000001 ::: Tunnel If : 0
Interface : port-channel2
Flags : 0x82004c25 ::: sclass : 16386 ::: Ref count : 5
EP Create Timestamp : 02/18/2024 13:41:32.100564
EP Update Timestamp : 02/18/2024 14:49:43.223444
EP Flags : local|vPC|peer-aged|IP|MAC|sclass|timer|mac-ckt|
leaf1# show system internal epm endpoint ip 172.16.31.12
MAC : 0050.5**0.2**2 ::: Num IPs : 1
IP# 0 : 172.16.31.12 ::: IP# 0 flags : peer-aged|sclass| ::: l3-sw-hit: No ::: sclass : 49154
Vlan id : 2 ::: Vlan vnid : 8547 ::: VRF name : bameur_MC:VRF_App
BD vnid : 16383955 ::: VRF vnid : 2850820
Phy If : 0x16000001 ::: Tunnel If : 0
Interface : port-channel2
Flags : 0x82004c25 ::: sclass : 16386 ::: Ref count : 5
EP Create Timestamp : 02/18/2024 14:21:02.037139
EP Update Timestamp : 02/18/2024 14:49:43.218700
EP Flags : local|vPC|peer-aged|IP|MAC|sclass|timer|mac-ckt|
leaf1# show vlan id 2 extended
VLAN Name Encap Ports
---- -------------------------------- ---------------- ------------------------
2 bameur_MC:My_App:App vlan-2155 Eth1/4, Eth1/6, Po1, Po2
Example-1: Traffic between VM-1 and Bridge domain gateway
On the leaf, if we make an ELAM packet capture we will see that the traffic sourced from VM-1 (172.16.31.2) toward its gateway (BD on the leaf: 172.16.31.254) hit the leaf with Encap 2155, which is the encap for the secondary Isolated VLAN:
--------------------------------------------------------------
Outer L2 Header
--------------------------------------------------------------
Destination MAC : 0022.B**8.19FF
Source MAC : 0050.5**0.CEAB
802.1Q tag is valid : yes( 0x1 )
CoS : 0( 0x0 )
Access Encap VLAN : 2155( 0x86B )
---------------------------------------------------------------
Outer L3 Header
---------------------------------------------------------------
L3 Type : IPv4
IP Version : 4
DSCP : 0
IP Packet Length : 84 ( = IP header(28 bytes) + IP payload )
Don't Fragment Bit : set
TTL : 64
IP Protocol Number : ICMP
IP CheckSum : 1465( 0x5B9 )
Destination IP : 172.16.31.254
Source IP : 172.16.31.2
...
==================================================================================================================================================
====
Contract Lookup ( FPC )
==================================================================================================================================================
====
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Contract Lookup Key
--------------------------------------------------------------------------------------------------------------------------------------------------
----
IP Protocol : ICMP( 0x1 )
L4 Src Port : 2048( 0x800 )
L4 Dst Port : 10552( 0x2938 )
sclass (src pcTag) : 49154( 0xC002 )
dclass (dst pcTag) : 1( 0x1 )
src pcTag is from local table : yes
derived from a local table on this node by the lookup of src IP or MAC
Unknown Unicast / Flood Packet : no
If yes, Contract is not applied here because it is flooded
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Contract Result
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Contract Drop : no
Contract Logging : no
Contract Applied : no
Contract Hit : yes
Contract Aclqos Stats Index : 81904
Example-2: Traffic between VM-1 and VM-2 (same uEPG)
Packet capture on leaf side (VM-1 to VM-2):
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Outer L2 Header
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Destination MAC : 0022.BDF8.19FF
Source MAC : 0050.5**0.C**B
802.1Q tag is valid : yes( 0x1 )
CoS : 0( 0x0 )
Access Encap VLAN : 2155( 0x86B )
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Outer L3 Header
--------------------------------------------------------------------------------------------------------------------------------------------------
----
L3 Type : IPv4
IP Version : 4
DSCP : 0
IP Packet Length : 84 ( = IP header(28 bytes) + IP payload )
Don't Fragment Bit : set
TTL : 64
IP Protocol Number : ICMP
IP CheckSum : 8151( 0x1FD7 )
Destination IP : 172.16.31.12
Source IP : 172.16.31.2
...
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Contract Lookup Key
--------------------------------------------------------------------------------------------------------------------------------------------------
----
IP Protocol : ICMP( 0x1 )
L4 Src Port : 2048( 0x800 )
L4 Dst Port : 29002( 0x714A )
sclass (src pcTag) : 49154( 0xC002 )
dclass (dst pcTag) : 49154( 0xC002 )
src pcTag is from local table : yes
derived from a local table on this node by the lookup of src IP or MAC
Unknown Unicast / Flood Packet : no
If yes, Contract is not applied here because it is flooded
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Contract Result
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Contract Drop : no
Contract Logging : no
Contract Applied : yes
Contract Hit : yes
Contract Aclqos Stats Index : 81875
Example-3: Traffic between VM-1 and VM-2 (different uEPG)
Now let’s put VM-1 and VM-2 in different micro EPG:
- VM 172.16.31.2 in uEPG Django:
- VM 172.16.31.2 in uEPG Flask:
If we rechecked the pctag assigned for each:
- VM-1 part of uEPG Django: sclass: 49154
- VM-2 part of uEPG Flask: sclass: 16387
leaf1# show system internal epm endpoint ip 172.16.31.2
MAC : 0050.5**0.c**b ::: Num IPs : 1
IP# 0 : 172.16.31.2 ::: IP# 0 flags : peer-aged|sclass| ::: l3-sw-hit: No ::: sclass : 49154
Vlan id : 2 ::: Vlan vnid : 8547 ::: VRF name : bameur_MC:VRF_App
BD vnid : 16383955 ::: VRF vnid : 2850820
Phy If : 0x16000001 ::: Tunnel If : 0
Interface : port-channel2
Flags : 0x82004c25 ::: sclass : 16386 ::: Ref count : 5
EP Create Timestamp : 02/18/2024 13:41:32.100564
EP Update Timestamp : 02/18/2024 15:15:07.271359
EP Flags : local|vPC|peer-aged|IP|MAC|sclass|timer|mac-ckt|
::::
leaf1# show system internal epm endpoint ip 172.16.31.12
MAC : 0050.5**0.2**2 ::: Num IPs : 1
IP# 0 : 172.16.31.12 ::: IP# 0 flags : peer-aged|sclass| ::: l3-sw-hit: No ::: sclass : 16387
Vlan id : 2 ::: Vlan vnid : 8547 ::: VRF name : bameur_MC:VRF_App
BD vnid : 16383955 ::: VRF vnid : 2850820
Phy If : 0x16000001 ::: Tunnel If : 0
Interface : port-channel2
Flags : 0x82004c25 ::: sclass : 16386 ::: Ref count : 5
EP Create Timestamp : 02/18/2024 14:21:02.037139
EP Update Timestamp : 02/18/2024 15:15:07.266787
EP Flags : local|vPC|peer-aged|IP|MAC|sclass|timer|mac-ckt|
You can notice that both are belong to the same PI VLAN 2 (base EPG PIVLAN or FD VLAN) and part of same BD VLAN 100 (VNID 16383955):
leaf1# show vlan id 2 extended
VLAN Name Encap Ports
---- -------------------------------- ---------------- ------------------------
2 bameur_MC:My_App:App vlan-2155 Eth1/4, Eth1/6, Po1, Po2
leaf1# show system internal epm vlan 2 detail
VLAN 2
VLAN type : FD vlan
hw id : 96 ::: sclass : 10
access enc : (802.1Q, 2155)
fabric enc : (VXLAN, 8547)
Object store EP db version : 24
BD vlan id : 100 ::: BD vnid : 16383955 ::: VRF vnid : 2850820
Valid : Yes ::: Incomplete : No ::: Learn Enable : Yes
pol_ctrl_flags: proxy_arp ::: dom_ctrl :
Endpoint count : 2 ::: Local Endpoint count : 2 On Peer Endpoint count 0
leaf1# show vlan id 100 extended
VLAN Name Encap Ports
---- -------------------------------- ---------------- ------------------------
100 bameur_MC:App_BD vxlan-16383955 Eth1/4, Eth1/6, Po1, Po2 Code language: PHP (php)
But from Contract policy enforcement, they are classified in different uEPG:
- VM-1 172.16.31.2 classified as pctag 49154 (Django uEPG)
- VM-2 172.16.31.12 classified as pctag 16387 (Flask uEPG)
As a result, without a contract between these uEPGs, traffic should be denied between these VM-1 and VM-2, let’s make a packet capture and confirm:
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Outer L2 Header
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Destination MAC : 0022.BDF8.19FF
Source MAC : 0050.56A0.CEAB
802.1Q tag is valid : yes( 0x1 )
CoS : 0( 0x0 )
Access Encap VLAN : 2155( 0x86B )
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Outer L3 Header
--------------------------------------------------------------------------------------------------------------------------------------------------
----
L3 Type : IPv4
IP Version : 4
DSCP : 0
IP Packet Length : 84 ( = IP header(28 bytes) + IP payload )
Don't Fragment Bit : set
TTL : 64
IP Protocol Number : ICMP
IP CheckSum : 39112( 0x98C8 )
Destination IP : 172.16.31.12
Source IP : 172.16.31.2 --------------------------------------------------------------------------------------------------------------------------------------------------
----
Contract Lookup Key
--------------------------------------------------------------------------------------------------------------------------------------------------
----
IP Protocol : ICMP( 0x1 )
L4 Src Port : 2048( 0x800 )
L4 Dst Port : 21549( 0x542D )
sclass (src pcTag) : 49154( 0xC002 )
dclass (dst pcTag) : 16387( 0x4003 )
src pcTag is from local table : yes
derived from a local table on this node by the lookup of src IP or MAC
Unknown Unicast / Flood Packet : no
If yes, Contract is not applied here because it is flooded
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Contract Result
--------------------------------------------------------------------------------------------------------------------------------------------------
----
Contract Drop : yes
Contract Logging : yes
Contract Applied : no
Contract Hit : yes
Contract Aclqos Stats Index : 81525 Code language: JavaScript (javascript)