Q in Q Dot1Q Tunnel Configuration in ACI

I- Access Policies configuration

We will configure the Interface eth1/17 to be an EDGE port of Dot1Q Tunnel, the initial status for the interface is illustrated below:

We see “Port mode is trunk”:

leaf1# show interface ethernet 1/17
Ethernet1/17 is up
admin state is up, Dedicated Interface
  Port description is leaf101_descr
  Hardware: 100/1000/10000/25000/auto Ethernet, address: a03d.6f2e.2f1f (bia a03d.6f2e.2f1f)
  MTU 9000 bytes, BW 10000000 Kbit, DLY 1 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, medium is broadcast
  Port mode is trunk
  full-duplex, 10 Gb/s, media type is 10G
  FEC (forward-error-correction) : disable-fec
  Beacon is turned off
  Auto-Negotiation is turned on
  Input flow-control is off, output flow-control is off
  Auto-mdix is turned off
  Rate mode is dedicated
  Switchport monitor is off
  EtherType is 0x8100
  EEE (efficient-ethernet) : n/a
  Last link flapped 1d14h

Step-1: Create L2 Interface Policy specifying the Dot1Q port type:

Step-2: Associate this L2 interface policy under the Interface Policy group:

After that, the interface went inactive since it’s not configured in Dot1Q tunnel yet, but we can see that port mode has changed –> “Port mode is qiq-tunnel-edgePort“

leaf1# show interface ethernet 1/17
Ethernet1/17 is down (inactive)
admin state is up, Dedicated Interface
  Port description is leaf101_descr
  Hardware: 100/1000/10000/25000/auto Ethernet, address: a03d.6f2e.2f1f (bia a03d.6f2e.2f1f)
  MTU 9000 bytes, BW 0 Kbit, DLY 1 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, medium is broadcast
  Port mode is qiq-tunnel-edgePort
  full-duplex, 10 Gb/s, media type is 10G
  FEC (forward-error-correction) : disable-fec
  Beacon is turned off
  Auto-Negotiation is turned on
  Input flow-control is off, output flow-control is off
  Auto-mdix is turned off
  Rate mode is dedicated
  Switchport monitor is off
  EtherType is 0x8100
  EEE (efficient-ethernet) : n/a
  Last link flapped 00:02:24

II- Tenant Configuration

Associating Ports with dot1Q tunnel

We create Dot1Q Tunnel, from tenant level configuration > Networking > Dot1Q Tunnels and right click “Create Dot1Q-Tunnel”:

• mainly, you need to specify the needed ports EDGE or Core (deepening on design) and associate them to this Tunnel.

III- Verification and Troubleshooting

We can see that a BD VNID was deployed representing the created tunnel and associated to previous added interface:

leaf1# show vlan extended | grep LearnDuty
 247  Tunnel_dot1Q_LearnDuty           vxlan-14712925   Eth1/17 
                 
leaf1# show vlan id 247 extended 
 VLAN Name                             Encap            Ports                    
 ---- -------------------------------- ---------------- ------------------------ 
 247  Tunnel_dot1Q_LearnDuty           vxlan-14712925   Eth1/17  


leaf1# show system internal epm vlan 247 
+----------+---------+-----------------+----------+------+----------+-----------
   VLAN ID    Type      Access Encap     Fabric    H/W id  BD VLAN    Endpoint  
                        (Type Value)     Encap                          Count   
+----------+---------+-----------------+----------+------+----------+-----------
 247        Tenant BD 802.1Q          0 14712925   224    247        2           Code language: PHP (php)

From Managed Object perspective on APIC or on leaf, we can verify the following MOs are created:

apic1# moquery -c fvTnlEpP
Total Objects shown: 1

# fv.TnlEpP
epgPKey          : uni/tn-LD/Tnlepg-Tunnel_dot1Q_LearnDuty
accEncap         : unknown
bcastP           : 225.1.44.64
bdDefDn          : 
bdDefStQual      : none
bdSegid          : 14712925
childAction      : 
crtrnEnabled     : no
ctxDefDn         : 
ctxDefStQual     : none
ctxPcTag         : 16386
ctxSeg           : 0
ctxSegid         : 2818048
deplSt           : deployable
descr            : 
dn               : uni/epp/tnlp-[uni/tn-LD/Tnlepg-Tunnel_dot1Q_LearnDuty]
enfPref          : hw
epgDn            : 
epgName          : Tunnel_dot1Q_LearnDuty
floodOnEncap     : disabled
fwdCtrl          : mac-learn-enable
intraSecConf     : 
l2FDSeg          : 0
l3CtxEncap       : unknown
lcOwn            : local
modTs            : 2024-08-11T11:47:02.392+02:00
monPolDn         : 
name             : 
nameAlias        : 
npName           : 
npNameAlias      : 
operSt           : allocated
ownerKey         : 
ownerTag         : 
pcEnfPref        : unenforced
pcTag            : 32770
pcTagAllocSrc    : idmanager
prefGrMemb       : exclude
prio             : unspecified
qiqL2ProtTunMask : cdp,lldp
rn               : tnlp-[uni/tn-LD/Tnlepg-Tunnel_dot1Q_LearnDuty]
scopeId          : 2818048
status           : 
tName            : 
tnNameAlias      : 
txId             : 2305843009233472726Code language: PHP (php)

leaf1# moquery -c l2RsPathDomAtt | grep 14712925 -B 3 -A 22
# l2.RsPathDomAtt
tDn                  : sys/conng/path-[eth1/17]
childAction          : 
dn                   : sys/ctx-[vxlan-2818048]/bd-[vxlan-14712925]/rspathDomAtt-[sys/conng/path-[eth1/17]]
forceResolve         : yes
lcOwn                : local
modTs                : 2024-08-11T11:47:02.429+02:00
monPolDn             : 
operSt               : up
operStQual           : unspecified
parentSKey           : 247
primaryEncap         : unknown
qinqCapable          : capable
rType                : mo
rn                   : rspathDomAtt-[sys/conng/path-[eth1/17]]
state                : formed
stateQual            : none
status               : 
tCl                  : nwPathEp
tSKey                : eth1/17
tType                : mo
type                 : native
vlanmgrCfgFailedBmp  : 
vlanmgrCfgFailedTs   : 00:00:00:00.000
vlanmgrCfgState      : 0Code language: PHP (php)

The physical Interface mode can be verified as well from MO:

leaf1# moquery -c l1.PhysIf -f  'l1.PhysIf.mode== "qiq-tunnel-edgePort"' 
Total Objects shown: 1

# l1.PhysIf
id                             : eth1/17
adminSt                        : up
autoNeg                        : on
breakT                         : nonbroken
bw                             : 0
childAction                    : 
delay                          : 1
descr                          : leaf101_descr
dfeDelayMs                     : 0
dn                             : sys/phys-[eth1/17]
dot1qEtherType                 : 0x8100
emiRetrain                     : disable
enablePoap                     : no
ethpmCfgFailedBmp              : 
ethpmCfgFailedTs               : 00:00:00:00.000
ethpmCfgState                  : 0
fcotChannelNumber              : Channel32
fecMode                        : inherit
inhBw                          : unspecified
isReflectiveRelayCfgSupported  : Supported
layer                          : Layer2
lcOwn                          : local
linkDebounce                   : 100
linkFlapErrorMax               : 30
linkFlapErrorSeconds           : 420
linkLog                        : default
mdix                           : auto
medium                         : broadcast
modTs                          : 2024-08-11T11:47:02.429+02:00
mode                           : qiq-tunnel-edgePort
monPolDn                       : uni/infra/moninfra-default
mtu                            : 9000
name                           : 
pathSDescr                     : 
portPhyMediaType               : auto
portT                          : leaf
prioFlowCtrl                   : auto
reflectiveRelayEn              : off
rn                             : phys-[eth1/17]
routerMac                      : not-applicable
snmpTrapSt                     : enable
spanMode                       : not-a-span-dest
speed                          : inherit
status                         : 
switchingSt                    : enabled
trunkLog                       : default
usage                          : epgCode language: PHP (php)

We also need to verify that Interface is UP and has the appropriate QinQ tunnel mode:

leaf1# show interface ethernet 1/17
Ethernet1/17 is up
admin state is up, Dedicated Interface
  Port description is leaf101_descr
  Hardware: 100/1000/10000/25000/auto Ethernet, address: a03d.6f2e.2f1f (bia a03d.6f2e.2f1f)
  MTU 9000 bytes, BW 10000000 Kbit, DLY 1 usec
  reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, medium is broadcast
  Port mode is qiq-tunnel-edgePort
  full-duplex, 10 Gb/s, media type is 10G
  FEC (forward-error-correction) : disable-fec
  Beacon is turned off
  Auto-Negotiation is turned on
  Input flow-control is off, output flow-control is off
  Auto-mdix is turned off
  Rate mode is dedicated
  Switchport monitor is off
  EtherType is 0x8100
  EEE (efficient-ethernet) : n/a
  Last link flapped 00:00:06
  Last clearing of "show interface" counters never

We can verify ETHPM and ELTMC to verify what actually getting pushed to leaf hardware:

leaf1# show system internal ethpm info interface ethernet 1/17

Admin Config Information:
  state(up), mode(dot1q-tunnel), mode_detail(dot1q-tunnel), speed(inherit), duplex(Auto), medium_db(120)
  layer(L2), dce-mode(edge), description(leaf101_descr),
  auto neg(on), auto mdix(on), beacon(off), num_of_si(0)
  medium(broadcast), snmp trap(on), MTU(9000),
  flowcontrol rx(off) tx(off), link debounce(100),
  span mode(0 - not a span-destination)
  delay(1), bw(10000000), rate-mode(dedicated)
  eee(n/a), eee_lpi(Aggressive), eee_latency(Variable)
  fabricpath enforce (DCE Core)(0) 
  load interval [1-3]: 30, 300, 0 (sec).
  Ethertype 0x8100 
  Slowdrain Congestion : mode core timeout[0], mode edge [0]
  Slowdrain Pause : mode core enabled [n] timeout[0]
  Slowdrain Pause : mode edge enabled [n] timeout[0]Code language: PHP (php)