Cisco ACI Bridge Domain Legacy Mode
Contents
Scaled L2 Only Mode (Also known as Legacy Mode)
In Cisco Application Centric Infrastructure (ACI), the same VLAN ID can be reused for any purpose as long as the VLAN is deployed on different leaf nodes. This allows the Cisco ACI fabric to overcome the theoretical maximum number of VLANs 4094 as a fabric. However, to accomplish this, and also to hide the complexity of underlying VxLAN implementation, each individual leaf node can contain smaller number of VLANs.
This may pose a problem when the density of VLANs per leaf node is required. In such a scenario, you can enable Scaled L2 Only mode, formerly known as legacy mode on the bridge domain. A bridge domain in scaled L2 only mode allows large number of VLANs per leaf node. However, such a bridge domain has some limitations.
Legacy mode bridge domain is intended for a specific use case that requires higher bridge domain (VLAN) numbers per switch: As of Cisco APIC release 4.2, ~2000 normal bridge domains can be deployed on the same leaf switch, while 3500 legacy mode bridge domains can be deployed on the same leaf switch. However, as a trade-off for the bridge domain (VLAN) numbers, legacy mode bridge domains lose various Cisco ACI-specific capabilities, such as contracts, pervasive gateway (bridge domain subnet).
Please refer to scalability matrix for specific version to get the precise values.
Limitations for Scaled L2 Only Mode
The following are limitations for legacy mode or scaled L2 only mode.
- The bridge domain can contain only one EPG and one VLAN.
- Unicast routing is not supported.
- Contracts are not supported.
- Dynamic VLAN allocation for VMM integration is not supported.
- Service graph is not supported.
- A QoS policy is not supported.
- The bridge domain essentially behaves as a VLAN in standalone Cisco NX-OS
Scaled L2 Only Mode Configuration
Bridge domain legacy mode allows only one VLAN per bridge domain. When bridge domain legacy mode is specified, bridge domain encapsulation is used for all EPGs that reference the bridge domain. EPG encapsulation, if defined, is ignored. Unicast routing does not apply for bridge domain legacy mode. A leaf switch can be configured with multiple bridge domains that operate in a mixture of legacy or normal modes. However, once a bridge domain is configured, its mode cannot be switched.
- VLAN ID is configured on the bridge domain.
- VLAN IDs configured under the EPG are overridden.
- Enabling or disabling a scaled L2 only mode on an existing bridge domain will impact service.
Cisco Application Policy Infrastructure Controller (APIC) will automatically undeploy and redeploy the bridge domain when the VLAN ID is different from what was used prior to the change.
When the same VLAN ID is used before and after the mode change, Cisco APIC will not automatically undeploy and redeploy the bridge domain. You must manually undeploy and re-deploy the bridge domain, which can be performed by deleting and recreating the static port configuration under the EPG.
When changing the VLAN ID for scaled L2 only mode, you must first disable the mode, then enable scaled L2 only mode with the new VLAN ID.
Let’s take an example of BD Legacy mode configuration and highlight the basic difference with regular BD:
I- Example of Regular BD (without Legacy mode enable)
In regular BD, the Encap Vlan is associated to the EPG:
leaf2# show vlan encap-id 1093
VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
238 bameur_MC:App-1:EPG-1 active Eth1/17
VLAN Type Vlan-mode
---- ----- ----------
238 enet CE
leaf2# show vlan id 238 extended
VLAN Name Encap Ports
---- -------------------------------- ---------------- ------------------------
238 bameur_MC:App-1:EPG-1 vlan-1093 Eth1/17
Code language: PHP (php)
The EPG is mapped to BD, the BD vlan doesn’t have any Access Encap associated directly to it :
leaf2# show system internal epm vlan 238
+----------+---------+-----------------+----------+------+----------+-----------
VLAN ID Type Access Encap Fabric H/W id BD VLAN Endpoint
(Type Value) Encap Count
+----------+---------+-----------------+----------+------+----------+-----------
238 FD vlan 802.1Q 1093 9285 129 151 0
leaf2# show system internal epm vlan 151
+----------+---------+-----------------+----------+------+----------+-----------
VLAN ID Type Access Encap Fabric H/W id BD VLAN Endpoint
(Type Value) Encap Count
+----------+---------+-----------------+----------+------+----------+-----------
151 Tenant BD NONE 0 15335397 160 151 0
bdsol-aci02-leaf2# show vlan id 151 extended
VLAN Name Encap Ports
---- -------------------------------- ---------------- ------------------------
151 bameur_MC:App_BD vxlan-15335397 Eth1/4, Eth1/6, Eth1/17,
Po2, Po5 Code language: PHP (php)
we can confirm from the following output that BD Vlan access enc property set to NONE:
bdsol-aci02-leaf2# show system internal epm vlan 151 detail
VLAN 151
VLAN type : Tenant BD
hw id : 160 ::: sclass : 16393
access enc : (NONE, 0)
fabric enc : (VXLAN, 15335397)
Object store EP db version : 1
BD vlan id : 151 ::: BD vnid : 15335397 ::: VRF vnid : 2359298
Valid : Yes ::: Incomplete : No ::: Learn Enable : Yes
EP retention policy valid : Yes
Local EP timeout : 900 ::: Remote EP timeout : 300
EP bounce timeout : 630 ::: EP hold timeout : 300
EP move frequency : 256
fwd_mode : route,bridge ::: fwd_ctrl : mdst-flood,arp-flood,ip-lrn-pfx-check,
bridge_mode: mac ::: unk_mac_ucast: flood ::: dom_ctrl :
is_sisf_enabled : No ::: fhs_mode :
HSRP vmac announce : Enabled ::: EP announce : Disabled
Endpoint count : 0 ::: Local Endpoint count : 0 On Peer Endpoint count 0
SVC MGR Registered EP count : 0
BD Subnet ip_pfx-1 : 172.16.31.254/24
v4 Subnets from PT tree -
BD : 151 ::: Prefix type : BD subnet ::: Prefix : 172.16.31.0/24 (172.16.31.254) ::: Learn Disable : False
::::Code language: PHP (php)II- Example of Bridge Domain configured as legacy BD
Under the BD Policy configuration, you can enable Legacy mode:
And you can see the following definition and limitation when applying the configuration:
This mode is to achieve a higher VLAN number per switch.
Please review the following trade-off limitations:
- The BD will only support a single VLAN and a single EPG
- Any static VLAN binding will be overridden
- Unicast Routing will be ignored and always disabled
- Various ACI features will not be supported, such as:
- Dynamic VLAN allocation for VMM domains
- Contracts
- Services Graphs
- Changing this mode will reprogram the BD which will result in traffic disruption
When BD with Legacy mode enabled, the EPG and BD Vlan are merged. Technically, only the VLAN encap specified under the BD is considered (EPG VLAN is overridden if the specified encap vlan is different).
Enabling BD Legacy Mode basically make ACI leaf acting like NXOS switch in some aspects, leaf doesn’t need to push two PIVLAN for EPG and BD. Only 1 PIVLAN for BD is needed.
We can see it from the following ouptuts:
leaf2# show vlan encap-id 1093
VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
213 bameur_MC:Legacy_BD active Eth1/17
VLAN Type Vlan-mode
---- ----- ----------
213 enet CE Code language: PHP (php)
We can see the BD PIVLAN is directly mapped to Encap VLAN:
leaf2# show vlan id 213 extended
VLAN Name Encap Ports
---- -------------------------------- ---------------- ------------------------
213 bameur_MC:Legacy_BD vxlan-16383915, Eth1/17
vlan-1093 Code language: PHP (php)
Also, can be verified from following output that BD is associated to the Access Encap VLAN:
bdsol-aci02-leaf2# show system internal epm vlan 213
+----------+---------+-----------------+----------+------+----------+-----------
VLAN ID Type Access Encap Fabric H/W id BD VLAN Endpoint
(Type Value) Encap Count
+----------+---------+-----------------+----------+------+----------+-----------
213 Tenant BD 802.1Q 1093 16383915 129 213 0 Code language: PHP (php)
There is no PIVLAN VLAN for EPG Encap:
bdsol-aci02-leaf2# show system internal epm vlan 213 detail
VLAN 213
VLAN type : Tenant BD
hw id : 129 ::: sclass : 16390
access enc : (802.1Q, 1093)
fabric enc : (VXLAN, 16383915)
Object store EP db version : 0
BD vlan id : 213 ::: BD vnid : 16383915 ::: VRF vnid : 2359298
Valid : Yes ::: Incomplete : No ::: Learn Enable : Yes
EP retention policy valid : Yes
Local EP timeout : 900 ::: Remote EP timeout : 300
EP bounce timeout : 630 ::: EP hold timeout : 300
EP move frequency : 256
fwd_mode : bridge ::: fwd_ctrl : mdst-flood,arp-flood,ip-lrn-pfx-check,
bridge_mode: mac ::: unk_mac_ucast: proxy ::: dom_ctrl :
is_sisf_enabled : No ::: fhs_mode :
HSRP vmac announce : Enabled ::: EP announce : Disabled
Endpoint count : 0 ::: Local Endpoint count : 0 On Peer Endpoint count 0
SVC MGR Registered EP count : 0
v4 Subnets from PT tree -
::::Code language: CSS (css)module-1# show system internal eltmc info vlan access_encap_vlan 1093
vlan_id: 213 ::: hw_vlan_id: 129
vlan_type: BD_VLAN ::: bd_vlan: 213
access_encap_type: 802.1q ::: access_encap: 1093
fabric_encap_type: VXLAN ::: fabric_encap: 16383915
sclass: 16390 ::: scope: 2359298
untagged: 0 ::: seclbl: 3
acess_encap_hex: 0x445 ::: fabric_enc_hex: 0xf9ffab
vlan_fd_list:
context:bameur_MC:VRF-MC3
pd_vlan_ft_mask: 0x8
learn_disable: 0
qos_class_id: 0 ::: qos_pap_id: 0
qq_met_ptr: 177 ::: ipmc_index: 0
ingressBdAclLabel: 0x81400 ::: ingBdAclLblMask: 0xc5600
egressBdAclLabel: 0 ::: egrBdAclLblMask: 0
rwBdAclLabel: 0 ::: rwBdAclLblMask: 0
qos_map_idx: 0 ::: qos_map_pri: 0
qos_map_dscp: 0 ::: qos_map_tc: 0
vlan_ft_mask: 0xa09ff ::: sisf_bmap: 0
fwd_mode: bridge
arp_mode: flood
ip_learn: 0
mac_learn: 1
unk_mc_flood: 1
copy_service_bd: 0
arp_bd_smac_mcast_act: 1
qinq provider bd: 0 ::: pgmQiqEpgLabel: 0
unk_uc_mode: proxy
multi_dest: flood
ep_move_detect_mode: undefined
hw_bd_idx: 646 ::: hw_epg_idx: 11399
intf_count: 1 ::: glbl_scp_if_cnt: 1
hasTransparentEp: 0
[SDK Info]:
vlan_id: 213 ::: fid: 646
hwBdId: 646 ::: hwEpgId: 11399
bdType: 1 ::: unknownmcflood: 1
accencaptype: 0 ::: fabencaptype: 2
accencapval: 1093 ::: fabencapval: 16383915
srcpolicyincom: 1 ::: forcemacsalkup: 0
sclass: 16390 ::: sglabel: 3
sclassprio: 2 ::: arpunicastmd: 0
v4addrfwdmd: 0 ::: v6addrfwdmd: 0
unknucflood: 0 ::: unknucproxy: 1
maclearnen: 1 ::: iplearnen: 0
sclasslrnen: 1 ::: forcemacsalkup: 0
ingressBdAclLabel: 0 ::: ingBdAclLblMask: 0
egressBdAclLabel: 0 ::: egrBdAclLblMask: 0
rwBdAclLabel: 0 ::: rwBdAclLblMask: 0
encfldfwdlkupen: 1 ::: encfldfwdrslten: 1
floodmetptr: 177 ::: ivxlandl: 0
rarprequnidis: 1 ::: rarpresunidis: 1
garpucmodedis: 0 ::: gnaucmodedis: 1
l3bindchecken: 0 ::: fibsalkupusevrf: 1
qinqproviderbd: 0 ::: hwshadowbdid: 0
hasTransparentEp: 0 ::: isLegacyBD: 1
l2bindchecken: 1
[SDB INFO]:
vlan_type: 11
access encap: 1093
fabric encap: 16383915
bd vlan: 213
hw vlan vlan: 129
scope: 2359298
sclass: 16390Code language: CSS (css)
