Cisco ACI Bridge Domain Legacy Mode

Scaled L2 Only Mode (Also known as Legacy Mode)

In Cisco Application Centric Infrastructure (ACI), the same VLAN ID can be reused for any purpose as long as the VLAN is deployed on different leaf nodes. This allows the Cisco ACI fabric to overcome the theoretical maximum number of VLANs 4094 as a fabric. However, to accomplish this, and also to hide the complexity of underlying VxLAN implementation, each individual leaf node can contain smaller number of VLANs.

This may pose a problem when the density of VLANs per leaf node is required. In such a scenario, you can enable Scaled L2 Only mode, formerly known as legacy mode on the bridge domain. A bridge domain in scaled L2 only mode allows large number of VLANs per leaf node. However, such a bridge domain has some limitations.

Legacy mode bridge domain is intended for a specific use case that requires higher bridge domain (VLAN) numbers per switch: As of Cisco APIC release 4.2, ~2000 normal bridge domains can be deployed on the same leaf switch, while 3500 legacy mode bridge domains can be deployed on the same leaf switch. However, as a trade-off for the bridge domain (VLAN) numbers, legacy mode bridge domains lose various Cisco ACI-specific capabilities, such as contracts, pervasive gateway (bridge domain subnet).

Please refer to scalability matrix for specific version to get the precise values.

Limitations for Scaled L2 Only Mode

The following are limitations for legacy mode or scaled L2 only mode.

• The bridge domain can contain only one EPG and one VLAN.
• Unicast routing is not supported.
• Contracts are not supported.
• Dynamic VLAN allocation for VMM integration is not supported.
• Service graph is not supported.
• A QoS policy is not supported.
• The bridge domain essentially behaves as a VLAN in standalone Cisco NX-OS

Scaled L2 Only Mode Configuration

Bridge domain legacy mode allows only one VLAN per bridge domain. When bridge domain legacy mode is specified, bridge domain encapsulation is used for all EPGs that reference the bridge domain. EPG encapsulation, if defined, is ignored. Unicast routing does not apply for bridge domain legacy mode. A leaf switch can be configured with multiple bridge domains that operate in a mixture of legacy or normal modes. However, once a bridge domain is configured, its mode cannot be switched.

• VLAN ID is configured on the bridge domain.

• VLAN IDs configured under the EPG are overridden.

• Enabling or disabling a scaled L2 only mode on an existing bridge domain will impact service.

Cisco Application Policy Infrastructure Controller (APIC) will automatically undeploy and redeploy the bridge domain when the VLAN ID is different from what was used prior to the change.

When the same VLAN ID is used before and after the mode change, Cisco APIC will not automatically undeploy and redeploy the bridge domain. You must manually undeploy and re-deploy the bridge domain, which can be performed by deleting and recreating the static port configuration under the EPG.

When changing the VLAN ID for scaled L2 only mode, you must first disable the mode, then enable scaled L2 only mode with the new VLAN ID.

Let’s take an example of BD Legacy mode configuration and highlight the basic difference with regular BD:

I- Example of Regular BD (without Legacy mode enable)

In regular BD, the Encap Vlan is associated to the EPG:

leaf2# show vlan encap-id 1093

 VLAN Name                             Status    Ports                           
 ---- -------------------------------- --------- ------------------------------- 
 238  bameur_MC:App-1:EPG-1            active    Eth1/17 

 VLAN Type  Vlan-mode  
 ---- ----- ---------- 
 238  enet  CE         

leaf2# show vlan id 238 extended 

 VLAN Name                             Encap            Ports                    
 ---- -------------------------------- ---------------- ------------------------ 
 238  bameur_MC:App-1:EPG-1            vlan-1093        Eth1/17                  
Code language: PHP (php)

The EPG is mapped to BD, the BD vlan doesn’t have any Access Encap associated directly to it :

leaf2# show system internal epm vlan 238
+----------+---------+-----------------+----------+------+----------+-----------
   VLAN ID    Type      Access Encap     Fabric    H/W id  BD VLAN    Endpoint  
                        (Type Value)     Encap                          Count   
+----------+---------+-----------------+----------+------+----------+-----------
 238          FD vlan 802.1Q       1093 9285       129    151        0         


leaf2# show system internal epm vlan 151
+----------+---------+-----------------+----------+------+----------+-----------
   VLAN ID    Type      Access Encap     Fabric    H/W id  BD VLAN    Endpoint  
                        (Type Value)     Encap                          Count   
+----------+---------+-----------------+----------+------+----------+-----------
 151        Tenant BD NONE            0 15335397   160    151        0    


bdsol-aci02-leaf2# show vlan id 151 extended 

 VLAN Name                             Encap            Ports                    
 ---- -------------------------------- ---------------- ------------------------ 
 151  bameur_MC:App_BD                 vxlan-15335397   Eth1/4, Eth1/6, Eth1/17, 
                                                        Po2, Po5                     Code language: PHP (php)

we can confirm from the following output that BD Vlan access enc property set to NONE:

bdsol-aci02-leaf2# show system internal epm vlan 151 detail

VLAN 151
VLAN type : Tenant BD
hw id : 160 ::: sclass : 16393
access enc : (NONE, 0)
fabric enc : (VXLAN, 15335397)
Object store EP db version : 1
BD vlan id : 151 ::: BD vnid : 15335397 ::: VRF vnid : 2359298
Valid : Yes ::: Incomplete : No  ::: Learn Enable : Yes
EP retention policy valid : Yes
Local EP timeout : 900 ::: Remote EP timeout : 300
EP bounce timeout : 630 ::: EP hold timeout : 300
EP move frequency : 256
fwd_mode : route,bridge ::: fwd_ctrl : mdst-flood,arp-flood,ip-lrn-pfx-check,
bridge_mode: mac ::: unk_mac_ucast: flood ::: dom_ctrl : 
is_sisf_enabled : No ::: fhs_mode : 
HSRP vmac announce : Enabled ::: EP announce : Disabled
Endpoint count : 0 ::: Local Endpoint count : 0 On Peer Endpoint count 0
SVC MGR Registered EP count : 0
BD Subnet ip_pfx-1 : 172.16.31.254/24

v4 Subnets from PT tree - 
BD : 151 ::: Prefix type : BD subnet ::: Prefix : 172.16.31.0/24 (172.16.31.254) ::: Learn Disable : False
::::Code language: PHP (php)

II- Example of Bridge Domain configured as legacy BD

Under the BD Policy configuration, you can enable Legacy mode:

And you can see the following definition and limitation when applying the configuration:

This mode is to achieve a higher VLAN number per switch.

Please review the following trade-off limitations:

• The BD will only support a single VLAN and a single EPG
• Any static VLAN binding will be overridden
• Unicast Routing will be ignored and always disabled
• Various ACI features will not be supported, such as:
  • Dynamic VLAN allocation for VMM domains
  • Contracts
  • Services Graphs
• Changing this mode will reprogram the BD which will result in traffic disruption

When BD with Legacy mode enabled, the EPG and BD Vlan are merged. Technically, only the VLAN encap specified under the BD is considered (EPG VLAN is overridden if the specified encap vlan is different).

Enabling BD Legacy Mode basically make ACI leaf acting like NXOS switch in some aspects, leaf doesn’t need to push two PIVLAN for EPG and BD. Only 1 PIVLAN for BD is needed.

We can see it from the following ouptuts:

leaf2# show vlan encap-id 1093

 VLAN Name                             Status    Ports                           
 ---- -------------------------------- --------- ------------------------------- 
 213  bameur_MC:Legacy_BD              active    Eth1/17 

 VLAN Type  Vlan-mode  
 ---- ----- ---------- 
 213  enet  CE         Code language: PHP (php)

We can see the BD PIVLAN is directly mapped to Encap VLAN:

leaf2# show vlan id 213 extended 

 VLAN Name                             Encap            Ports                    
 ---- -------------------------------- ---------------- ------------------------ 
 213  bameur_MC:Legacy_BD              vxlan-16383915,  Eth1/17                  
                                       vlan-1093                                 Code language: PHP (php)

Also, can be verified from following output that BD is associated to the Access Encap VLAN:

bdsol-aci02-leaf2# show system internal epm vlan 213        


+----------+---------+-----------------+----------+------+----------+-----------
   VLAN ID    Type      Access Encap     Fabric    H/W id  BD VLAN    Endpoint  
                        (Type Value)     Encap                          Count   
+----------+---------+-----------------+----------+------+----------+-----------
 213        Tenant BD 802.1Q       1093 16383915   129    213        0         Code language: PHP (php)

There is no PIVLAN VLAN for EPG Encap:

bdsol-aci02-leaf2# show system internal epm vlan 213 detail 

VLAN 213
VLAN type : Tenant BD
hw id : 129 ::: sclass : 16390
access enc : (802.1Q, 1093)
fabric enc : (VXLAN, 16383915)
Object store EP db version : 0
BD vlan id : 213 ::: BD vnid : 16383915 ::: VRF vnid : 2359298
Valid : Yes ::: Incomplete : No  ::: Learn Enable : Yes
EP retention policy valid : Yes
Local EP timeout : 900 ::: Remote EP timeout : 300
EP bounce timeout : 630 ::: EP hold timeout : 300
EP move frequency : 256
fwd_mode : bridge ::: fwd_ctrl : mdst-flood,arp-flood,ip-lrn-pfx-check,
bridge_mode: mac ::: unk_mac_ucast: proxy ::: dom_ctrl : 
is_sisf_enabled : No ::: fhs_mode : 
HSRP vmac announce : Enabled ::: EP announce : Disabled
Endpoint count : 0 ::: Local Endpoint count : 0 On Peer Endpoint count 0
SVC MGR Registered EP count : 0

v4 Subnets from PT tree - 
::::Code language: CSS (css)

module-1# show system internal eltmc info vlan access_encap_vlan 1093


             vlan_id:            213   :::      hw_vlan_id:            129
           vlan_type:        BD_VLAN   :::         bd_vlan:            213
   access_encap_type:         802.1q   :::    access_encap:           1093
   fabric_encap_type:          VXLAN   :::    fabric_encap:       16383915
              sclass:          16390   :::           scope:        2359298
            untagged:              0   :::          seclbl:              3
     acess_encap_hex:          0x445   :::  fabric_enc_hex:       0xf9ffab
        vlan_fd_list: 
             context:bameur_MC:VRF-MC3
     pd_vlan_ft_mask:            0x8
       learn_disable:              0
        qos_class_id:              0   :::      qos_pap_id:              0
          qq_met_ptr:            177   :::      ipmc_index:              0
   ingressBdAclLabel:        0x81400   ::: ingBdAclLblMask:        0xc5600
    egressBdAclLabel:              0   ::: egrBdAclLblMask:              0
        rwBdAclLabel:              0   :::  rwBdAclLblMask:              0
         qos_map_idx:              0   :::     qos_map_pri:              0
        qos_map_dscp:              0   :::      qos_map_tc:              0
        vlan_ft_mask:        0xa09ff   :::       sisf_bmap:              0
            fwd_mode:         bridge
            arp_mode:          flood
            ip_learn:              0
           mac_learn:              1
        unk_mc_flood:              1
     copy_service_bd:              0
arp_bd_smac_mcast_act:              1
    qinq provider bd:              0   :::  pgmQiqEpgLabel:              0
         unk_uc_mode:          proxy
          multi_dest:          flood
 ep_move_detect_mode:      undefined
           hw_bd_idx:            646   :::      hw_epg_idx:          11399
          intf_count:              1   ::: glbl_scp_if_cnt:              1
    hasTransparentEp:              0

          [SDK Info]:
             vlan_id:            213   :::             fid:            646
              hwBdId:            646   :::         hwEpgId:          11399
              bdType:              1   :::  unknownmcflood:              1
        accencaptype:              0   :::    fabencaptype:              2
         accencapval:           1093   :::     fabencapval:       16383915
      srcpolicyincom:              1   :::  forcemacsalkup:              0
              sclass:          16390   :::         sglabel:              3
          sclassprio:              2   :::    arpunicastmd:              0
         v4addrfwdmd:              0   :::     v6addrfwdmd:              0
         unknucflood:              0   :::     unknucproxy:              1
          maclearnen:              1   :::       iplearnen:              0
         sclasslrnen:              1   :::  forcemacsalkup:              0
   ingressBdAclLabel:              0   ::: ingBdAclLblMask:              0
    egressBdAclLabel:              0   ::: egrBdAclLblMask:              0
        rwBdAclLabel:              0   :::  rwBdAclLblMask:              0
     encfldfwdlkupen:              1   ::: encfldfwdrslten:              1
         floodmetptr:            177   :::        ivxlandl:              0
       rarprequnidis:              1   :::   rarpresunidis:              1
       garpucmodedis:              0   :::    gnaucmodedis:              1
       l3bindchecken:              0   ::: fibsalkupusevrf:              1
      qinqproviderbd:              0   :::    hwshadowbdid:              0
    hasTransparentEp:              0   :::      isLegacyBD:              1
       l2bindchecken:              1

[SDB INFO]:
           vlan_type:             11
        access encap:           1093
        fabric encap:       16383915
             bd vlan:            213
        hw vlan vlan:            129
               scope:        2359298
              sclass:          16390Code language: CSS (css)