Cisco ACI Internal Route maps Explained

Note: This post consist of some notes from the Cisco Live session “BRKACI-2642”.

I- Bridge Domain Advertisement:

• When a Bridge domain is deployed on the Leaf, a pervasive route is created,

LEAF_1# show ip route vrf PROD:Legacy_VRF 172.28.10.154

172.28.10.0/24, ubest/mbest: 1/0, attached, direct, pervasive
    *via 10.3.224.65%overlay-1, [1/0], 00:00:07, static, tag 4294967294
         recursive next hop: 10.3.224.65/32%overlay-1

In order to advertise a Bridge domain subnet, we basically have 3 options:

• Associate BD to L3OUT.
• Route-map (default export).
• Export the subnet under the External EPG.

As a result of The previous configuration, the Bridge Domain subnet is added to an internal route map used for redistribution to the L3OUT protocol.

Redistribution of Bridge Domain via OSPF/EIGRP L3OUT:

• When L3out is deployed:
  – A route map from static/direct is created (initially empty)
  – Redistribution of direct static routes to the L3OUT routing protocol is created.

For OSPF/EIGRP, the same route map is applied for redistribution of direct/static routes per VRF on the same Leaf, naming: exp-ctx-st-VRF_VNID:

LEAF_1# show ip ospf vrf PROD:PROD
 Redistributing External Routes from
   static route-map exp-ctx-st-2359296
   direct route-map exp-ctx-st-2359296

LEAF_1# show ip eigrp vrf PROD:PROD
 Redistributing External Routes from
   static route-map exp-ctx-st-2359296
   direct route-map exp-ctx-st-2359296

Verify the route map used for direct subnet redistribution:

LEAF_1# show route-map exp-ctx-st-2359296
route-map exp-ctx-st-2359296, deny, sequence 1
  Match clauses:
    tag: 4294967294
  Set clauses:
route-map exp-ctx-st-2359296, permit, sequence 15801
  Match clauses:
    ip address prefix-lists: IPv4-st18-2359296-exc-ext-inferred-export-dst
    ipv6 address prefix-lists: IPv6-deny-all

Redistribution of Bridge Domain via BGP L3OUT:

By default BGP redistributes all direct routes, then limit the routes with an outbound route-map per BGP neighbor or L3OUT (per neighbor starting from 4.2 release):

show bgp process vrf TK:VRF1
Redistribution
        direct, route-map permit-all

----- to check outbound route-map per neighbor --
LEAF1# show ip bgp neighbors vrf PROD:PROD
  BGP neighbor is x, remote AS 65001, ebgp link, Peer index 1
    Outbound route-map configured is exp-l3out-L3OUT_BGP-peer-2097152, handle obtained


LEAF1# show route-map exp-l3out-L3OUT_BGP-peer-2097152
route-map exp-l3out-L3OUT_BGP-peer-2097152, permit, sequence 15801
  Match clauses:
   ip address prefix-lists: IPv4-peer49157-2097152-exc-int-inferred-export-dst
   ipv6 address prefix-lists: IPv6-deny-all
  Set clauses:
route-map exp-l3out-L3OUT_BGP-peer-2097152, deny, sequence 16000
  Match clauses:
   route-type: direct
  Set clauses:

II- Export Route Control

In order to advertise (export) routes from ACI to the L3OUT routing protocol, various internal route maps need to be deployed to allow the redistribution.

• for OSPF/EIGRP L3OUT:

For OSPF/EIGRP the same route map is applied for redistribution of routing protocols per VRF on the same Leaf, exp-ctx-proto-VRF_VNID:

One difference between OSPF and EIGRP Export route maps is that EIGRP doesn’t support Transit Routing on the same LEAF. No equivalent filter like OSPF area-filter in EIGRP.

• for BGP L3OUT:

BGP has a route-map per L3Out. Starting from 4.2, route-map can be per BGP-peer.

border-leaf# show ip bgp neighbors vrf TK:VRF1
   BGP neighbor is 17.0.0.1, remote AS 65001, ebgp link, Peer index 1
      Outbound route-map configured is exp-l3out-L3OUT_BGP-peer-2097152, handle obtained

border-leaf# show route-map exp-l3out-L3OUT_BGP-peer-2097152

Note: For BGP the same outbound route-map used to advertise static/direct route is also used for ospf/eigrp.

III- Import Route Control

• OSPF:

Note that the route-map for the table does not only prevent the routes (with no import check) from being leaked from OSPF LSDB to the RIB, but also, block routes with the VRF tag to reach the RIB.

• BGP:

BGP uses an inbound route-map (per L3Out) instead of table-map, starting from release 4.2, the inbound route map can also be applied per BGP neighbor.

border-leaf# show ip bgp neighbors vrf TK:VRF1
   BGP neighbor is 17.0.0.1, remote AS 65001, ebgp link, Peer index 1
     Inbound route-map configured is imp-l3out-L3OUT_BGP-peer-2097152, handle obtained

IV- Shared Route Control Subnet

By default , Leaf2 will import routes with VRF2 route-target in VRF2 IPv4 BGP, since VRF 2 is deployed on the Leaf2.
In order to allow VRF Route leaking, VRF2 IPv4 BGP also will import routes with VRF1 route-target according to the shared route maps (based on a contract).

Verification:

V- Shared Security Import Subnet

• When leaking external routes between VRF via “Shared Route control Subnet”, The route will be leaked into routing table of the VRF. but no communication is allowed with the leaked subnet.

• Even if the contract is deployed between an EPG and External EPG (or Ext EPG and Ext EPG) and zoning rules are in place, the leaked subnet should be classified (L3OUT pctag) in order to enforce policy.

• Shared security import subnet allow the leaking of prefix-pctag mapping to other VRF (based on the Contract).

• In order to check the VRF VNID:

LEAF_1# show vrf common:Global-VRF detail extended
     Encap: vxlan-2356851

• In order to check the prefix – pctag mapping for a VRF:

vsh -c "show system internal policy-mgr prefix" | grep 2356851

Vrf-Vni VRF-Id Table-Id Table-State  VRF-Name                    Addr                                Class Shared Remote Complete
======= ======  =========== =======  ============================ ================================= ====== ====== ====== ========
2356851 8      0x8           Up     common:Global-VRF                                 10.112.1.0/24  18      True   True   False

Route Control Internal route map Summary

For OSPF L3OUT Internal route-maps:

Export:

• Redistribute static routes (include Static and Direct) into OSPF Routing protocol.
• Redistribute BGP routes into OSPF.
• Redistribute EIGRP routes into OSPF or Area-Filter in from another OSPF L3OUT.
  

Import:

• Table-map to prevent routes from being installed in the RIB, also, prevent route with VRF tag from being learned from External devices
• Area-filter out to other OSPF Areas in the same Leaf. A route that was block from being imported should not be leaked to other OSPF Areas.

For EIGRP L3OUT Internal route-maps:

Export:

• Redistribute static and direct routes into EIGRP.
• Redistribute BGP into EIGRP.
• Redistributed another L3OUT protocol into EIGRP (from another EIGRP or OSPF)

Import:

Import control is not support for EIGRP, an internal route-map (used by table-map) is just used for prevent internal route from being learned from outside (according to VRF tag)

For BGP L3OUT internal route maps:

• In order to enable MP-BGP vpnv4 external route propagation inside ACI Fabric, by default, a permit-all is used to redistribute L3OUT routing protocol into BGP IPv4 (from OSPF or EIGRP).

• By default, BD subnets (direct routes) are redistributed into BGP via a permit all route-map. Then we have to limit BD subnets from being propagated via MP-BGP vpnv4 by preventing the route import with an interleak route-map which will deny pervasive route. BD subnets (direct routes) should only being distributed via APIC MO.

Export:

Outbound route-map which will include all subnets (from BD, routing protocols) to be advertised to BGP neighbors (it can be granular per BGP neighbor).

Import:

Inbound route-map to limit the routes learned from a BGP neighbor.