Cisco ACI Floating L3OUT [Explained & Configuration]

This post consists of some notes from Cisco ACI Floating L3OUT white Paper.

What is a Floating L3OUT

The floating L3Out feature enables you to configure a L3Out without specifying logical interfaces. The feature saves you from having to configure multiple L3Out logical interfaces to maintain routing when virtual machines move from one host to another. Floating L3Out is supported for VMware vSphere Distributed Switch (VDS).

For example, if you had a hypervisor cluster with 12 leaf switches, virtual machines could potentially move to every one of those 12 leaf switches. That meant that you had to create a policy to deploy an L3Out from every leaf node interface to every corresponding server.

• Floating L3OUT is supported for VMM domains (VDS) from release 4.2(1).
• Floating L3OUT is supported for Physical domains from release 5.0(1) with enhancements on 5.2 version.

Floating L3OUT Design and Components

Topology:

• Anchor nodes (Leaf-1 and Leaf-2) is where the routing session runs for L3Out peering. Anchor nodes have the primary IP address and floating IP address and can have a secondary IP and floating secondary IP, if needed.

• Non-anchor Node (Leaf-103 and Leaf-104): The non-anchor leaf node does not create any routing sessions for L3Out peering. It acts as a passthrough between the anchor node and the L3Out router.
  
  A non-anchor leaf node has the floating IP address and can have a floating secondary IP.
  
  The floating IP address is deployed only:
  
  – For VMM VDS: when the virtual router is connected to the leaf node (on a host connected to leaf node)
  – For physical domain: when the leaf port uses AEP that has an L3Out domain associated to the floating L3Out.

The floating IP address is the common IP address for non-anchor leaf nodes. It is used to locate the router virtual machine (VM) if it moves behind any non-anchor leaf node through the data path.

Once a virtual router moves to a host on a non-anchor node, Cisco Application Policy Infrastructure Controller (APIC) deploys the L3Out bridge domain on the new non-anchor leaf. It also installs the floating IP address and the data center routes and contracts for policy enforcement.

Traffic Flow:

* Before any virtual routers move: External traffic through an anchor node goes to the spine switch and then to the web endpoint in Host 4. Return traffic goes back to the virtual router through an anchor leaf node:

* If a virtual router moves to Host 3 under non-anchor Leaf3: External traffic comes to the fabric through Leaf-3 and then to the web endpoint in Host 4 through the spine switch. The return traffic goes back to an anchor leaf node, and then goes back to the virtual router:

This sub-optimal path can be avoided by using ACI 5.0 and later version.

Avoiding Suboptimal Traffic From an ACI Internal EP to a Floating L3Out

Avoiding this suboptimal path requires configuring next-hop propagation and direct host advertisement route-control profiles.

From release 5.0(1) to 5.2(1):

avoiding this suboptimal path using next-hop propagation is supported with BGP only. For those releases, local adjacency learned at the non-anchor leaf nodes is redistributed in BGP and is carried as a BGP route within the ACI fabric:

From release 5.2(1) and later:

TBC