Replace Leaf or Spine in Cisco ACI Fabric [Step By Step]

Why do you need to replace a Leaf or Spine in ACI Fabric

You probably should change a Leaf or Spine in your ACI Fabric for various reasons, for example:

• The node has a defect or hardware issue (SSD issue for example) and you need to RMA it.
• The node model or generation is old and maybe close to the end of support date.
• Or maybe an upgrade of the ACI Fabric nodes should to be done.

ACI Fabric Leaf/Spine Replacement Process

Step-1: Convert the node to ACI mode (In case It arrived in NX-OS mode)

• Generally a leaf or Spine from the RMA depot will be running NXOS software instead of ACI mode.

• So, the first step should be converting the switch from NX-OS mode to ACI mode, please refer to the following link to convert the switch to ACI mode in simple steps:

Step-2: Clean the Switch before adding it to the Fabric

In case the leaf or the spine is from a different fabric (not same configuration), you need to use clean the nodes in order to avoid mismatching parameters, you can this via the commands

Switch# acidiag touch clean
Switch# reload

or for older ACI releases:

Switch# setup-clean-config.sh
Switch# reload 

Step-3: Remove the Switch to replace from APIC “Remove from Controller”

At this stage the old switch is ready to be replaced and the new leaf switch is ready for registration, remove the leaf to be replaced from the APIC via GUI by selecting the ‘Remove from Controller’ option.

• The ‘Remove from Controller’ option will completely remove the node from the APIC, releasing the node ID, SN association, and TEP address which was assigned by the APIC.
  
  Remove from Controller option will wipe and reload the switch (clean reload).
  
  Note: you need to wait almost 10 min for the switch to be removed from the APIC. If you need to add same node after “removing from controller” you need to re-register it from: Fabric > Inventory > Fabric Membership > node pending registration.

• The ‘Decommission’ option is only used when the expectation is that the same node will rejoin the fabric with the same node ID and SN.
  

If the device to be replaced has not yet released its node ID, and a new switch is registered with the same node ID, a fault will be thrown referencing the fact that the ID is already associated to another leaf node. A node ID can be assigned to a single switch in a specific time.

Step-4: Connect the new leaf to the Fabric

Once the leaf switch to be replaced is removed on the Fabric Membership page, the new leaf can be connected to the fabric. Once the leaf is discovered by the APIC, it will show up in the Fabric Inventory under node pending registration tab. It’s now ready to be registered or commissioned in the Fabric.

• If the new node does not show up on the Fabric Membership submenu, there could be a cabling issue, to verify this, check the LLDP neighbors via the ‘show lldp neighbors detail’ command on the spine switches connecting to the newly attached leaf switch (or vice versa).

Note: If the infra VLAN is modified, all leaf nodes must be clean rebooted at the same time. If all leaf switches are not cleaned at the same time, a clean reloaded switch will come online and receive the old infra VLAN via LLDP from a not-yet-cleaned leaf, and the clean reloaded leaf will fail to register with the APIC.

Register the new node:

• Navigate to Fabric membership page and verify the SN of the new leaf to add in Node pending registration menu.
• Register the new Leaf/Spine by entering the node ID, name (should be the same as the old one)

Note For Spine replacement: If you use the same Node ID, the BGP route reflector Policy should be consistent, especially if you have only two spines, it is best practice to verify the BGP RR policy after replacement:

Navigate to ‘System > System Settings > BGP Route Reflectors’ under Route Reflector Nodes. Ensure at least two active Route Reflectors after the changes are completed.

• Reference: Cisco