IPv6 Neighbor Discovery in ACI Explained with Examples

Why NDP (Neighbor Discovery Protocol) is needed

The IPv6 Neighbor Discovery (ND) responsible for:

• Discovery of other nodes on the link (NS/NA)

• Address prefix discovery, Address auto configuration of nodes, Finding available routers and DNS servers (RS/RA)

• Duplicate Address Detection (DAD)

Neighbor Solicitation or Neighbor Advertisement (NS or NA) and Router Solicitation or Router Advertisement (RS or RA) packet are supported on L3 interfaces. But, RS/RA packets are used for auto configuration for all Layer 3 interfaces but are only configurable for pervasive SVIs.

Configuring IPv6 Neighbor Discovery on a Bridge Domain

Mainly 2 Policies for IPv6 ND on Bridge Domain:

• ND interface policy under BD.
• ND prefix policy under the subnet.

I- ND interface policy under BD:

ND interface policy is configured under the Bridge domain level. In APIC, there is already an ND interface default policy available, The ND interface policy is deployed on all IPv6 interfaces by default.

• Hop-limit: Specifies the maximum number of hops an IPv6 packet can traverse.
  
  The hop limit is used by hosts in outgoing packets and link parameters such as the link MTU. This facilitates centralized administration of critical parameters that can be set on routers and automatically propagated to all attached hosts. The range is from 0 to 255. The default is 64.
  
  
• NS Transmission Interval:
  
  The neighbor solicitation interval is sent by a node to determine the link-layer address of a neighbor, or to verify that a neighbor is still reachable through a cached link-layer address. Neighbor solicitations are also used for duplicate address detection. The range is from 1000ms to 3600000ms. The default is 1000ms.
  

• MTU:
  
  In the RA message, the maximum transmission unit (MTU) value that a host should use in packets that it originates. The range is from 1280 to 65535 bytes. The default is 9000.

Please refer to following article for more details about RS/RA:

• Retransmission Retry Count:
  The retransmission retry count for sending NS (neighbor solicitation) messages. The range is from 1 to 100. The default is 3.

• Router Advertise Interval:
  The interval for sending router advertisement messages. The range is from 4 to 1800 seconds. The default is 600.

• Reachable Time:
  The reachable time is the time in milliseconds that a node assumes a neighbor is reachable after receiving a reachability confirmation. The range is from 0 to 3600000 milliseconds. The default is 0. A value of zero means unspecified

• Retransmit Timer:
  The retransmit timer has the time in milliseconds between retransmitted neighbor solicitation messages. The range is from 0 to 4294967295 milliseconds. The default is 0.

II- ND RA prefix policy:

While ND interface policies are deployed under BDs, ND prefix policies are deployed for individual subnets. The ND prefix policy is on a subnet level. Every BD can have multiple subnets, and each subnet can have a different ND prefix policy.

Neighbor Discovery (ND) uses ICMPv6 messages known as Router advertisement (RA) messages, which have a value of 134 in the Type field of the ICMP packet header, are periodically sent out each configured interface of an IPv6 device. For stateless autoconfiguration (SLAAC) to work properly, the advertised prefix length in RA messages must always be 64 bits.

Please refer to the following article about IPv6 SLAAC process:

–> So, The ND prefix policy is on a subnet level. Every BD can have multiple subnets, and each subnet can have a different ND prefix policy.

There is already a default policy available that will be deployed on all IPv6 interfaces. Alternatively, you can create an ND prefix policy to use as shown in this example.

• Under the BD subnet configuration, In the Subnet Control field, ensure that the ND RA Prefix check box is checked:
  
  By default, the IPv6 gateway subnets are advertised as ND prefixes in the ND RA messages. You can choose to not advertise the subnet in ND RA messages by un-checking the ND RA prefix check box.

Let’s verify the different between checking and unchecking ND RA Prefix in Wireshark and host configuration:

Scenario-1: ND RA Prefix is checked:

We can see in the RA (Router Advertisement) packet send by ACI BD, we have multiple ICMPv6 Option types.

Including ICMPv6 Option type 3: Prefix Information

• Contain the BD prefix and length
• RA Prefix Flags
• Valid Prefix Lifetime: The length of time for the prefix to remain valid. The length of time that the prefix is valid for the purpose of on-link determination.
  The range is from 0 to 4294967295 milliseconds. The default is 2592000.
  
• Preferred Prefix Lifetime: The preferred lifetime of the prefix. it’s the length of time that addresses generated from the prefix via stateless address autoconfiguration remain preferred.
  
  The range is from 0 to 4294967295 milliseconds. The default is 604800.

On the host side, we see global unicast address is assigned using this prefix information from RA message sent by ACI:

(Note: host isn’t using EUI-64 for assigning global IPv6 address, instead Random 64-bit used, which is basically for security reasons)

Scenario-2: ND RA Prefix is unchecked:

Once we uncheck the ND RA Prefix under BD subnet level, ACI leaf will send an RA message with Valid lifetime equal to 0, indicating prefix information expired.

Later RA message doesn’t include the Prefix information option in ICMPv6 header:

The ND RA Prefix Policy under BD subnet dictates the flags values used in the Prefix information option in the ICMPv6 RA message sent by ACI, the following are the default values:

According to rfc4861 have the following format, including L and A flag:

       0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |     Type      |    Length     | Prefix Length |L|A| Reserved1 |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                         Valid Lifetime                        |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                       Preferred Lifetime                      |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                           Reserved2                           |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
      |                                                               |
      +                                                               +
      |                                                               |
      +                            Prefix                             +
      |                                                               |
      +                                                               +
      |                                                               |
      +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Auto Configuration (A):

1-bit autonomous address-configuration flag. When set indicates that this prefix can be used for stateless address configuration.

Please refer to this article for more details about RA flags.

On-link (L): (1-bit on-link flag)

• When set, indicates that this prefix can be used for on-link determination. It means that hosts can use addresses within that prefix to communicate directly with other hosts on the same link.

• When not set the advertisement makes no statement about on-link or off-link properties of the prefix. In other words, if the L flag is not set a host MUST NOT conclude that an address derived from the prefix is off-link. That is, it MUST NOT update a previous indication that the address is on-link.

Summary for the NDP in ACI

Scenario-1: ND RA prefix checked and host IPv6 configured with Autoconfiguration:

• Initially the Host (EP) doesn’t have a global unicast address:

1- EP sending Router Solicitation:

It is an ICMPv6 message sent to the all-routers multicast address (ff02::2), please refer to this article for more details:

2- ACI BD replying with Router Advertisement:

ACI BD will reply and send a router Advertisement to the all-nodes multicast address (ff02::1).

RA include mainly the following information:

• Link layer information (MAC) for router (ACI BD).
• Prefix-information (Prefix set on the BD subnet).

3- EP IPv6 Address assignment:

Host (EP) will use the Router Advertisement information and configure its global Unicast IPv6 address:

4- DAD (Duplicate Address Detection)

Once global IPv6 address is assigned either by combining RA Prefix with EUI-64 or via random 64-bit, the host will send a Neighbor Solicitation with Target address equal to its address to confirm if there is any duplicate address:

For every configured unicast address (link-local or global), the host joins the associated solicited-node multicast group.

Please refer to SLAAC article for more details about the “solicited-node multicast group“

That’s basically all what the host need to autoconfigure it’s global unicast IPv6 address from ICMPv6 RA (Router Advertisement) and verify its uniqueness via DAD (Duplicate Address Detection) mechanism.

Start traffic from host:

Once, we start a communication from the host (example Ping toward the BD gateway):

This will trigger Neighbor Solicitation (NS) from host to the target IP address (ACI BD subnet in this example).

Please refer to the following document for more details about NDP NS/NA:

NS from host with target address -> BD address:

NA (from BD):

ACI Leaf respond with solicited Neighbor advertisement including its link layer information (MAC):

and after that communication starts.

Also, it’s worth mentioning that host sending Neighbor Solicitation for link-local address for the ACI BD as well:

and ACI replying with BD MAC:

Scenario-2: ND RA prefix unchecked and Host IPv6 statically configured:

1- Configure EP (host) with static global Unicast IPv6 address:

• Manual global unicast IPv6 address: 2001:dbf:ba:0:baba:abab::
• Gateway: 2001:dbf:ba:: (which is ACI BD subnet)

2- DAD (Duplicate Address Detection)

Now, it much simpler, the host will first send a NS (Neighbor Solicitation for its own IPv6 Global unicast address (as part of DAD mechanism) for verify there is no duplicate IPv6 address.

I accidental kept ND RA Prefix checked under the BD subnet, and saw ACI sending RA with prefix information in ACI, but it doesn’t matter since, host is configured to have manual IPv6 global unicast IPv6 address:

and Host will just ignore it: