ARP Flooding

ARP Flooding

In the bridge domain in Cisco ACI, you can use the ARP flooding option, or disable it when needed. Still, you should know the fabric behavior regarding ARP flooding, so you can successfully troubleshoot Layer 2 issues.

If ARP flooding is enabled, ARP traffic will be flooded inside the fabric as per regular ARP handling in traditional networks. ARP flooding is required when you need Gratuitous ARP (GARP) requests to update host ARP caches or router ARP caches. This is the case when an IP address may have a different MAC address (for example, with clustering of failover of load balancers and firewalls).

If ARP flooding is disabled, the fabric attempts to use unicast to send the ARP traffic to the destination. Therefore, a Layer 3 lookup occurs for the target IP address of the ARP packet. ARP behaves like a Layer 3 unicast packet until it reaches the destination leaf switch. Note that this option applies only if unicast routing is enabled on the bridge domain. If unicast routing is disabled, ARP flooding is implicitly enabled.

ARP Flooding: Same EPG, Bridge Domain, and Encapsulation

When two endpoints are part of the same EPG, the same bridge domain, and use the same VLAN access encapsulation while connected to the same leaf switch, the traffic is locally switched when the leaf switch knows their endpoint information. This behavior will be the same, when one endpoint, such as H1, sends an ARP request towards the other (H2), and ARP flooding is disabled. Since the leaf switch knows where H2 is connected and checks the ARP target IP address (which is H2 IP address), there is no need to flood the traffic or redirect it to the spine layer. Therefore, it sends the ARP request toward H2, as in depicted in this figure:

Cisco ACI performs unicast forwarding for ARP requests using its ARP target IP only when ARP flooding is disabled.

In a similar scenario, when ARP flooding is disabled and the ingress leaf does not know where the ARP target IP address is located, an ARP request will be sent to the anycast spine-proxy TEP instead of flooding. This figure shows the ARP request from H1 to H2, when the bridge domain ARP flooding is disabled and H2 information is unknown to the ingress switch.

ARP traffic from H1 to H2 is the following:

• H1 sends ARP request for H2 using a broadcast destination MAC.
• The ACI attempts to use unicast forwarding to send the ARP request. The local leaf switch does not know the IP address of the endpoint H2 (the ARP target IP is unknown to the ingress leaf), so it sends the ARP request to the spine switch for spine-proxy.
• Since H2 endpoint information is missing from the COOP database on the spine switch, the spine will drop the original packet, but instead it will trigger ARP glean to detect the target IP, so that subsequent ARP requests will not be dropped.

ARP gleaning is explained in more details further in this section.

However, if the ARP flooding is enabled in the bridge domain, the ARP request from H1 will reach H2 through flooding, as in this figure:

ARP traffic from H1 to H2 is the following:

• H1 sends ARP request for H2 using a broadcast destination MAC.
• The ARP request is flooded to all interfaces in the bridge domain. H2 receives the frame and replies, while it is learned in the fabric.

When the endpoints are on different leaf switches, while part of the same EPG and bridge domain, and using the same VLAN access mapping, the ARP request (for example, from H1 to H3) has to be forwarded across the fabric. This figure depicts an example when the ARP flooding is disabled at bridge domain level, while unicast routing is enabled.

ARP traffic from H1 to H3 is the following:

• H1 sends ARP request for H3 using a broadcast destination MAC.
• The ACI attempts to use unicast forwarding to send the ARP request, so the local leaf switch checks the ARP target IP address, which is the H3 IP address. Since, the local leaf switch does not know the IP address of the endpoint H3, it sends the ARP request to the spine switch for spine-proxy.
• The spine has the H3 information in the COOP database (unicast routing is enabled) and forwards the ARP request to the destination leaf switch across the fabric, which forwards it to H3. Once H3 receives the traffic, it replies to H1.

However, if the ARP flooding is enabled in the bridge domain, the ARP request from H1 will reach H3 through flooding, as in this figure:

ARP traffic from H1 to H3 is the following:

• H1 sends ARP request for H3 using a broadcast destination MAC.
• The ARP request is flooded to all interfaces in the bridge domain. Across the fabric, it is encapsulated with the outer destination IP of the bridge domain multicast IP address (GIPo). The leaf switch where H3 is connected will decapsulate the packet and send it to H3. H3 receives the frame and replies, while it is learned in the fabric.

When ARP flooding is enabled, the ARP requests are flooded, disregard whether uncast routing is enabled at the bridge domain level. In the example above the uncast routing is enabled, but spines are not referenced.

Similarly, when unicast routing is disabled (no IP spine mapping), as in the following figure, the ARP request from H1 to H3 is flooded to all interfaces in the bridge domain, and H3 is again learned.

ARP Flooding: Same EPG and Bridge Domain, Different Access/Encapsulation

When two endpoints are part of the same EPG and the same bridge domain, but use different VLAN access encapsulation while connected to the same leaf switch, the traffic is locally switched when the leaf switch knows their endpoint information.