Layer 2 Unknown Unicast Flooding

Layer 2 Unknown Unicast Flooding

Cisco ACI can optimize traffic by preventing flooding for Layer 2 unknown unicast packets. If a bridge domain is in Layer 2 Unknown Unicast hardware-proxy mode, which is the default, the packet is forwarded to the spine proxy. You can change this behavior and turn on flooding mode Layer 2 unknown unicast packets. Thus, during troubleshooting, it is important to understand the process of Layer 2 unknown unicast packets forwarding.

When the Layer 2 Unknown Unicast is hardware-proxy, there are two possible outcomes:

• When the leaf switch does not know where the MAC address of the destination endpoint resides (possibly MAC has timed out or there is a silent host or the endpoint had never communication with anyone in that leaf switch), it will encapsulate the packet with an anycast TEP for spine-proxy (MAC) to send a packet to spines for spine-proxy (hardware-proxy mode). If it is for IPv4 lookup, an anycast TEP for IPv4 is used. Same is for IPv6. If the COOP database on the spine has information on MAC address of the referred endpoint, the spine forwards the packet to a destination (egress) leaf.
• If MAC address of an endpoint is unknown to the spine COOP database (for example, silent host), the spine drops the packet.

When the bridge domain is in Layer 2 Unknown Unicast Flood mode, the packet is flooded within the bridge domain. The endpoint table and COOP database are still populated with the MAC-to-VTEP information. However, the forwarding does not use the spine-proxy database. Layer 2 unknown unicast packets are flooded in the bridge domain using one of the multicast trees rooted in the spine that is scoped to the bridge domain. Thus, the flooded packets are encapsulated in a multicast IP called Group IP Outer (GIPo) instead of TEP of leaf nodes. Each bridge domain is assigned one GIPo internally for this purpose.

Layer 2 Unknown Unicast: Same EPG, Bridge Domain, and Encapsulation

When the two endpoints are part of the same EPG, the same bridge domain, and use the same VLAN access encapsulation while connected to the leaf switch, the traffic is locally switched when the leaf switch knows their MAC addresses, as in the following example:

When H1 sends traffic to H2, and this is a known unicast traffic (leaf MAC address table is full), the traffic is sent out interface 1/2 where H2 is connected.

In a similar scenario, where the H2 MAC address is not known to the leaf switch, the traffic behavior will depend on the bridge domain Layer 2 Unknown Unicast settings. This figure shows the traffic from H1 to H2, when the bridge domain Layer 2 Unknown Unicast is set to hardware-proxy.

Traffic from H1 to H2 is the following:

• H1 sends traffic towards H2. Destination is H2 MAC address.
• Local leaf switch does not know the address of the endpoint H2. It sends the original packet to the spine.
• Since H2 information is missing from the COOP database on the spine switch, the traffic is dropped.

The following figure shows an example when two endpoints (H1 and H2) that are part of the same EPG, the same bridge domain, and use the same access encapsulation while connected to the same leaf switch. The bridge domain Layer 2 Unknown Unicast is set to flood.

Traffic from H1 to H2 is the following:

• H1 sends traffic towards H2. Destination is H2 MAC address.
• Local leaf switch does not know the address of the endpoint H2.
• The traffic is flooded to all interfaces in the bridge domain. H2 receives the frame and replies, while it is learned in the fabric.

In ACI, forwarding tag (FTAG) trees are built to load-balance multidestination traffic and to avoid any loop within the infra VRF. One of the spine switches is chosen as an FTAG tree root. Root builds branches to every leaf switch directly connected to that particular spine switch. So, the path to all the nodes is now built with a loop-free path. Packets are then flooded by following the FTAG tree branches (encapsulated in a multicast GIPo, instead of TEP of leaf nodes); a mechanism to ensure that the packet is sent to each node only once, without any duplicates.

When the bridge domain Layer 2 Unknown Unicast settings are set to hardware-proxy, and endpoints are on different leaf switches, such as H1 and H3 as in this figure, the unknown unicast traffic forwarding also depends on the COOP database on the spine switch.

Traffic from H1 to H3 is the following:

• H1 sends traffic towards H3. Destination is H3 MAC address.
• Local leaf switch does not know where the address of the endpoint H3. It sends the traffic to the spine switch (using the spine-proxy anycast VTEP address).
• Since the spine has the H3 mapping in the COOP database, it sends the traffic to the destination leaf switch across the fabric, which forwards the traffic to H3. Once H3 receives the traffic, it can reply to H1, which will enable the fabric to learn its information.

However, if the H3 information is missing from the mapping in the COOP database on the spine switch, the traffic is dropped at the spine switch, and will not reach H3, as in this example.

The following figure depicts two endpoints (H1 and H3) that are part of the same EPG, the same bridge domain, and use the same access encapsulation while connected to different leaf switches. The bridge domain Layer 2 Unknown Unicast is set to flood.

Traffic from H1 to H3 is the following:

• H1 sends traffic towards H3. Destination is H3 MAC address.
• Local leaf switch does not know the address of the endpoint H3.
• The traffic is flooded to all interfaces in the bridge domain. H3 receives the frame and replies, while it is learned in the fabric.