Troubleshooting L3Outs

Troubleshooting L3Outs

An L3Out represents a set of configurations that define how traffic is forwarded outside of the ACI fabric using routing. It is used to discover the addresses of other nodes, select routes, select quality of service, and forward the traffic that is entering, exiting, and transiting the fabric. Typically, the ACI fabric is connected to network devices that have other subnets behind it, so it is important for you to know how to troubleshoot issues related to L3Outs.

The L3Out functions as a logical connection that is established between the ACI fabric and an existing network, WAN routers, firewalls, mainframes, or any other Layer 3 device. A Layer 3 connection facilitates a routing exchange between Cisco ACI and the external routers. External Layer 3 devices connect to the front panel interface of a leaf switch. The leaf switch that provides such connectivity is known as a border leaf. The border leaf switch can also perform all the functions of a normal leaf switch.

In the ACI bridge domain/EPG, every IP address is learned as an endpoint with /32 (or /128 for IPv6). Hence, connecting a network device that contains multiple subnets behind it to ACI via a bridge domain/EPG will end up with an endpoint that has a huge number of /32-IP-addresses, which is not efficient and will likely hit a scalability limit.

With L3Out, each VRF deployed inside the fabric can use those L3Out connections for establishing VRF-Lite connectivity with external routers. Alternatively, if all the bridge domains (or VRFs) defined inside the ACI fabric must have access to a common external routing domain, it is possible to define a single L3Out connection that all those entities share and usually define as part of the common tenant.

The following example depicts a Cisco ACI fabric connected to an external network with multiple subnets using an L3Out.

The L3Out provides the necessary configuration objects for five key functions:

• Learn external routes through routing protocols, such as OSPFv2 (IPv4) and OSPFv3 (IPv6), EIGRP (IPv4 and IPv6), and BGP (IPv4 and IPv6), as well as through static routes.
• Distribute learned external routes (or static routes) to other leaf switches. The control plane that is used for this function is MP-BGP; using the VPNv4/v6 address-family for sending external routing information to all leaf nodes, for each defined VRF. Spine switches function as MP-BGP VPNv4/v6 route reflectors. Two route reflector spines per pod are recommended.
• Advertise Cisco ACI internal routes (bridge domain subnets) to outside Cisco ACI. For example, a border leaf can advertising subnets to the external router via OSPF L3Out.
• Advertise learned external routes to other L3Outs (transit routing).
• Allow traffic to arrive from or be sent to external networks via L3Out by using a contract.

In the following example, Cisco ACI routing to external networks is done through an external Layer 3 domain. The border leaf assumes an individual identity to exchange routes with Cisco ACI. MP-BGP is used to distribute externally learned routes throughout the fabric. Hence, the endpoint with IP address 2.2.2.2 in the external network, can be reachable through the fabric.

As long as there is not a firewall or other transit device blocking ICMP, an outside host can ping the IP address of the ACI L3Out interface, and vice versa. The following example shows how you can ping the outside host with IP address 2.2.2.2 from a border leaf switch, using the IP address 10.2.1.1 from the L3Out interface, as source address.