BD VRF & EPG Design consideration – Service Chaining
Different deployment modes of L4-L7 devices:
There are different modes of deploying L4-L7 devices together with service graph.
Transparent Mode: In this L4-L7 device bridge the traffic between two bridge domains and in ACI it’s called as go-through mode.
Routed Mode: In this L4-L7 device is used for routing the traffic between two bridge domains and in ACI is it called as go-to-mode
One Arm Mode: In the L4-L7 device is connected to fabric via single interface over dedicated bridge domains
Policy Based Redirect (PBR): L4-L7 device is deployed in separate BD and client and server can redirect the traffic based on protocol and port number.
BD Design Consideration:
When we design the BD o for L4-L7 deployment, we generally considers two BD, one for Outside and one BD for inside.
Following are the default BD configuration option mostly used in deployment:
- Unknown Unicast flooding
- AR flooding
- No routing (except if the BD is acting as default gw for Servers or for L4-L7 device)
- No subnet ( except if the BD is acting as default gw for Servers or for L4-L7 device)
Below figure shows the basic deployment solution:
Here Two BD ae there one is connected to outside interface and one is connected to inside interface of L4-L7 device , there are two EPG , One is associated with BD used of outside and one EPG is associated to BD used for Inside. A VRF is associated to each BD which is not used for routing of traffic but used to simply meet the object model of ACI.
Now if you want to use the graph with redirect model, then you need to define one or Two BD on which this L4-L7 device will connect and must have data-plane learning disabled and GARP detection enabled. These BD will not be used to connect the endpoints.
Below figure will demonstrate, as Service Graph example showing on which BD Cisco ACI performs Layer 2 and Layer 3 Forwarding.
In figure, if BD1 has subnet configured and is GW for FW that traffic is switched at Layer 3, Now if BD2 is Next of LB is outside interface of FW and Next hop of FW inbound interface is load balancer in this traffic is switched at layer 2. Now in BD3 GW for Servers is LB so that traffic is switched at Layer 2.
In this scenario, you should enable Limit IP learning on BD1, else mapping DB can learn the IP address of endpoints from BD3, as if they were on BD1.
To avoid flood in ACI there are two option which can be used while configuring BD: Hardware Proxy and No ARP flooding.
But there are certain scenarios where, these option can be used, are discussed below:
- Devices configured in Go-through mode (transparent mode) require flowing to build its forwarding tables.
- If GRAP is used by any L4-L7 device, to inform its MC address to fabric, ARP flowing is enabled.
Now if you configure service graph in go-through mode ACI itself will change the BD setting and enable unknown unicast flooding and ARP flooding.
Now if Service Graph is configured in go-to mode, as per below figure BD3, flooding optimization Is useful as it has several VMs and servers, rater BD2 and BD1 will have such benefit to enable Flood optimization as it has only two interface connected to it.
Below figure demonstrate the topology where ARP flooding should be enabled. Let’s suppose as per below topology, Hardware Proxy is enabled and flooding is disabled, and if service device fails and if the L4-L7 device comes UP it (may or may not change its MAC address), GRAP will be generated upon its return and due to flooding is disabled, GRAP will not be flooded to fabric and END points will not be aware now and traffic may be dropped.
But if the same MAC address is allowed to configure in both L4-L7 devices which are in HA pair , you can enable Hardware Proxy and disable ARP flooding else , you have to enable ARP flooding.
In summary if ACI admin wants to reduce the ARP flooding, you need to configure the BD with the following configuration:
- Turn on Hardware Proxy
- ARP flooding must be disabled
- IP routing must be enabled
- Subnet IP address must be configured
- Subnet check ( Limit IP Learning must be used )
Note: for most deployment ARP flooding is ON.
BD tuning for Service Graph with PBR:
Now if Service Graph redirect is used, The BD that connect to L4-L7 devices must be configured with following options:
- Enable IP routing
- Configure BD subnets
- GRAP based detection is on
- Data Plane learning should be disabled.