Fabric Path VPC+

Fabric Path VPC+

Posted on Jan 24, 2020 (0)

FabricPath VPC+

What causes VPC+ in to picture, even we have VPC already in place on Nexus 7000 or Nexus 5000 series switches.

Let’s understand this by an example. Consider the below figure to understand this.

Now in left Figure, VPC is configured and Host A is sending the traffic, to Host B which is connected to Switch S3. Now for one flow it can take the S1 to reach to Host B, when this happens the following will be the Header structure when Ethernet frame will get encapsulated to fabric Path frame.

  • Outer DA: S3
  • Source SA: S1
  • DMAC : B
  • SMAC: A

Now let’s suppose Host A started another type of traffic, and due to Port-channel hash algo, it took S2 to reach S3, in this Fabric Path frame will be like this:

  • Outer DA: S3
  • Source SA: S2
  • DMAC : B
  • SMAC: A

Now On S3, S3 will make the MAC table but it will confuse that same MAC A can be reached by two S1 and S2 switch and on to which switch it should send the reply? S1 or S2, and this can also cause MAC flapp issue.

This Problem was solved by VPC+, in which we configured another virtual Switch ID under VPC domain, So in S1 and S2 will be seen as single switch let’s say S4 to all another switch in fabric path domain. See right side of the same figure.

Now S3 can easily reply to Host A and can send the packet to S4 (which is S1 or S2) which further can be forwarded to Host A.

VPC+ basics:

VPC+ is similar to VPC except some configuration changes, VPC+ requires a virtual switch ID to be configured in VPC domain apart from its own unique switch ID. All Host which are behind the VPC + will be identified by this Virtual Switch ID and all other Orphan port and single attached host will be identified by or associated by each Switch individual Switch ID.

The Virtual Switch ID is also distributed by IS-IS protocol to all other switch in fabric Path network.

Following configuration changes are to be done:

  • Under VPC domain , fabric Path switch ID must be configured like fabricpath switch-id <>
  • VPC peer Link must be configured with switchport mode fabricpath

VPC+ Sub Switch ID and Port ID:

Let’s discuss VPC SSID and Port ID role in Fabric Path frame.

The virtual SID is used by the vPC+ peers to populate the outer SA SID field in Cisco Fabric Path frames originating from a vPC+ Port Channel. SubSwitch ID SSID identifies the specific originating vPC+ Port Channel and the port ID (or local ID) specify the logical number of interface participating in VPC Port channel and though which packet is sourced or destined to.

In above Figure,

When any fabric Path frame is sourced from VPC+ then Outer SID will be vPC+ virtual SID (5) and port-ID is the shared local ID value (2134), this value will be same from either vPC+-attached switch. However, the sSID differs, since this field uniquely identifies each vPC+ Port Channel interface in the vPC+ domain.

After other Cisco Fabric Path switches learn MAC A and MAC B as remote MAC address entries, any frames forwarded back to the vPC+ domain use the virtual SID address as the destination. When the frames arrive at either vPC+ peer switch (S1 or S2), the SSID value identifies the output vPC+ Port Channel on which the frame should be forwarded.

VPC+ Packet Flow

As we know that for Broadcast, Multicast and unknown unicast frame, fabric Path uses Multidestination tree , but in case of VPC+ if BUM packet is sent by any one of VPC+ peer using Virtual Switch ID , RPF check will fail in some of the FabricPath switches. Now to solve this VPC+ has introduced the following rule:

Both VPC+ Peer can be part of Multidestination Tree (Tree-1 and Tree-2), but One peer will associate its self or will show its affinity to one Tree Tree-1 and another VPC+ Peer switch will associate or show its affinity to another Tree-2. So for Every VPC+ Port channels represented by Virtual Switch, its affiliated tree must be announced.

Now the Switch who has shown its affiliation to particular Tree will only responsible for decpatulating the frame and forward it to downstream and those only switch will encapsulate the frame with fabricpath header using FTag.   

To understand this let’s assume that there are two trees in the FabricPath. Switch Sw1 has announced Its affinity to FTAG1 and Switch Sw2 to FTAG2 likewise Switch Sw3 has announced its affinity to FTAG1 and switch Sw4 to FTAG2.

If any BUM frames that arrive at Switch Sw1 or Switch Sw3 from the host, the switches use the tree corresponding to FTAG1 to forward the traffic. In same way if BUM frames that arrive at Switch Sw2 and Switch Sw4 from the hosts, it uses FTAG2. If any BUM traffic reaches at both the vPC+ peer switches using the tree corresponding to FTAG1, only Sw1 and Sw3 decapsulate the traffic and send it toward the vPC+ member port. Similarly, Sw2 and Sw4 act on the BUM frame arriving on the tree corresponding to FTAG2.

Please refer the following below diagram for traffic flow 

Now as per above diagram, let’s suppose Host A and Host B wants to talk to each other and Host A has MAC A and Host has MAC B. The traffic flow is just described in below steps:

Host A wants to talk to with Host B and in ARP table of Host A do not have the IP-MAC binding of Host B, then it will send an ARP request frame with the broadcast destination MAC (ffff.ffff.fff) and source MAC 0000.0000.000A of Host A.  

  • The ARP request reaches to the access switch which learns the MAC address of Host A and forwards, the frame to any of the vPC+ peer switches based on the computed hash for the port channel. Assume that the ARP request frame goes to Sw1. Switch Sw1 receives the frame from Host A on the vPC+ (po10) link in VLAN 100. Sw1 Will see the following entry in its MAC table first and ends up in a miss for {VLAN=100, MAC=0000.0000.000a} also Sw1 will learns the Host A MAC address on the ingress interface. As Switch Sw1 is the vPC+ peer of Switch Sw2, the MAC address of Host A is synchronized with the peer Switch Sw2 over CFSoE.
  • Now the ARP request is a broadcast frame, so it needs to be sent on a distribution tree. Switch Sw1 has announced it affinity to FTAG1, it encapsulates the ARP request in a FabricPath header and sends the frame over Tree 1 (multicast tree corresponding to FTAG1).
  • The source switch-ID will be used in the virtual switch-ID 100 based on the vPC+ configuration between Switch Sw1 and Sw2. The FTAG field will be the value of 1. The vPC+ peer Switches Sw3 and Sw4 receive the frame, and as Switch Sw3 has announced its affinity to FTAG1, it decapsulates the frame and sends it on the vPC+ port channel (po20). Because the frame is a multidestination frame that arrived over the fabricPath network, source MAC address learning is not performed.
  • The access switch receives the ARP request floods it over the VLAN 100. Host B receives the frame and sends an ARP reply frame with its own source MAC (MAC=0000.0000.000b) and destination MAC of Host A (MAC=0000.0000.000a).
  • The ARP request reaches the access switch connected to Host B. The access switch forwards the ARP response Switch Sw4 based on its hash computation. Switch Sw4 receives the unicast frame and does a source MAC address lookup (VLAN=100, MAC=0000.0000.0000b), which it will not found in its MAC table and hence it is a miss. So Sw4 learns the MAC address of Host B on its local/edge port (po20). The MAC address of Host B is synced to its corresponding vPC+ peer Switch Sw3 using CFSoE.
  • Switch Sw4 will encapsulate the APR response with Fabricpath header with following information: Source Switch-ID is virtual switch-ID 200 in the source switch-ID with FTag is 2 because the inner destination MAC address (VLAN=100, MAC =0000.0000.000a) was a miss and Switch Sw4 has announced its affinity for FTAG2, the multidestination frame is sent on the distribution tree corresponding to FTAG2.
  • The vPC+ peer switches Sw1 and Sw2 will receive the ARP reply. The switches decapsulate the frame and destination MAC of inner frame (VLAN=100, MAC=0000.0000.000a) is known, and they also learn the inner source MAC (VLAN=100, MAC=0000.0000.000b) against the virtual switch-ID of 200. Switch Sw1 will not forward the frame on the vPC+ member link for po10 because the frame arrived with an FTAG value of 2 and Sw1 does not have affinity to FTAG2. Instead, Sw2 forwards the frame to Host A via the vPC+ port channel.
  • Host A receives the ARP response now Host A will send the unicast data to Host B. The switch (Sw1 or Sw2) encapsulates the frame and sends it toward virtual switch-ID 200 this is why because of the hash also used by port-channel 10. As this is a unicast frame, it arrives at either Sw3 or Sw4 from the FabricPath domain. As soon as this unicast data frame reaches the Switch Sw3 or Sw4, the frame is decapsulated and the inner source MAC address (VLAN=100, MAC=0000.0000.000a) is learned against the virtual switch-ID 100 and layer 2 lookup with destination MAC (VLAN=100, MAC=0000.0000.000b) is hit as it is present in MAC address table. The MAC address of Host A will be synced to the vPC+ peer switch using CFSoE. Either Switch Sw3 or Sw4, depending on where it arrived from the FabricPath cloud, then forwards the frame to Host B.



    You are will be the first.


Please login here to comment.