SD-WAN VeloCloud Traffic Flow
Before we talk about how talk to SD-WAN network Branch and how migration is done, let’s understand what ports and Protocols are being used for communication on SD-WAN NSX VeloCloud environment.
Ports & Protocols for Communication:
- Branch to VCO ( Orchestrator ) : for Management , TCP/443
- Branch to Hub : data Traffic UDP/2426 ( tunnel to hub is created )
- Branch to Hosted VCG or On-Prim VCC : Control and data UDP/2426
- Hosted VCG or On-Prim to Orchestrator : TCP 443 for Management
SD-WAN Reference Topology:
Below topology will explain how NSX SD-WAN are connected by different method, how SD-WAN CPE is connected to MPLS and Internet via VRRP, How SD-WAN with L3 switch is connected, How Hybrid SD-WAN site is connected and How Non SD-WAN site is connected with SD-WAN CPE Internet connection.
Below is the figure that explains that:
HUB VCE Placement:
There are two options to which HUB can create Tunnel:
- Option1: Route the private tunnel traffic through the firewall
- Option2: Route the private tunnel traffic directly to the L3 core
Migration Work Flow
Following are the Migration flow of any WAN sites which has to be migrated from Typical WAN to SD-WAN using VeloCloud technology.
- Ensure firewall ports are opened to allow SD-WAN components to communicate
- Define the traffic flow, do I need BGP at branch WAN or not. How does my branch users reach the Internet. Filter what you do not need.
- Install and activate the hub & enabling routing protocol
- Ensure routes are learned properly by SD-WAN but do not advertise anything into the network until you are ready
- Install and activate the branch
- Validate the traffic flow
Communicate with Non SD-WAN Site:
There are two ways to connect the Non SD-WAN sites to SD-WAN sites, which are as follows:
- Use HUB Sites as Transit
- Connect the Non SD-WAN site directly to SD-WAN Branch Site
Use HUB Site as Transit:
- Traffic to/from non-SD-WAN sites go through transit site first to reach SD-WAN site
- If non-SD-WAN sites are high BW, allow SD-WAN sites to use combined link BW
- Will introduce latency due to backhaul
Connect the Non SD-WAN site directly to SD-WAN Branch Site
- Traffic to/from non-SD-WAN sites go directly to MPLS and use transit site as backup
- Lowest latency but less available bandwidth
Understand SD-WAN Routing
Let’s understand how routing stack integration is done with SD-WAN Overlay. In this Control Plane channel peer is created between edge to gateway to share route and a separate data plane dynamic tunnel is created between edges. Due to this separate Tunnel, it achieves the separation between control plane and data plane.
Following are the some highlights of SD-WAN VeloCloud.
- Support overlay and underlay routes over the same interface
- Underlay route options: Static (with IP SLA), OSPF, BGP
- OSPF and/or BGP underlay routing protocol at each site
- Underlay routes are redistributed to the overlay and vice versa while retaining the BGP attributes
Branch becoming Transit Site: issue
In the SD-WAN environment, our design should be in such a way that Branch site should not become transit site, to understand this let’s assume the below topology:
Let’s discuss it point by point
- Hub sites can reach to 10.10.10.0/24
- Hub sites advertise this 10.10.10.0/24 network to MPLS via BGP
- Non SD-WAN sites stores prefix in its routing table stating that it can reach to 10.10.10.0/24 by HUB.
- BR1 gets activated on SD-WAN and starting learning the 10.10.10.0/24 by SD-WAN overlay
- BR1 advertises the 10.10.10.0/24 to MPLS which its learned from SD-WAN
- Non SD-WAN site puts the 10.10.10.0/24 reachability via BR1
- BR2 gets activated over SD-WAN
- sends that 10.10.10.0/24 is reachable via BR2
- Non-SD-WAN sites puts 10.10.10.0/24 is reachable from BR2
Non-SD-WAN sites puts 10.10.10.0/24 is reachable from BR2
Now when sending traffic to 10.10.10.0/24, which routes Non SD-WAN will take, as it has three entry n its routing table
To avoid this following steps should be taken:
- Stop mutual re-distribution between overlay and MPLS at branches
- Prefixes belong to the branch should still be advertised into the overlay
But this can be more complicated, so there is various technique developed by Velo cloud which is been discussed below:
BGP Uplink Neighbor
Uplink neighbor is the neighbor who takes you in and out from network and is CPE or PE router. So if you mark the neighbor as uplink, to the int which is connected to that neighbor, mutual re-distribution between overlay and underlay will be prevented.
In the above figure, prefix learned from internet will be redistributed to LAN but we have made the MPLS Router 192.168.10.10 as neighbor of SD-WAN router, so SD-WAN router will not redistribute prefix learned from Overlay (Internet) to underlay (Internet).
This BGP Uplink Neighbor helps in following way:
- Assist with stopping a site from being a transit
- Control route exchange between underlay and overlay
BGP Uplink Community:
This is the case when the Traditional CE and VCE is connected via L3 switch , (typically used in DC ) , in this case we cannot define the CE router as uplink neighbor, now if you want to tag the prefix coming from CE router – tag it , that means anything that is coming from CE router will be tagged by a certain community , so now when VCE will learn the prefix from its neighbor , it can easily distinguish that which prefix is learned by CE and which prefix is learned by LAN and so that based on tagging it can redistribute the LAN prefix to Overlay and not CE learned Prefix to Overlay.
It does the following:
- Tagged the prefixes learned from the MPLS
- VCE knows not to re-advertise the prefixes tagged as uplink to the overlay
Lets suppose take an another example , here I want to reach to legacy MPLS site from Hybrid branch , and not via Overlay , but if my MPLS goes down I want to send the traffic to MPLS router via HUB.
Sol: To achieve this tag the MPLS prefix with community so that as soon as it reaches to Hybrid Router, Hybrid Router will guess that this prefix is not belong to SD-Wan site and will use it as backup also it will not advertise the prefix to underlay( MPLS) again.
Route Preferences, for a given prefix
- Take the overlay if prefixes are owned by SD-WAN site
- Take the underlay, e.g. direct MPLS, if the route exists
- Send to the hub