Viptela Specific Port Terminology
Port 12346 is the default base port that all Viptela devices uses for its connection that handle control and traffic in overlay network.
Let’s suppose, if multiple Viptela devices are installed behind single NAT, so it is necessary to configure different port number for each device so that NAT can properly identify each individual device and its connection. This can be achieved by port offset from its base value port 12346.
Let’s say, if any device is configured with port offset value 1, then the device will use 12347 port for connection. The Port offset value range from 0 to 19 and default offset value is 0.
Note: for NAT device that can differentiate among devices and its connection behind NAT, then in this case you don’t have to configure port offset.
It is a process or method through which devices can try or use different ports when they are trying to establish connection with each other if connection attempt to first port fails. And if this happens, the port value is incremented and connection is again re-attempted with new port.
This process, rotates through a total of five base port.
Now let’s say that port offset is not configured then the device will use default port 12346 and pot hopping is done among following ports 12346, 12366, 12386, 12406, 12426 and then return to original port 12346.
Now let’s say that port offset is configured then the initial port value is used and next port is incremented by 20.
For example, port offset value is 3, then initial port is 12349, and its subsequntial port would be 12369, 12389, 12409, 12429 and returning port would be 12349.
vEdge router uses port hopping when it tries to make connection with vManage, vBond and vSmart.
vSmart and vManage are normally installed behind NAT device, so port hopping is not needed. vBond always uses to other Viptela devices using port 12346 and they never use port hopping.
Below dig best describe how port hopping works well if device uses 12346 and connection is not succeed with in certain time, router will hop to next base port.
Why Port Hopping is used:
When vBond orchestrator crashes, the vManage may lose or close all its control connection. Then vManage NMS use port hopping to establish connection to vSmart controller on different port.
All control sessions on all vSmart controllers go down, and BFD sessions on the vEdge routers remain up. When any one of the vSmart controllers comes back up, the BFD sessions on the routers go down and then come back up because the vEdge routers have already port hopped to a different port in an attempt to reconnect to the vSmart controllers
Port used by vEdge Routers:
Base port 12346 is used by vEdge router to establish DTLS connections and the same port is used by vEdge router for IPSEC connection and BFD sessions to other vEdge router in the overlay network. And if the port offset is used or configured, then the port offset rule and port hopping rule will be used.
To check which port DTLS and BFD uses for control and data connection, check private port column in output column of Show control local-properties
In general Network Design, Firewall always exits and if this is in case you must open Viptela ports (base port and its four base port) on the Firewall device to allow traffic through it across overlay network.
Now if vEdge router TLS tunnel, on TCP protocol, router uses any random TCP port, so in this case you must configure proper NAT for vManage, vSmart to be able to communicate with vEdge routers.
For vEdge router configured to use DTLS tunnel, which uses UDP, to start or at minimum default base port along with its next four port must be opened as given below:
- Port 12346
- Port 12366
- Port 12386
- Port 12406
- Port 12426
And if port offset is used or configured then you must open first offset port and its next four ports.
Ports Used by Viptela Devices Running Multiple vCPUs
The vManage NMSs and vSmart controllers can run on a virtual machine (VM) with up to eight virtual CPUs (vCPUs). The vCPUs are designated as Core0 through Core7.
Each core is allocated separate base ports for control connections. Depending on whether the connection is over a DTLS tunnel (which uses UDP) or a TLS tunnel (which uses TCP) the base port differs
Note: vBond orchestrators do not support multiple cores. vBond orchestrators always use DTLS tunnels to establish control connections with other Viptela devices, so they always use UDP. The UDP port is 12346.
The following table lists the port used by each vCPU core for the vManage NMS. Each port is incremented by the configured port offset, if offset is configured.
Administrative Ports Used by vManage NMS
vManage NMS uses the following administrative ports for protocol-specific communication
vManage clusters use the following ports for communication among the NMSs that comprise the cluster
Configure the Port Offset
When two or more Viptela devices are behind the same full-cone NAT device, one device use default port offset. And following configuration for port offset is done on remaining devices:
The port offset can be a value from 0 through 19. The default port offset is 0.
In the following example, vEdge-1 uses the default port offset of 0, and on vEdge-2 the port offset is set to 1.
- vEdge-1 attempts to connect first using base port 12346. If that attempt is not successful, the router attempts port 12366, 12386, 12406 and 12426.
- vEdge-2 has a port offset of 1, so the first port it attempts to connect on is 12347 (12346 plus offset of 1). If it fails to connect using port 12347, the router hops by increments of 20 and attempts to connect on ports 12367, 12387, 12407, and 12427.
Perform Port Hopping Manually
You can manually request a vEdge router to port-hop:
One reason to use this command is if the router’s control connections are up, but BFD is not starting. The request porthop command restarts the control connections on the next port number, and BFD should then also start.