Cisco SD-WAN vEDGEs Onboarding Process explained [with Configuration]
Contents
vEDGE Onboarding Options
Cisco SD-WAN offers several methods for onboarding vEDGEs devices:
- Automated Deployment: This is the preferred method, providing a zero-touch experience. It utilizes Zero-Touch Provisioning (ZTP) for Viptela vEdge devices and Cisco Plug-and-Play (PnP) for IOS-XE devices. Both processes involve the device dynamically discovering the vBond orchestrator’s IP address by resolving specific FQDN and then establishing connections with the SD-WAN controllers.
This method typically require connecting the SD-WAN EDGE Device to a WAN transport (with DHCP) that can provide a dynamic IP address, default-gateway and DNS information.
- Bootstrap Deployment: Applicable to Cisco IOS-XE devices, this method is used when automated deployment isn’t feasible, such as in environments without DHCP or internet access. It involves creating a device template in vManage and loading the configuration file onto the device via its internal flash memory or a USB stick. The configuration file must be named “ciscosdwan.cfg” for the device to recognize and load it.
- Manual Deployment: This method involves configuring the device directly through the command-line interface (CLI). It’s typically used when both automated and bootstrap methods are unsuitable.
vEDGE Onboarding Process Overview
The onboarding process for Cisco SD-WAN vEdge devices involves several steps to securely establish a connection between the vEdge device and the vBond orchestrator, we will illustrate in this post these steps:
1- vEdge Initialization:
- When a vEdge device is powered on, it first needs to reach out to the vBond orchestrator to initiate the onboarding process.
- The device must be pre-configured with certain information, such as the vBond’s IP address or domain name. This can be done using methods like Zero-Touch Provisioning (ZTP) or bootstrap.
2- vBond Discovery:
- The vEdge device uses DNS (if DNS resolution is configured) or a predefined IP address to locate the vBond orchestrator.
- The vEdge device sends a request to the vBond to establish a secure connection (DTLS).
3- vEDGE Authentication by vBond:
- After receiving the connection request from vEDGE device, the vBond verifies the identity of the vEdge node by checking the device’s certificates (specifically, the device certificate, which should be signed by the root CA trusted by vBond).
- The vBond checks if the vEdge device is authorized to join the SD-WAN network (based on the serial number or other attributes).
4- Certificate Exchange:
- If the vEdge device’s identity is authenticated, the vBond sends back its own certificate, and the vEdge device verifies the authenticity of the vBond certificate.
- The vEdge device and vBond exchange certificates and establish a mutual trust relationship.
5- vBond Configuration:
- Once the authentication is complete, vBond sends configuration information to the vEdge device, such as the vSmart controller and vManage IP addresses.
- The vEdge device now knows which SD-WAN controllers (vSmart and vManage) to communicate with.
6- Connection control establishement to vSmart and vManage:
- The vEdge device then establishes a secure connection using SSL TLS/DTLS with the vSmart controller (for control plane communication) and vManage (for management and configuration).
- From control plane connection perspective, this is what we will achieve after the configuration detailed in the end of this post:
LearnDuty-vEdge1# show control connections
PEER PEER CONTROLLER
PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB GROUP
TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT LOCAL COLOR PROXY STATE UPTIME ID
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
vsmart dtls 10.10.0.12 10 1 222.2.2.2 12446 222.2.2.2 12446 mpls No up 0:19:38:00 0
vsmart dtls 10.10.0.12 10 1 222.2.2.2 12446 222.2.2.2 12446 public-internet No up 0:20:59:04 0
vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 mpls - up 0:19:38:00 0
vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 public-internet - up 0:20:59:05 0
vmanage dtls 10.10.0.10 10 0 222.2.2.1 12646 222.2.2.1 12646 public-internet No up 0:20:58:48 0
Code language: CSS (css)
7- OMP peering with vSmart:
- The vEdge devices and vSmart controller set up secure data tunnels using DTLS to securely exchange routing information and policy data with other vEDGEs across the SD-WAN network via OMP protocol (Overlay Management Protocol).
Below output illustrates OMP peering, as result of the done onboarding configuration in later section if this article:
- OMP peering on vEDGE-1 with vSmart controller:
LearnDuty-vEdge1# show omp peers
R -> routes received
I -> routes installed
S -> routes sent
DOMAIN OVERLAY SITE
PEER TYPE ID ID ID STATE UPTIME R/I/S
------------------------------------------------------------------------------------------
10.10.0.12 vsmart 1 1 10 up 2:22:35:16 0/0/0Code language: PHP (php)- OMP peering on vSmart with all vEDGE nodes:
LearnDuty-vSmart# show omp peers
R -> routes received
I -> routes installed
S -> routes sent
DOMAIN OVERLAY SITE
PEER TYPE ID ID ID STATE UPTIME R/I/S
------------------------------------------------------------------------------------------
10.110.0.11 vedge 1 1 11 up 2:22:50:52 0/0/0
10.120.0.12 vedge 1 1 12 up 2:22:51:01 0/0/0
10.130.0.13 vedge 1 1 13 up 2:22:51:06 0/0/0Code language: PHP (php)
8- vEDGEs TLOC Routes exchange via OMP peering (via vSmart):
- After the vEdge device establishes connections to the vSmart controller, it uses the Overlay Management Protocol (OMP) to exchange TLOC (Transport Locator) information with the vSmart controller.
- TLOC routes: This information includes details about other vEdge devices in the network, such as their system IPs, TLOC colors (e.g., public-internet, MPLS), and encapsulation types.
Output below shows an Example of vEDGE-2 OMP TLOC route (Mpls color) received on vEDGE-1 (received from the vSmart):
LearnDuty-vEdge1# show omp tlocs received
---------------------------------------------------
tloc entries for 10.120.0.12
mpls
ipsec
---------------------------------------------------
RECEIVED FROM:
peer 10.10.0.12
status C,I,R
loss-reason not set
lost-to-peer not set
lost-to-path-id not set
Attributes:
attribute-type installed
encap-key not set
encap-proto 0
encap-spi 264
encap-auth sha1-hmac,ah-sha1-hmac
encap-encrypt aes256
public-ip 172.16.12.1
public-port 12346
private-ip 172.16.12.1
private-port 12346
public-ip ::
public-port 0
private-ip ::
private-port 0
bfd-status up
domain-id not set
site-id 12
overlay-id not set
preference 0
tag not set
stale not set
weight 1
version 3
gen-id 0x80000001
carrier default
restrict 1
groups [ 0 ]
border not set
unknown-attr-len not setCode language: CSS (css)9- IPsec Tunnel Formation Between vEdge Devices
Since at this point, vEDGE-1 (for example) received the needed TLOC information for the other vEDGEs in the SDWAN fabric, it will proceed with forming IPSec tunnel for Dataplane traffic:
The process for forming IPsec tunnels between vEdges is as follows:
a- TLOC Discovery via OMP:
- After forming OMP peering with vSmart, the vEdge device advertises its local TLOC information to the vSmart controller using OMP.
- The vSmart controller consolidates TLOC information from all vEdges and sends the relevant peer TLOC information back to the vEdge, enabling it to identify other vEdge devices in the SD-WAN fabric.
b- IPsec Tunnel Establishment:
- Using the TLOC routes received from vSmart, the vEdge identifies the transport IP addresses and colors of its peers.
- The vEdge initiates IPsec tunnel negotiations with its peer devices over the underlay network using the Internet Key Exchange (IKE) protocol.
Note
“Color” dictates the use of private-ip vs public-ip (dest) for Tunnel Establishment when there is NAT present.
Example:
* If two ends have a private color: private IP address/port used for DTLS/TLS or IPSec
* If endpoint has public color: Public IP is used for DTLS/TLS or IPSec
c- Mutual Authentication:
- During the IKE negotiation process, the vEdges exchange their signed certificates, which are authenticated against the trusted root CA certificate to establish trust.
d- Key Exchange and Tunnel Formation:
- Once mutual authentication is successful, the vEdges establish symmetric encryption keys (Session keys are advertised through vSmart using OMP).
- The IPsec tunnel is then formed between the TLOCs of the vEdge devices, with each tunnel uniquely identified with source TLOC and destination TLOC (TLOC is a combination of system IP, color, and encapsulation type).
Note
By default, WAN Edge routers attempt to establish overlay tunnels with every reachable TLOC across all available WAN transports, including TLOCs associated with different colors, as long as there is IP connectivity between the transport networks.
However, vEDGE devices do not form overlay tunnels with other devices located within the same site, identified by having the same site-id.
Example below show output ipsec tunnels for vEDGE-1 To vEDGE-2 and vEDGE-3:
LearnDuty-vEdge1# show ipsec outbound-connections
SOURCE SOURCE DEST DEST REMOTE REMOTE AUTHENTICATION NEGOTIATED PEER PEER
IP PORT IP PORT SPI TUNNEL MTU TLOC ADDRESS TLOC COLOR USED KEY-HASH ENCRYPTION ALGORITHM TC SPIs KEY-HASH SPI
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
172.16.11.1 12386 172.16.12.1 12346 264 1441 10.120.0.12 mpls AH_SHA1_HMAC *****1105 AES-GCM-256 8 NONE 0
172.16.11.1 12386 172.16.13.1 12426 264 1441 10.130.0.13 mpls AH_SHA1_HMAC *****1501 AES-GCM-256 8 NONE 0
192.10.11.1 12346 192.10.12.1 12346 264 1441 10.120.0.12 public-internet AH_SHA1_HMAC *****2843 AES-GCM-256 8 NONE 0
192.10.11.1 12346 192.10.13.1 12346 264 1441 10.130.0.13 public-internet AH_SHA1_HMAC *****9dac AES-GCM-256 8 NONE 0Code language: CSS (css)e- Data Plane Communication:
- The IPsec tunnels encrypt all data plane traffic between the vEdge devices, ensuring secure and reliable communication as it traverses the underlay network.
- The tunnel state and performance metrics (e.g., loss, latency, jitter) are continuously monitored via the control connections with vSmart.
We will refer to the same Topology used when we performed the onboarding of the SD-WAN controllers:
I- vEDGE-1 Onboarding
We will use the CLI option first, (later on we will convert to config via templates), We define:
vEDGE-1 initial configuration
System information:
- Host name
- System-ip
- Site-id
- Organization name
- vbond
vedge(config)# system
vedge(config-system)# host-name LearnDuty-vEdge1
vedge(config-system)# system-ip 10.110.0.11
vedge(config-system)# site-id 11
vedge(config-system)# organization-name learnduty.com
vedge(config-system)# vbond 222.2.2.3
vedge(config-system)# exit
vedge(config)# commit
Commit complete.Code language: PHP (php)For the VPN0 interfaces:
- Assign an IP address to the interface
- Define the interface as a tunnel interface to participate in SD-WAN overlay (via “tunnel-interface”).
Under the tunnel-interface mode, we specify:- Encapsulation: IPSec
- Color (to identify WAN link type: MPLS, public-internet, etc)
- NAT : Enable NAT if the interface is connected to a network requiring NAT for external connectivity
- I’m also defining some static route to reach underlay network, we can use BGP instead.
LearnDuty-vEdge1(config-vpn-0)# interface ge0/0
LearnDuty-vEdge1(config-interface-ge0/0)# ip add 172.16.11.1/24
LearnDuty-vEdge1(config-interface-ge0/0)# no shutdown
LearnDuty-vEdge1(config-interface-ge0/0)# tunnel-interface
LearnDuty-vEdge1(config-tunnel-interface)# encapsulation ipsec
LearnDuty-vEdge1(config-tunnel-interface)# allow-service all
LearnDuty-vEdge1(config-tunnel-interface)# exit
LearnDuty-vEdge1(config-interface-ge0/0)# exit
LearnDuty-vEdge1(config-vpn-0)# interface ge0/1
LearnDuty-vEdge1(config-interface-ge0/1)# ip address 192.10.11.1/24
LearnDuty-vEdge1(config-interface-ge0/1)# no shutdown
LearnDuty-vEdge1(config-interface-ge0/1)# tunnel-interface
LearnDuty-vEdge1(config-tunnel-interface)# encapsulation ipsec
LearnDuty-vEdge1(config-tunnel-interface)# allow-service all
LearnDuty-vEdge1(config-tunnel-interface)# exit
LearnDuty-vEdge1(config-interface-ge0/1)# exit
LearnDuty-vEdge1(config-vpn-0)# int ge0/0
LearnDuty-vEdge1(config-interface-ge0/0)# tunnel-interface
LearnDuty-vEdge1(config-tunnel-interface)# color ?
Description: Set color for TLOC
Possible completions:
<3g biz-internet blue bronze custom1 custom2 custom3 default gold gre
en lte metro-ethernet mpls public-internet red silver private1 private2 pri
vate3 private4 private5 private6>[default]
LearnDuty-vEdge1(config-tunnel-interface)# color mpls
LearnDuty-vEdge1(config-tunnel-interface)# int ge0/1
LearnDuty-vEdge1(config-interface-ge0/1)# tunnel-interface
LearnDuty-vEdge1(config-tunnel-interface)# color public-internet
LearnDuty-vEdge1(config-tunnel-interface)# exit
LearnDuty-vEdge1(config-interface-ge0/1)# exit
LearnDuty-vEdge1(config)# vpn 0
LearnDuty-vEdge1(config-vpn-0)# ip route 172.16.12.0/24 172.16.11.2
LearnDuty-vEdge1(config-vpn-0)# ip route 172.16.13.0/24 172.16.11.2
LearnDuty-vEdge1(config-vpn-0)# ip route 222.2.2.0/24 172.16.11.2
LearnDuty-vEdge1(config-vpn-0)# commit
Commit complete.
LearnDuty-vEdge1(config-vpn-0)#
Code language: HTML, XML (xml)Certificate Installation on vEDGE-1
Request Root certificate from CA:
LearnDuty-vEdge1# vshell
LearnDuty-vEdge1:~$ tftp -g -r PKI.ca 222.2.2.4
earnDuty-vEdge1# request root-cert-chain install home/admin/PKI.ca
Uploading root-ca-cert-chain via VPN 0
Copying ... /home/admin/PKI.ca via VPN 0
Updating the root certificate chain..
Successfully installed the root certificate chainCode language: PHP (php)CA#
*Dec 22 16:05:22.794: TFTP: Server request for port 55063, socket_id 0xFCEAE40 for process 121
*Dec 22 16:05:22.794: TFTP: read request from host 172.16.11.1(55063) via GigabitEthernet0/0
*Dec 22 16:05:22.794: TFTP: Looking for PKI.ca
*Dec 22 16:05:22.803: TFTP: Opened flash0:PKI.ca, fd 0, size 1143 for process 121
*Dec 22 16:05:22.804: TFTP: Sending block 1 (retry 0), len 512, socket_id 0xFCEAE40
*Dec 22 16:05:22.826: TFTP: Received ACK for block 1, socket_id 0xFCEAE40
*Dec 22 16:05:22.827: TFTP: Sending block 2 (retry 0), len 512, socket_id 0xFCEAE40
*Dec 22 16:05:22.845: TFTP: Received ACK for block 2, socket_id 0xFCEAE40
*Dec 22 16:05:22.846: TFTP: Sending block 3 (retry 0), len 119, socket_id 0xFCEAE40
*Dec 22 16:05:22.866: TFTP: Received ACK for block 3, socket_id 0xFCEAE40
*Dec 22 16:05:22.867: TFTP: Finished flash0:PKI.ca, time 00:00:00 for process 121Code language: PHP (php)
Generate CSR:
LearnDuty-vEdge1# request csr upload home/admin/csr.txt
Uploading CSR via VPN 0
Enter organization-unit name : learnduty.com
Re-enter organization-unit name : learnduty.com
Generating private/public pair and CSR for this vedge device
Generating CSR for this vedge device ........[DONE]
Copying ... /home/admin/csr.txt via VPN 0
CSR upload successfulCode language: PHP (php)on the CA, Sign the vEDGE generated certificate and generate a granted certificate:
CA#crypto pki server PKI request pkcs10 terminal
PKCS10 request in base64 or pem
% Enter Base64 encoded or PEM formatted PKCS10 enrollment request.
% End with a blank line or "quit" on a line by itself.
-----BEGIN CERTIFICATE REQUEST-----
MIIDSzCCAjMCAQAwgcoxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh
MREwDwYDVQQHEwhTYW4gSm9zZTEWMBQGA1UECxMNbGVhcm5kdXR5LmNvbTEUMBIG
A1UEChMLVmlwdGVsYSBMTEMxQTA/BgNVBAMTOHZlZGdlLWJhYzg0NzM1LWIwYzAt
...
snip
8vmMQ+gwvSwV0cd5gKlMOCMTYkW3eCHsnTsg8naWTM6wZNSs5IzuMk8++DrlsS+q
3kgXsfbM1DRApuBJHn/Yhu32V970mucqjX2JQJNGAp56OYx1yuaAvqpxY3TxJL9L
fAHoxnA+bVwqcH9rXC+InDe+jYY7IZX97Dwkh6+WMBm14eX3YdcWRRq8l7v15hcP
GMo3wYmZAo0oQzbtmwdIlD5EdgKFK3KiAjC4lctyyQ==
-----END CERTIFICATE REQUEST-----
quit
% Granted certificate:
-----BEGIN CERTIFICATE-----
MIIDujCCAqKgAwIBAgIBBTANBgkqhkiG9w0BAQsFADAhMR8wHQYDVQQDExZyb290
Y2EubGVhcm5kdXR5LmxvY2FsMB4XDTI0MTIyMjE2MTAyNVoXDTI1MTIyMjE2MTAy
NVowgcoxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMREwDwYDVQQH
EwhTYW4gSm9zZTEWMBQGA1UECxMNbGVhcm5kdXR5LmNvbTEUMBIGA1UEChMLVmlw
dGVsYSBMTEMxQTA/BgNVBAMTOHZlZGdlLWJhYzg0NzM1LWIwYzAtNDhkYy05OTUy
...
SNIP
fdcbZ3QCnOcFr9h0GSjDimcwRAXJiail9MDSICIzSY8aKnUulcdscr/UeOTtLYFv
cYjNWHHCMeN/vWV6aniNVN/7H2pb6MU9ODspD2EnqZFOd6oVLVG1s5GmefI+QdmY
/XFjNWLDYAiJ9CUpzURTKiM0eRboq7EBA9ha8VVHzfmMsjBB2VGmKzU9suOB6Q==
-----END CERTIFICATE-----Code language: PHP (php)Copy granted certificate into cert.txt in vEDGE-1:
LearnDuty-vEdge1:~$ ls
PKI.ca archive_id_rsa.pub cert.txt csr.txt
Install the signed certificate on vEDGE-1:
LearnDuty-vEdge1# request certificate install home/admin/cert.txt
Installing certificate via VPN 0
Copying ... /home/admin/cert.txt via VPN 0
Successfully installed the certificateCode language: PHP (php)
Get the vEDGE serial which will be used to Authenticate with vBond:
LearnDuty-vEdge1# show certificate serial
Chassis number: bac84735-...-f6d40c8c1531 serial number: 05Code language: CSS (css)Register and Authenticate the vEDGE on vManage and vBond:
LearnDuty-vBond# request vedge add chassis-num bac84735-...-f6d40c8c1531 serial-num 05Code language: CSS (css)LearnDuty-vManage# request vedge add chassis-num bac84735-...-f6d40c8c1531 serial-num 05
LearnDuty-vManage#
Code language: PHP (php)
After few seconds, we can verify directly from vManage GUI that vEDGE-1 was added:
Next, we click on “Send to Controllers”:
Send to Controllers: Send the WAN edge router chassis and serial numbers to the controllers (vManage nodes, vSmart and vBonds).
Control connection state verification
At this point, the vSmart controller recognizes the vEdge and establishes a DTLS connection with it:
LearnDuty-vSmart# show control connections
PEER PEER
PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB
INDEX TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT REMOTE COLOR STATE UPTIME
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 default up 0:05:25:38
0 vmanage dtls 10.10.0.10 10 0 222.2.2.1 12346 222.2.2.1 12346 default up 0:04:54:02
1 vedge dtls 10.110.0.11 11 1 172.16.11.1 12366 172.16.11.1 12366 mpls up 0:00:00:09
1 vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 default up 0:05:25:38
Code language: CSS (css)
and OMP peering is established between vSmart and vEDGE-1:
LearnDuty-vSmart# show omp peers
R -> routes received
I -> routes installed
S -> routes sent
DOMAIN OVERLAY SITE
PEER TYPE ID ID ID STATE UPTIME R/I/S
------------------------------------------------------------------------------------------
10.110.0.11 vedge 1 1 11 up 0:00:00:48 0/0/0
Code language: PHP (php)The same process will be repeated for vEDGE-2 and vEDGE-3.
II- vEDGE-2 Onboarding
vEDGE-2 Initial configuration
vedge(config)# system
vedge(config-system)# host-name LearnDuty-vEdge2
vedge(config-system)# system-ip 10.120.0.12
vedge(config-system)# site-id 12
vedge(config-system)# organization-name learnduty.com
vedge(config-system)# vbond 222.2.2.3
vedge(config-system)# vpn 0
vedge(config-vpn-0)# interface ge0/0
vedge(config-interface-ge0/0)# ip address 172.16.12.1/24
vedge(config-interface-ge0/0)# ipv6 dhcp-client
vedge(config-interface-ge0/0)# tunnel-interface
vedge(config-tunnel-interface)# encapsulation ipsec
vedge(config-tunnel-interface)# color mpls
vedge(config-tunnel-interface)# allow-service all
vedge(config-tunnel-interface)# no shutdown
vedge(config-tunnel-interface)# interface ge0/1
vedge(config-interface-ge0/1)# ip address 192.10.12.1/24
vedge(config-interface-ge0/1)# tunnel-interface
vedge(config-tunnel-interface)# encapsulation ipsec
vedge(config-tunnel-interface)# color public-internet
vedge(config-tunnel-interface)# allow-service all
vedge(config-tunnel-interface)# no shutdown
vedge(config-tunnel-interface)# ip route 172.16.11.0/24 172.16.12.2
vedge(config-tunnel-interface)# ip route 172.16.13.0/24 172.16.12.2
vedge(config-tunnel-interface)# ip route 222.2.2.0/24 172.16.12.2
vedge(config-vpn-0)# commit
Commit complete.Certificate Installation on vEDGE-2
LearnDuty-vEdge2# ping 222.2.2.4
Ping in VPN 0
PING 222.2.2.4 (222.2.2.4) 56(84) bytes of data.
64 bytes from 222.2.2.4: icmp_seq=1 ttl=253 time=22.6 ms
64 bytes from 222.2.2.4: icmp_seq=2 ttl=253 time=15.5 ms
^C
--- 222.2.2.4 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 15.555/19.101/22.647/3.546 ms
LearnDuty-vEdge2# vshell
LearnDuty-vEdge2:~$ tftp -g -r PKI.ca 222.2.2.4
LearnDuty-vEdge2:~$ exit
exit
LearnDuty-vEdge2# request root-cert-chain install home/admin/PKI.ca
Uploading root-ca-cert-chain via VPN 0
Copying ... /home/admin/PKI.ca via VPN 0
Updating the root certificate chain..
Successfully installed the root certificate chain
LearnDuty-vEdge2# request csr upload home/admin/csr.txt
Uploading CSR via VPN 0
Enter organization-unit name : learnduty.com
Re-enter organization-unit name : learnduty.com
Generating private/public pair and CSR for this vedge device
Generating CSR for this vedge device ........[DONE]
Copying ... /home/admin/csr.txt via VPN 0
CSR upload successful
LearnDuty-vEdge2:~$ more csr.txtCode language: PHP (php)
Generate signed certificate:
Use the generated CSR on vEDGE-2 (csr.txt) to generate a granted certificate via the CA:
CA#crypto pki server PKI request pkcs10 terminal
PKCS10 request in base64 or pem
% Enter Base64 encoded or PEM formatted PKCS10 enrollment request.
% End with a blank line or "quit" on a line by itself.
-----BEGIN CERTIFICATE REQUEST-----
MIIDSzCCAjMCAQAwgcoxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh
MREwDwYDVQQHEwhTYW4gSm9zZTEWMBQGA1UECxMNbGVhcm5kdXR5LmNvbTEUMBIG
SNIP
jYc9b/dY7g+fqI7gHNCHsz2VF+gPr6BRZ/ILjKIYpGCodWy8OZ7L+BFq3iSnoi2Z
Z/LJwuzJtHz3A/kycmATcY5i1Bon3Cj2GZtutE+NhsM4RbcBiDTK3bTJLR97+TEZ
fwzmfQYmPb/XVJngcdJjaQxAI4gbEAKlb+dPrwR5bs4cYqwc/IJdZY9hjqzKzsu3
zAaYBjCKw6JA+fUSo6k7a6LBrKCrvKgjU9c9e6WBkQ==
-----END CERTIFICATE REQUEST-----
quit
% Granted certificate:
-----BEGIN CERTIFICATE-----
MIIDujCCAqKgAwIBAgIBBjANBgkqhkiG9w0BAQsFADAhMR8wHQYDVQQDExZyb290
Y2EubGVhcm5kdXR5LmxvY2FsMB4XDTI0MTIyMjE3MTAyM1oXDTI1MTIyMjE3MTAy
M1owgcoxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlhMREwDwYDVQQH
SNIP
rTiuIGNUJ9o/n7COeMugE6mYSq2QYpMU6KGVxLO1xCXJW8A11TLHnaKoBPjyDmKz
SLdTPB+Nd4OapYGXqtvUF3rCSkSijagByYDeRNwYHM5xWcscbFJ+0ni3HunIQqnK
nDwJTFfFTniBQivpDtu2y52+bMyhdGBum1fnJxGW4MMjiS2l+fx9aAeX4eAlM7UK
AFF2xJMC4L8dE5Hh3QfEU8Agpl/rhSIHlqJ2eGC5nX5TGzhWEK+VpxsJ13QQhQ==
-----END CERTIFICATE-----
Code language: PHP (php)
copy granted certificate into cert.txt in vEDGE2:
LearnDuty-vEdge2:~$ ls
PKI.ca archive_id_rsa.pub cert.txt csr.txtinstall certificate on vEDGE2:
LearnDuty-vEdge2# request certificate install home/admin/cert.txt
Installing certificate via VPN 0
Copying ... /home/admin/cert.txt via VPN 0
Successfully installed the certificate
LearnDuty-vEdge2# show certificate serial
Chassis number: 0a398bdf-1cf...6f6b2a3 serial number: 06Code language: PHP (php)Register vEDGE-2 to Controllers
Based on chassis number and serial numner, register vEDGE-2 to vManage and select “Send to Controllers”:
request vedge add chassis-num 0a398bdf-1cf...6f6b2a3 serial-num 06Code language: CSS (css)
Control connection state verification
- Verify the secure control connection (DTLS) state is UP:
LearnDuty-vSmart# show control connections
PEER PEER
PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB
INDEX TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT REMOTE COLOR STATE UPTIME
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 vedge dtls 10.120.0.12 12 1 172.16.12.1 12366 172.16.12.1 12366 mpls up 0:00:00:56
0 vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 default up 0:06:17:50
0 vmanage dtls 10.10.0.10 10 0 222.2.2.1 12346 222.2.2.1 12346 default up 0:05:46:14
1 vedge dtls 10.110.0.11 11 1 172.16.11.1 12366 172.16.11.1 12366 mpls up 0:00:52:21
1 vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 default up 0:06:17:51Code language: CSS (css)- and OMP peering established betwen vEDGE-2 and vSmart:
LearnDuty-vSmart# show omp peers
R -> routes received
I -> routes installed
S -> routes sent
DOMAIN OVERLAY SITE
PEER TYPE ID ID ID STATE UPTIME R/I/S
------------------------------------------------------------------------------------------
10.110.0.11 vedge 1 1 11 up 0:00:52:24 0/0/0
10.120.0.12 vedge 1 1 12 up 0:00:01:00 0/0/0Code language: PHP (php)III- vEDGE-3 Onboarding
vEDGE-3 Initial configuration
vedge# conf t
Entering configuration mode terminal
vedge(config)# system
vedge(config-system)# host-name LearnDuty-vEdge3
vedge(config-system)# system-ip 10.130.0.13
vedge(config-system)# site-id 13
vedge(config-system)# organization-name learnduty.com
vedge(config-system)# vbond 222.2.2.3
vedge(config-system)# vpn 0
vedge(config-vpn-0)# interface ge0/0
vedge(config-interface-ge0/0)# ip address 172.16.13.1/24
vedge(config-interface-ge0/0)# ipv6 dhcp-client
vedge(config-interface-ge0/0)# tunnel-interface
vedge(config-tunnel-interface)# encapsulation ipsec
vedge(config-tunnel-interface)# color mpls
vedge(config-tunnel-interface)# allow-service all
vedge(config-tunnel-interface)# no shutdown
vedge(config-tunnel-interface)# interface ge0/1
vedge(config-interface-ge0/1)# ip address 192.10.13.1/24
vedge(config-interface-ge0/1)# tunnel-interface
vedge(config-tunnel-interface)# encapsulation ipsec
vedge(config-tunnel-interface)# color public-internet
vedge(config-tunnel-interface)# allow-service all
vedge(config-tunnel-interface)# no shutdown
vedge(config-tunnel-interface)# ip route 172.16.11.0/24 172.16.13.2
vedge(config-tunnel-interface)# ip route 172.16.12.0/24 172.16.13.2
vedge(config-tunnel-interface)# ip route 222.2.2.0/24 172.16.13.2
vedge(config-vpn-0)# commit
Commit complete.Certificate Installation on vEDGE-3
Generate CSR based on Root CA certificate:
LearnDuty-vEdge3# ping 222.2.2.4
Ping in VPN 0
PING 222.2.2.4 (222.2.2.4) 56(84) bytes of data.
64 bytes from 222.2.2.4: icmp_seq=1 ttl=253 time=24.6 ms
64 bytes from 222.2.2.4: icmp_seq=2 ttl=253 time=17.3 ms
^C
--- 222.2.2.4 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 17.395/21.046/24.697/3.651 ms
LearnDuty-vEdge3# vshell
LearnDuty-vEdge3:~$ tftp -g -r PKI.ca 222.2.2.4
LearnDuty-vEdge3:~$ exit
exit
LearnDuty-vEdge3# request root-cert-chain install home/admin/PKI.ca
Uploading root-ca-cert-chain via VPN 0
Copying ... /home/admin/PKI.ca via VPN 0
Updating the root certificate chain..
Successfully installed the root certificate chain
LearnDuty-vEdge3# request csr upload home/admin/csr.txt
Uploading CSR via VPN 0
Enter organization-unit name : learnduty.com
Re-enter organization-unit name : learnduty.com
Generating private/public pair and CSR for this vedge device
Generating CSR for this vedge device ........[DONE]
Copying ... /home/admin/csr.txt via VPN 0
CSR upload successfulCode language: PHP (php)
Generate a signed certificate on the CA:
CA#crypto pki server PKI request pkcs10 terminal
PKCS10 request in base64 or pem
% Enter Base64 encoded or PEM formatted PKCS10 enrollment request.
% End with a blank line or "quit" on a line by itself.
-----BEGIN CERTIFICATE REQUEST-----
MIIDSzCCAjMCAQAwgcoxCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh
MREwDwYDVQQHEwhTYW4gSm9zZTEWMBQGA1UECxMNbGVhcm5kdXR5LmNvbTEUMBIG
... SNIP
gZevuXI3edg4sib+hKVB0zqx/eP4UrqRvCCCRYj0B/wjx3U4kEQ0z9AOTR9A7MjR
eDqLcHaOv8d+hljSx5axvwLKjZBEIFrWvXGyHPDHfA==
-----END CERTIFICATE REQUEST-----
quit
% Granted certificate:
-----BEGIN CERTIFICATE-----
MIIDujCCAqKgAwIBAgIBBzANBgkqhkiG9w0BAQsFADAhMR8wHQYDVQQDExZyb290
Y2EubGVhcm5kdXR5LmxvY2FsMB4XDTI0MTIyMjE3NDQwMFoXDTI1MTIyMjE3NDQw
... SNIP
VnnGgKjAW8oUYKNvvXSwJOW6+dca6QjQJ6BTWHGCHU9piM3vEKv2f0FV7psQRA==
-----END CERTIFICATE-----
Code language: PHP (php)
Install the granted certificate on vEDGE-3:
LearnDuty-vEdge3:~$ ls
PKI.ca archive_id_rsa.pub cert.txt csr.txt
LearnDuty-vEdge3# request certificate install home/admin/cert.txt
Installing certificate via VPN 0
Copying ... /home/admin/cert.txt via VPN 0
Successfully installed the certificate
LearnDuty-vEdge3# show certificate serial
Chassis number: 23b5da6d-...051d5 serial number: 07Code language: PHP (php)
Register vEDGE-3 to Controllers
request vedge add chassis-num 23b5da6d-...051d5 serial-num 07Code language: CSS (css)LearnDuty-vBond# request vedge add chassis-num 23b5da6d-...051d5 serial-num 07Code language: CSS (css)Control connection state verification
IV- Control plane and Data plane verification
LearnDuty-vManage# show control connections
PEER PEER PEER
PEER PEER PEER CONFIGURED SITE DOMAIN PEER PRIV PEER PUB
INDEX TYPE PROT SYSTEM IP SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT ORGANIZATION REMOTE COLOR STATE UPTIME
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 vedge dtls 10.120.0.12 10.120.0.12 12 1 172.16.12.1 12366 172.16.12.1 12366 learnduty.com mpls up 0:01:26:08
0 vedge dtls 10.130.0.13 10.130.0.13 13 1 172.16.13.1 12366 172.16.13.1 12366 learnduty.com mpls up 0:00:58:32
0 vsmart dtls 10.10.0.12 10.10.0.12 10 1 222.2.2.2 12346 222.2.2.2 12346 learnduty.com default up 0:07:09:57
0 vbond dtls 10.10.0.13 10.10.0.13 0 0 222.2.2.3 12346 222.2.2.3 12346 learnduty.com default up 0:08:48:10
1 vbond dtls 0.0.0.0 - 0 0 222.2.2.3 12346 222.2.2.3 12346 learnduty.com default up 0:08:48:13
2 vbond dtls 0.0.0.0 - 0 0 222.2.2.3 12346 222.2.2.3 12346 learnduty.com default up 0:08:48:28
3 vedge dtls 10.110.0.11 10.110.0.11 11 1 172.16.11.1 12366 172.16.11.1 12366 learnduty.com mpls up 0:02:31:11
3 vbond dtls 0.0.0.0 - 0 0 222.2.2.3 12346 222.2.2.3 12346 learnduty.com default up 0:08:48:29
Code language: CSS (css)LearnDuty-vBond# show orchestrator connections
PEER PEER
PEER PEER PEER SITE DOMAIN PEER PRIVATE PEER PUBLIC ORGANIZATION
INSTANCE TYPE PROTOCOL SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT REMOTE COLOR STATE NAME UPTIME
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 vedge dtls 10.110.0.11 11 1 172.16.11.1 12366 172.16.11.1 12366 mpls up learnduty.com 0:02:32:03
0 vedge dtls 10.120.0.12 12 1 172.16.12.1 12366 172.16.12.1 12366 mpls up learnduty.com 0:01:27:02
0 vedge dtls 10.130.0.13 13 1 172.16.13.1 12366 172.16.13.1 12366 mpls up learnduty.com 0:00:59:27
0 vsmart dtls 10.10.0.12 10 1 222.2.2.2 12346 222.2.2.2 12346 default up learnduty.com 0:07:42:27
0 vsmart dtls 10.10.0.12 10 1 222.2.2.2 12446 222.2.2.2 12446 default up learnduty.com 0:07:42:29
0 vmanage dtls 10.10.0.10 10 0 222.2.2.1 12346 222.2.2.1 12346 default up learnduty.com 0:08:49:23
0 vmanage dtls 10.10.0.10 10 0 222.2.2.1 12446 222.2.2.1 12446 default up learnduty.com 0:08:49:24
0 vmanage dtls 10.10.0.10 10 0 222.2.2.1 12546 222.2.2.1 12546 default up learnduty.com 0:08:49:23
0 vmanage dtls 10.10.0.10 10 0 222.2.2.1 12646 222.2.2.1 12646 default up learnduty.com 0:08:49:24
Code language: CSS (css)LearnDuty-vSmart# show control connections
PEER PEER
PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB
INDEX TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT REMOTE COLOR STATE UPTIME
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
0 vedge dtls 10.120.0.12 12 1 172.16.12.1 12366 172.16.12.1 12366 mpls up 0:01:23:40
0 vedge dtls 10.130.0.13 13 1 172.16.13.1 12366 172.16.13.1 12366 mpls up 0:00:55:53
0 vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 default up 0:07:40:35
0 vmanage dtls 10.10.0.10 10 0 222.2.2.1 12346 222.2.2.1 12346 default up 0:07:08:59
1 vedge dtls 10.110.0.11 11 1 172.16.11.1 12366 172.16.11.1 12366 mpls up 0:02:15:07
1 vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 default up 0:07:40:36
LearnDuty-vSmart# show omp peers
R -> routes received
I -> routes installed
S -> routes sent
DOMAIN OVERLAY SITE
PEER TYPE ID ID ID STATE UPTIME R/I/S
------------------------------------------------------------------------------------------
10.110.0.11 vedge 1 1 11 up 0:02:15:18 0/0/0
10.120.0.12 vedge 1 1 12 up 0:01:23:54 0/0/0
10.130.0.13 vedge 1 1 13 up 0:00:56:05 0/0/0
Code language: PHP (php)LearnDuty-vEdge3# show bfd sessions
SOURCE TLOC REMOTE TLOC DST PUBLIC DST PUBLIC DETECT TX
SYSTEM IP SITE ID STATE COLOR COLOR SOURCE IP IP PORT ENCAP MULTIPLIER INTERVAL(msec) UPTIME TRANSITIONS
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10.110.0.11 11 up mpls mpls 172.16.13.1 172.16.11.1 12366 ipsec 7 1000 0:01:00:09 0
10.120.0.12 12 up mpls mpls 172.16.13.1 172.16.12.1 12366 ipsec 7 1000 0:01:00:09 0
LearnDuty-vEdge3# show ipsec outbound-connections
SOURCE SOURCE DEST DEST REMOTE REMOTE AUTHENTICATION NEGOTIATED PEER PEER
IP PORT IP PORT SPI TUNNEL MTU TLOC ADDRESS TLOC COLOR USED KEY-HASH ENCRYPTION ALGORITHM TC SPIs KEY-HASH SPI
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
172.16.13.1 12366 172.16.11.1 12366 256 1441 10.110.0.11 mpls AH_SHA1_HMAC *****07fa AES-GCM-256 8 NONE 0
172.16.13.1 12366 172.16.12.1 12366 256 1441 10.120.0.12 mpls AH_SHA1_HMAC *****6e91 AES-GCM-256 8 NONE 0
Code language: CSS (css)Adding another transport connection
At this point, we only have one transport connection on vEDGEs via MPLS TLOC, that’s why we see Control status is “Partial”:
LearnDuty-vEdge1# show control connections
PEER PEER CONTROLLER
PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB GROUP
TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT LOCAL COLOR PROXY STATE UPTIME ID
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
vsmart dtls 10.10.0.12 10 1 222.2.2.2 12446 222.2.2.2 12446 mpls No up 0:01:12:53 0
vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 mpls - up 0:00:35:14 0
vmanage dtls 10.10.0.10 10 0 222.2.2.1 12646 222.2.2.1 12646 mpls No up 0:01:12:54 0Code language: CSS (css)LearnDuty-vEdge1# show bfd sessions
SOURCE TLOC REMOTE TLOC DST PUBLIC DST PUBLIC DETECT TX
SYSTEM IP SITE ID STATE COLOR COLOR SOURCE IP IP PORT ENCAP MULTIPLIER INTERVAL(msec) UPTIME TRANSITIONS
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10.120.0.12 12 up mpls mpls 172.16.11.1 172.16.12.1 12406 ipsec 7 1000 0:00:21:33 0
10.130.0.13 13 up mpls mpls 172.16.11.1 172.16.13.1 12386 ipsec 7 1000 0:00:21:33 0Code language: CSS (css)LearnDuty-vEdge1# show ipsec outbound-connections
SOURCE SOURCE DEST DEST REMOTE REMOTE AUTHENTICATION NEGOTIATED PEER PEER
IP PORT IP PORT SPI TUNNEL MTU TLOC ADDRESS TLOC COLOR USED KEY-HASH ENCRYPTION ALGORITHM TC SPIs KEY-HASH SPI
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
172.16.11.1 12346 172.16.12.1 12406 259 1441 10.120.0.12 mpls AH_SHA1_HMAC *****5764 AES-GCM-256 8 NONE 0
172.16.11.1 12346 172.16.13.1 12386 259 1441 10.130.0.13 mpls AH_SHA1_HMAC *****a161 AES-GCM-256 8 NONE 0Code language: CSS (css)
We will the connection via public-internet:
LearnDuty-vEdge1#
vpn 0
ip route 0.0.0.0/0 192.10.11.2Code language: PHP (php)Now, we see connection via “public-internet” with status “up”:
LearnDuty-vEdge1# show control connections
PEER PEER CONTROLLER
PEER PEER PEER SITE DOMAIN PEER PRIV PEER PUB GROUP
TYPE PROT SYSTEM IP ID ID PRIVATE IP PORT PUBLIC IP PORT LOCAL COLOR PROXY STATE UPTIME ID
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
vsmart dtls 10.10.0.12 10 1 222.2.2.2 12446 222.2.2.2 12446 mpls No up 0:01:20:43 0
vsmart dtls 10.10.0.12 10 1 222.2.2.2 12446 222.2.2.2 12446 public-internet No up 0:00:00:10 0
vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 mpls - up 0:00:43:03 0
vbond dtls 0.0.0.0 0 0 222.2.2.3 12346 222.2.2.3 12346 public-internet - up 0:00:00:11 0
vmanage dtls 10.10.0.10 10 0 222.2.2.1 12646 222.2.2.1 12646 mpls No up 0:01:20:44 0Code language: CSS (css)BFD session is up to every TLOC on other vEdges:
LearnDuty-vEdge1# show bfd sessions
SOURCE TLOC REMOTE TLOC DST PUBLIC DST PUBLIC DETECT TX
SYSTEM IP SITE ID STATE COLOR COLOR SOURCE IP IP PORT ENCAP MULTIPLIER INTERVAL(msec) UPTIME TRANSITIONS
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
10.120.0.12 12 up mpls mpls 172.16.11.1 172.16.12.1 12406 ipsec 7 1000 0:01:21:43 0
10.120.0.12 12 up public-internet public-internet 192.10.11.1 192.10.12.1 12366 ipsec 7 1000 0:00:01:11 0
10.130.0.13 13 up mpls mpls 172.16.11.1 172.16.13.1 12386 ipsec 7 1000 0:01:21:43 0
10.130.0.13 13 up public-internet public-internet 192.10.11.1 192.10.13.1 12386 ipsec 7 1000 0:00:01:12 0
Code language: CSS (css)LearnDuty-vEdge1# show ipsec outbound-connections
SOURCE SOURCE DEST DEST REMOTE REMOTE AUTHENTICATION NEGOTIATED PEER PEER
IP PORT IP PORT SPI TUNNEL MTU TLOC ADDRESS TLOC COLOR USED KEY-HASH ENCRYPTION ALGORITHM TC SPIs KEY-HASH SPI
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
172.16.11.1 12346 172.16.12.1 12406 259 1441 10.120.0.12 mpls AH_SHA1_HMAC *****5764 AES-GCM-256 8 NONE 0
172.16.11.1 12346 172.16.13.1 12386 259 1441 10.130.0.13 mpls AH_SHA1_HMAC *****a161 AES-GCM-256 8 NONE 0
172.16.11.1 12346 192.10.12.1 12366 258 1442 10.120.0.12 public-internet AH_SHA1_HMAC *****8dc8 AES-GCM-256 8 NONE 0
172.16.11.1 12346 192.10.13.1 12386 258 1442 10.130.0.13 public-internet AH_SHA1_HMAC *****8384 AES-GCM-256 8 NONE 0Code language: CSS (css)
