ACI Fabric Discovery
ACI Fabric discovery ( Preparing the Fabric Infrastructure )
This topic covers the discovery process for an ACI fabric in detail.
As soon as the ACI hardware installation is done, all Spines and Leafs are connected to each other and powered up and once basic configuration of APIC is completed, Fabric discovery process starts working.
In the discovery process, ACI uses Intra-Fabric Messaging (IFM) process in which APIC and nodes exchange heartbeat massage. The process used by the APIC to push policy to the fabric leaf nodes is called as IFM Process.
ACI Fabric discovery completes in three stages.
- In First stage the leaf node directly connected to the APIC is discovered.
- In the second stage of discovery brings in the spines connected to that initial leaf where APIC was connection.
- In third stage processes the discovery of the other leaf nodes and APICs in the cluster.
The Complete steps of IFM (Intra-Fabric Messaging) are as follows. After this all process is completed the fabric is ready for Production configuration.
- Link Layer Discovery Protocol (LLDP) Neighbor Discovery
- Tunnel End Point (TEP) IP address assignment to the node via DHCP
- Node software upgraded if necessary
- ISIS adjacency mode
- Certification Validation
- Start of DME Process on switches.
- Tunnel Setup (iVxlan)
- Policy Element IFM Setup
Node status may fluctuate between several states during the fabric registration process. The states are shown in the Fabric Node Vector table. The APIC CLI command to show the Fabric Node Vector table acidiag fnvread .
Following are the States and descriptions:
• Unknown – It states that Node discovered but no Node ID policy configured
• Undiscovered – It states that Node ID configured but not yet discovered
• Discovering – It states that Node discovered but IP not yet assigned
• Unsupported – It states that Node is not a supported model
• Disabled – when Node has been decommissioned, it will show Disabled
• Inactive – if you have No IP connectivity
• Active – When Node is active
ACI uses inter-fabric messaging (IFM) packets to communicate between the different nodes or between leaf and spine. These IFM packets are typically TCP packets, which are secured by 1024-bit SSL encryption, and the keys used for encryption are stored on secure storage. These keys are signed by Cisco Manufacturing Certificate Authority (CMCA). Any issues with IFM process can prevent fabric nodes communicating and from joining the fabric.
Below command shows the health of the registration process.
tcp 0 0 10.0.0.1:12151 0.0.0.0:* LISTEN
tcp 0 0 10.0.0.1:12215 0.0.0.0:* LISTEN
tcp 0 0 10.0.0.1:12471 0.0.0.0:* LISTEN
tcp 0 0 10.0.0.1:12279 0.0.0.0:* LISTEN
tcp 0 0 10.0.0.1:12567 10.0.248.29:49187 ESTABLISHED
tcp 0 0 10.0.0.1:12343 10.0.248.30:45965 ESTABLISHED
tcp 0 0 10.0.0.1:12343 10.0.248.31:47784 ESTABLISHED
tcp 0 0 10.0.0.1:12343 10.0.248.29:49942 ESTABLISHED
tcp 0 0 10.0.0.1:12343 10.0.248.30:42946 ESTABLISHED
tcp 0 0 10.0.0.1:50820 10.0.248.31:12439 ESTABLISHED
apic1# openssl s_client -state -connect 10.0.0.1:12151
SSL_connect:SSLv2/v3 write client hello A
SSL_connect:SSLv3 read server hello A
depth=1 O = Cisco Systems, CN = Cisco Manufacturing CA
verify error:num=19:self signed certificate in certificate chain
SSL_connect:SSLv3 read server certificate A
SSL_connect:SSLv3 read server key exchange A
SSL_connect:SSLv3 read server certificate request A
SSL_connect:SSLv3 read server done A
SSL_connect:SSLv3 write client certificate A
SSL_connect:SSLv3 write client key exchange A
SSL_connect:SSLv3 write change cipher spec A
SSL_connect:SSLv3 write finished A
SSL_connect:SSLv3 flush data
SSL3 alert read:fatal:handshake failure
SSL_connect:failed in SSLv3 read server session ticket A
139682023904936:error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure:s3_pkt.c:1300:SSL alert number 40
139682023904936:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:s23_lib.c:177:
0 s:/CN=serialNumber=PID:APIC-SERVER-L1 SN:TEP-1-1, CN=TEP-1-1
i:/O=Cisco Systems/CN=Cisco Manufacturing CA
1 s:/O=Cisco Systems/CN=Cisco Manufacturing CA
i:/O=Cisco Systems/CN=Cisco Manufacturing CA
-----BEGIN CERTIFICATE----- -----END CERTIFICATE-----
subject=/CN=serialNumber=PID:APIC-SERVER-L1 SN:TEP-1-1, CN=TEP-1-1
issuer=/O=Cisco Systems/CN=Cisco Manufacturing CA
No client certificate CA names sent
SSL handshake has read 2171 bytes and written 210 bytes
New, TLSv1/SSLv3, Cipher is DHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: zlib compression
Protocol : TLSv1.2
Cipher : DHE-RSA-AES256-GCM-SHA384
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Compression: 1 (zlib compression)
Start Time: 1481059783
Timeout : 300 (sec)
Verify return code: 19 (self signed certificate in certificate chain)