Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Some BGP peering not coming up between 2 directly connected devices on subinterfaces.
Nexus 93180YC-EX running version 9.2(2) DIrectly connected iBGP session over subinterfaces. Has 6 working and 3 non working BGP peerings on this parent interface.
Reload
From BGP debug: 2020 Mar 3 22:04:34.153477 bgp: [29881] () EVT: PARTIALWRITE: OPEN to 10.246.181.242 wanted to write 70 bytes, only -1 written: No route to host 2020 Mar 3 22:04:34.153496 bgp: [29881] () EVT: 10.246.181.242 cleaning up active peer setup, thread id 0x0 SWITCH2# ethanalyzer local interface inband display-filter "tcp && ip.addr==10.246.181.243" limit-captured-frames 0 Capturing on inband 2020-03-03 22:58:04.955996 10.246.181.242 -> 10.246.181.243 TCP 24307 > bgp [SYN] Seq=0 Win=29200 Len=0 MSS=1460 TSV=3357935997 TSER=0 WS=2 2020-03-03 22:58:05.958626 10.246.181.242 -> 10.246.181.243 TCP 24307 > bgp [SYN] Seq=0 Win=29200 Len=0 MSS=1460 TSV=3357936298 TSER=0 WS=2 2020-03-03 22:58:07.965293 10.246.181.242 -> 10.246.181.243 TCP 24307 > bgp [SYN] Seq=0 Win=29200 Len=0 MSS=1460 TSV=3357936900 TSER=0 WS=2 SWITCH2# show ip adjacency 10.246.181.242 vrf V-CNHI-IVECO-SPA-2499 Flags: # - Adjacencies Throttled for Glean G - Adjacencies of vPC peer with G/W bit R - Adjacencies learnt remotely CP - Added via L2RIB, Control plane Adjacencies PS - Added via L2RIB, Peer Sync RO - Re-Originated Peer Sync Entry IP Adjacency Table for VRF V-CNHI-IVECO-SPA-2499 Total number of entries: 1 Address MAC Address Pref Source Interface Flags 10.246.181.242 502f.a8cf.ce07 50 arp Ethernet1/15.107 SWITCH2# show forwarding adjacency 10.246.181.242 detail slot 1 ======= IPv4 adjacency information Module-1# show hardware internal tah l3 v4host | i 10.246.181.242 0/0/0x1dd5 | 10.246.181.242 | 15 | No | 0 | 0xd04ee | | | | |Y | | | | module-1# show hardware internal tah l3 10.246.181.242 table 15 DLeft location: 0x4001dd5 FP location : 0/0/0x1dd5 **EPE label *Flags: CC=Copy To CPU, SR=SA Sup Redirect, DR=DA Sup Redirect, TD=Bypass TTL Dec, DC=SA Direct Connect,DE=Route Default Entry, LI=Route Learn Info,HR=Host as Route HW Loc | Ip Entry | VRF | MPath | NumP | Base/L2ptr |CC|SR|DR|TD|DC|DE|LI|HR| -----------|----------------------|---------|-------|-------|------------|--|--|--|--|--|--|--|--| 0/0/0x1dd5 | 10.246.181.242 | 15 | No | 0 | 0xd04ee | | | | |Y | | | | AdjId | FP | BD | DMac | DstIdx | DstIsPtr | -----------|--------------|-------|-------------------|--------|----------| 0xd04ee | 13/0/0x4ee | 4190 | 50:2f:a8:cf:ce:07 | 57 | No | module-1# show hardware internal tah l3 adjacency 0xd04ee AdjId | FP | BD | DMac | DstIdx | DstIsPtr | -----------|--------------|-------|-------------------|--------|----------| 0xd04ee | 13/0/0x4ee | 4190 | 50:2f:a8:cf:ce:07 | 57 | No | module-1# show hardware internal tah l3 v4lpm | egrep 10.246.181.24 4/1121 | 15 | 10.246.181.242/31 | 0 | 0 | 0xd0000 | | | | | | | | | 7/212 | 15 | 10.246.181.242/31 | 0 | 0 | 0xfffff | | | | |Y | | | | KSTACK checks: bash-4.3# ip netns exec V-CNHI-IVECO-SPA-2499 ip -4 -o route 10.246.181.178/31 dev Eth1-27.107 proto kernel scope link src 10.246.181.179 10.246.181.242/31 dev Eth1-15.107 proto kernel scope link src 10.246.181.243 bash-4.3# We could see that the ARP was incomplete in KSTACK however to ensure that this did not occur due to time out or some other factor, we first cleared the ARP from NX-OS , performed a ping and then verified both Nexus and Kstack programming. Also, we took a working situation for a reference (10.246.181.254 on sub-interface belonging to the same parent interface where BGP was set up sucessfully) bash-4.3# ip netns exec arp -a | grep 10.246.181.254 ? (10.246.181.254) at 50:2f:a8:cf:ce:07 [ether] PERM on Eth1-15.101 bash-4.3# ip netns exec arp -a ? (10.246.181.242) at on Eth1-15.107 Additional log: We are unable to confirm that this is correlated as the coiunters did not increment at the time of the troubleshooting but from KSTACK perspective we see some drops for BGP TX-Drops and FDB_LOOKUP_FAILED TX_drops : bash-4.3# cat /proc/lcnd/lcnd_proto_stats Global Protocol stats: Protocol tx_pkts tx_bytes rx_pkts rx_bytes tx_drops rx_drops rx_alloced ipv4_frag_reid ipv6_frag_reid ARP 14042391 709206878 248520544 54315779004 0 0 0 0 0 LLDP 41551371 13757202510 20966384 6881828685 0 0 0 0 0 CDP 15638721 4225215284 13829089 5573240952 0 0 0 0 0 LACP 22648879 2808460996 22626677 5294642418 0 0 0 0 0 FEX 26444 8514968 26446 11424672 0 0 0 0 0 STP 7748549225 493791211883 2920121787 519585229722 0 0 0 0 0 OSPF 37698493 4262745958 43675767 10117514082 13 0 0 0 0 IGMP 19595295 1061207778 21261498 4577882918 0 0 0 0 0 BGP 404239757 28222340425 399120824 29772507844 21767 0 0 202133553 0 DHCP 0 0 1924313 585578504 0 0 0 0 0 TCP_OTHER 75 6884 2049277 669636067 0 0 0 25 0 UDP_OTHER 1851619 427272748 283796851 77490713483 0 0 0 794 0 IP_OTHER 0 0 149502508 32451595718 0 0 0 0 0 DIAG 2182724 1291083262 2182724 1400220038 0 0 0 0 0 INVALID 225109178 32703736235 152099385 37509078189 0 1749 0 0 0 BFD 0 0 4181976779 468404779188 0 0 0 0 0 FDB_LOOKUP_FAILED 0 0 0 0 21770 0 0 0 0 ICMP 663636 34359262 69689121 18072974926 0 0 0 0 0 PIM 54376945 4326392530 56836439 10553037718 0 0 0 0 0 bash-4.3#