Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
This is what seen on FS4 for packet drop reason [31] Mar 9 07:38:03.314, len: 996, hits: 1, buffhdr type: 1 i/p i/f: Bundle-Ether111 punt reason: DROP_PACKET Ingress Headers: port_ifh: 0x10001b8, sub_ifh: 0x0, bundle_ifh: 0x800005c logical_port: 0x111, pd_pkt_type: 1 punt_reason: DROP_PACKET (0) payload_offset: 29, l3_offset: 29 FTMH: pkt_size: 0x403, tc: 1, tm_act_type: 0, ssp: 0xc308 PPH: pph_fwd_code: CPU Trap (7), fwd_hdr_offset: 0 inlif: 0x0, vrf: 0xffff, rif: 0x61c FHEI: trap_code: Rx_UNKNOWN_PACKET (63), trap_qual: 17 [ether dst: 0000.0000.0000 src: 3203.332a.6000 type/len: 0x800] [IPV4: source 6.6.11.221, dest 5.5.10.221 ihl 5, ver 4, tos 0 id 0, len 982, prot 61, ttl 64, sum 5527, offset 0] ON FS3 RP/0/RP0/CPU0:FS3-1#show mpls forwarding labels 78249 Fri Mar 9 08:00:45.822 PST Local Outgoing Prefix Outgoing Next Hop Bytes Label Label or ID Interface Switched ------ ----------- ------------------ ------------ --------------- ------------ 78249 74548 5.5.10.220/30 tt299 100.100.1.3 0 74548 5.5.10.220/30 tt300 100.100.1.3 0 RP/0/RP0/CPU0:FS3-1# RP/0/RP0/CPU0:FS3-1#show route 5.5.10.221 Fri Mar 9 07:50:31.009 PST Routing entry for 5.5.10.220/30 Known via "isis core", distance 115, metric 50, type level-2 Installed Mar 9 07:23:55.473 for 00:26:35 Routing Descriptor Blocks 100.100.1.3, from 192.168.85.2, via tunnel-te299 Route metric is 50 100.100.1.3, from 192.168.85.2, via tunnel-te300 Route metric is 50 No advertising protos. RP/0/RP0/CPU0:FS3-1#show cef 5.5.10.221 hardware egress detail location 0/4/CPU0 Fri Mar 9 07:50:41.407 PST 5.5.10.220/30, version 1039150, internal 0x1000001 0x0 (ptr 0x896a8548) [1], 0x0 (0x993b69b8), 0xa28 (0x91fce948) Updated Mar 9 07:38:03.087 Prefix Len 30, traffic index 0, precedence n/a, priority 3 gateway array (0x98586738) reference count 3006, flags 0x68, source lsd (5), 1 backups [1003 type 5 flags 0x8401 (0x977d5308) ext 0x0 (0x0)] LW-LDI[type=5, refc=3, ptr=0x993b69b8, sh-ldi=0x977d5308] gateway array update type-time 1 Mar 9 07:24:04.033 LDI Update time Mar 9 07:24:04.033 LW-LDI-TS Mar 9 07:38:03.086 via 100.100.1.3/32, tunnel-te299, 3 dependencies, weight 0, class 0 [flags 0x0] path-idx 0 NHID 0x0 [0x9509c120 0x0] next hop 100.100.1.3/32 local adjacency local label 78249 labels imposed {74548} via 100.100.1.3/32, tunnel-te300, 3 dependencies, weight 0, class 0 [flags 0x0] path-idx 1 NHID 0x0 [0x9509bca0 0x0] next hop 100.100.1.3/32 local adjacency local label 78249 labels imposed {74548} LEAF - HAL pd context : sub-type : IPV4, ecd_marked:0, has_collapsed_ldi:0 collapse_bwalk_required:0, ecdv2_marked:0, HW Walk: LEAF: PI:0x30896a8548 PD:0x30896a85e8 Rev:11792702 type: IPV4 (0) FEC key: 0x131a902000c84 leaf: 5.5.10.220/30 push_label: 0 LWLDI: PI:0x30993b69b8 PD:0x30993b69f8 rev:11792701 p-rev:11751176 11751176 ldi type:IMP_EOS0_EOS1 FEC key: 0x13f60c40000c84 fec index: 0x0(0) num paths:2, bkup paths: 0 IMP LDI: IMP pattern:3 PI:0x30993b69b8 PD:0x30993b69f8 rev:11792701 p-rev:11751176 11751176 FEC key: 0x131a902000c84 fec index: 0x2000030f(783) num paths:2 Path:0 fec index: 0x20004619(17945) DSP fec index: 0x2000d65a(54874) MPLS encap key: 0xf1b000004001e37a MPLS encap id: 0x4001e37a Remote: 0 Path:1 fec index: 0x2000461a(17946) DSP fec index: 0x2000d656(54870) MPLS encap key: 0xf1b000004001e37b MPLS encap id: 0x4001e37b Remote: 0 SHLDI: PI:0x30977d5308 PD:0x30977d5388 rev:11751176 dpa-rev:108666747 flag:0x0 FEC key: 0x13f60c40000c84 fec index: 0x2000000f(15) num paths: 2 bkup paths: 0 p-rev:11700388 11700392 Path:0 fec index: 0x20004018(16408) DSP:0xc000008 Dest fec index: 0x2000d65a(54874) Path:1 fec index: 0x20004019(16409) DSP:0xc000008 Dest fec index: 0x2000d656(54870) TE-NH: Rev: 11700388 p-rev: 11622359 11686352 11699757 dpa-rev: 80726463 PD: 0x309509c1a0 Flag: HW Protect MP reopt_trigger , Push: 0x4001430c Swap: 0 TE IFH: 0x8000964 llabel: 65661 FEC key: 0xc000000d08000964 FEC index: 0x2000d65a(54874) TE LL(encap: 0x8007d64416f220000000000000 IFH: 0x8007d64) or SRTE FEC key: 0 MPush: 0x4001430e MSwap: 0, Bkup TE(PD: 0x30953668a0 Push: 0x400140b9 Swap: 0) PROT-FRRNH: PI:(nil) PD:0x30921dea50 Rev: 11686352 Prot_eng dpa-rev: 75368022 dpa-rev: 75368022 FRR Active: 0, FRR Protect Handle: 0x3097dee218, Id 0x40000004, Status 0 BKUP-FRRNH: PI:(nil) PD:0x30921de600 Rev: 11686353, p-rev: 11533299 11686352 , Flag:0x0 TE-NH: Rev: 11700392 p-rev: 11622359 11686352 11699761 dpa-rev: 80726467 PD: 0x309509bd20 Flag: HW Protect MP reopt_trigger , Push: 0x40014310 Swap: 0 TE IFH: 0x800096c llabel: 65662 FEC key: 0xc000000d0800096c FEC index: 0x2000d656(54870) TE LL(encap: 0x8007d64416f220000000000000 IFH: 0x8007d64) or SRTE FEC key: 0 MPush: 0x40014312 MSwap: 0, Bkup TE(PD: 0x30953668a0 Push: 0x400140b9 Swap: 0) PROT-FRRNH: PI:(nil) PD:0x30921dea50 Rev: 11686352 Prot_eng dpa-rev: 75368022 dpa-rev: 75368022 FRR Active: 0, FRR Protect Handle: 0x3097dee218, Id 0x40000004, Status 0 BKUP-FRRNH: PI:(nil) PD:0x30921de600 Rev: 11686353, p-rev: 11533299 11686352 , Flag:0x0 Load distribution: 0 1 (refcount 1003) Hash OK Interface Address 0 Y tunnel-te299 point2point 1 Y tunnel-te300 point2point RP/0/RP0/CPU0:FS3-1#
GR needs to be enabled on both ends RP/0/RP0/CPU0:FS3-1#show running-config mpls ldp Thu Mar 8 16:29:40.249 PST mpls ldp nsr graceful-restart graceful-restart reconnect-timeout 600 graceful-restart forwarding-state-holdtime 600 igp sync delay on-proc-restart 60 discovery targeted-hello holdtime 30 targeted-hello interval 10 ! router-id 100.100.1.2 session protection address-family ipv4 discovery targeted-hello accept neighbor 100.100.1.8 targeted label local allocate for ldp_filter ! ! ! interface Bundle-Ether3 ! interface Bundle-Ether4 ! interface Bundle-Ether66 ! interface Bundle-Ether67 ! interface Bundle-Ether68 ! interface Bundle-Ether109 ! interface Bundle-Ether110 ! interface Bundle-Ether111 ! interface Bundle-Ether112 ! interface Bundle-Ether116 RP/0/RP0/CPU0:FS3-1#show mpls ldp neighbor brief Thu Mar 8 16:29:48.026 PST Peer GR NSR Up Time Discovery Addresses Labels ipv4 ipv6 ipv4 ipv6 ipv4 ipv6 ----------------- -- --- ---------- ---------- ---------- ------------ 100.100.1.7:0 Y Y 00:31:50 2 0 14 0 3011 0 100.100.1.6:0 Y Y 00:31:50 3 0 9 0 3011 0 100.100.1.3:0 Y Y 00:31:46 3 0 19 0 3011 0 100.100.1.8:0 Y Y 00:31:46 7 0 17 0 3011 0 RP/0/RP0/CPU0:FS3-1#show mpls ldp graceful-restart Thu Mar 8 16:29:53.662 PST Forwarding State Hold timer : Not Running GR Neighbors : 4 Neighbor ID Up Connect Count Liveness Timer Recovery Timer --------------- -- ------------- ------------------ ------------------ 100.100.1.3 Y 1 - - 100.100.1.6 Y 1 - - 100.100.1.7 Y 1 - - 100.100.1.8 Y 1 - - RP/0/RP0/CPU0:FS3-1# RP/0/RP0/CPU0:FS3-1#process restart mpls_ldp Thu Mar 8 16:30:50.449 PST RP/0/RP0/CPU0:FS3-1#show mpls ldp neighbor brief Thu Mar 8 16:30:53.602 PST RP/0/RP0/CPU0:FS3-1#show mpls ldp graceful-restart Thu Mar 8 16:30:58.130 PST Forwarding State Hold timer : Running (592 sec remaining) GR Neighbors : 1 Neighbor ID Up Connect Count Liveness Timer Recovery Timer --------------- -- ------------- ------------------ ------------------ 100.100.1.8 Y 1 - - RP/0/RP0/CPU0:FS3-1#show mpls ldp neighbor brief Thu Mar 8 16:31:04.868 PST Peer GR NSR Up Time Discovery Addresses Labels ipv4 ipv6 ipv4 ipv6 ipv4 ipv6 ----------------- -- --- ---------- ---------- ---------- ------------ 100.100.1.8:0 Y N 00:00:09 7 0 17 0 3011 0 100.100.1.7:0 Y N 00:00:05 2 0 14 0 3011 0 100.100.1.3:0 Y N 00:00:01 3 0 19 0 3011 0 RP/0/RP0/CPU0:FS3-1#show mpls ldp graceful-restart Thu Mar 8 16:31:08.712 PST Forwarding State Hold timer : Running (582 sec remaining) GR Neighbors : 4 Neighbor ID Up Connect Count Liveness Timer Recovery Timer --------------- -- ------------- ------------------ ------------------ 100.100.1.3 Y 1 - - 100.100.1.6 Y 1 - - 100.100.1.7 Y 1 - - 100.100.1.8 Y 1 - - RP/0/RP0/CPU0:FS3-1#
This will be fixed in 7.0.1. This has been existed since 6.x Graceful restart works as following: When LDP GR is enabled, LDP passes a ?flag? to FIB through LSD to let FIB know that GR is enabled. So now, when LDP is restarted, it gets a update from RIB but at this point in time it wouldn't have learnt the out label info and forwards an update to LSD with ?UNKNOWN? label as out label. LSD will forward this info to FIB and when FIB sees an update with ?UNKNOWN? label, it checks if GR specific flag has been set. If it has been set, then FIB will use the previous out label info until it gets an update when LDP learns the out label info. It gets a little bit complicated now with encap id and here's the issue: When LSD gets an update from LDP regardless of out label, LSD creates an encap id per path. LSD will now create an encap ID for the ?UNKNOWN? out label and send it to FIB. FIB will see this as an update to what it had before and updates it. Subsequent update(s) from LDP through LSD will have expected out label and LSD will now pass the same encap id it had before and FIB will update itself. It's between this first and second update the traffic loss is seen. This has been an issue since day 1 of encap ID.