Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Customer seeing link flaps on (FP40 | 4-10GbE) kind of HW. All the HW pertaining to the CRS has been swapped, still customer sees the issue Ex# LC/0/2/CPU0:Sep 19 02:21:41.715 : ifmgr[196]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/2/0/1, changed state to Down LC/0/2/CPU0:Sep 19 02:21:41.715 : ifmgr[196]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/2/0/1, changed state to Down LC/0/2/CPU0:Sep 19 02:22:02.773 : ifmgr[196]: %PKT_INFRA-LINK-3-UPDOWN : Interface TenGigE0/2/0/1, changed state to Up LC/0/2/CPU0:Sep 19 02:22:02.775 : ifmgr[196]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/2/0/1, changed state to Up Upon Investigation from the PD DE, the WANPHY stats on the SW and HW counters do not match. #show controllers wanphy 0/2/0/1 all Interface: wanphy0/2/0/1 Configuration Mode: WAN Mode SECTION LOF = 0, LOS = 0, BIP(B1) = 1572861 LINE AIS = 0, RDI = 0, FEBE = 103079215080, BIP(B2) = 103079215101 PATH AIS = 0, RDI = 0, FEBE = 0, BIP(B3) = 1572861 LOP = 0, NEWPTR = 0, PSE = 0, NSE = 0 WIS ALARMS SEF = 0, FEPLMP = 0, FEAISP = 0 WLOS = 0, PLCD = 0 LFEBIP = 103079215080, PBEC = 1572861, PLMP = 0 Active Alarms[All defects]: Active Alarms[Highest Alarms]: Rx(K1/K2): N/A, Tx(K1/K2): N/A S1S0 = N/A, C2 = N/A PATH TRACE BUFFER Remote IP addr: BER thresholds: SF = E-3 SD = E-6 TCA thresholds: N/A Alarm reporting enabled for:los, lof, path lop, line rdi, path ais, line ais, path lcd, path fe plm, path fe ais, path plm, path rdi, REGISTERS P_FEBE : 44136 L_FE_BIP: 1211409217 L_BIP : 9802 P_BEC : 32085 S_BIP : 193 J1-Rx0 : 0x 0 J1-Rx1 : 0x 0 J1-Rx2 : 0x 0 J1-Rx3 : 0x 0 J1-Rx4 : 0x 0 J1-Rx5 : 0x 0 J1-Rx6 : 0x 0 J1-Rx7 : 0x 0
Customer started seeing the issue after they converted their routers from 4.2.4 to 5.3.3
No workaround yet, too many link flaps seen, and they come up on their own. They are creating lots of alarms at the customer end due to these flaps.
Customer has been seeing an issue when they convert from 4.2.4 -> 5.3.3 specifically on port 0/2/0/1. Issue that Customer notices# LC/0/2/CPU0:Aug 4 06:53:25.535 : ifmgr[196]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/2/0/1, changed state to Down LC/0/2/CPU0:Aug 4 06:53:46.580 : ifmgr[196]: %PKT_INFRA-LINEPROTO-5-UPDOWN : Line protocol on Interface TenGigE0/2/0/1, changed state to Up On each of these routers, a specific port begins ?bouncing? in the logs after the 5.3.3 code upgrade date. -- Port = 0/2/0/1 The effect of the ?bounce? is normal, in that it *appears* to be a circuit bounce from the cluster perspective, and causes the bundles to auto-cost out, due to min-bandwidth statements. Customer has worked to troubleshoot every aspect of a circuit issue - (these routers did not have circuit troubles prior to the upgrade dates) Customer has engaged transport team at their end. ? transport team sees no hits. >>> In most cases, the Remote end of the link (to the CRS ) does NOT see a link bounce (it does see protocol timeouts) Customer has swapped XFPs, RMAd entire slot hardware (front & Back cards + XFPs) ? issue persists with new hardware. Customer has swapped hardware with known good working hardware from another slot, and the issue remains on 0/2/0/1 Customer has moved the circuit from slot 2 to slot 4, and the issue does not travel, remains a problem on slot 2, port 1. Customer has loop tested, and the routers report bounces in the logs, with a Hard loop at the port itself. Customer has fully removed the interface/controller configuration and added it back, but that has not changed the results. Customer has changed delay timers, but that has not had an impact either. Most recently, Customer has setup a test-set on the port 0/2/0/1 itself, and the test-set does Not register any errors or LOS during the ?bounce? events in the log.