...
Switch rebooted with following reset reason. *************** module reset reason (1) ************* 0) At 2020-03-24T12:31:20.398-05:00 Reason: kernel-panic Service:system crash Version: 14.2(2f) ---------------------------- Module: 1 ---------------------------- Last log in OBFL was written at time Wed Mar 25 11:46:53 2020 Reset Reason for this card: Image Version : 14.2(2f) Reset Reason (LCM): Unknown (0) at time Tue Mar 24 12:43:44 2020 Reset Reason (SW): Kernel Panic (19) at time Tue Mar 24 12:31:21 2020 Service (Additional Info): system crash Reset Reason (HW): Kernel Panic (19) at time Tue Mar 24 12:43:44 2020 Reset Cause (HW): 0x0c at time Tue Mar 24 12:43:44 2020 Reset internal (HW): 0x00 at time Tue Mar 24 12:43:44 2020 Last log in OBFL was written at time Tue Mar 24 11:47:07 2020
show logging onboard stack-trace << shows following output. [6998079.298862] vsan 730, vlan12 [6998079.298867] Transmitting packet on tx_dev: ffff88083329e1c0, len = 84 [6998079.299922] fc2_add_vegas2_hdr_aci vhe->vsan : 730 proto: 0x8906 [6998079.299927] Received FCoE Packet [6998679.327218] FC2 DI 0xffc0, SI 0xffc0 [6998679.327223] Sup hi prio idx from fcfwd_idx_from_intf: 0xffff [6998679.327225] SI set in Vegas hdr: 0xffff [6998679.327228] vsan 730, vlan12 [6998679.327234] Transmitting packet on tx_dev: ffff88083329e1c0, len = 84 [6998679.328201] fc2_add_vegas2_hdr_aci vhe->vsan : 730 proto: 0x8906 [6998679.328212] Received FCoE Packet [6999279.360307] FC2 DI 0xffc0, SI 0xffc0 [6999279.360311] Sup hi prio idx from fcfwd_idx_from_intf: 0xffff [6999279.360313] SI set in Vegas hdr: 0xffff [6999279.360316] vsan 730, vlan12 [6999279.360322] Transmitting packet on tx_dev: ffff88083329e1c0, len = 84 [6999279.361376] fc2_add_vegas2_hdr_aci vhe->vsan : 730 proto: 0x8906 [6999279.361387] Received FCoE Packet [6999381.436836] MACHINE CHECK ERROR [6999381.436838] MACHINE CHECK ERROR [6999381.436839] MACHINE CHECK ERROR [6999381.436840] MACHINE CHECK ERROR [6999381.436842] MACHINE CHECK ERROR [6999381.436844] MACHINE CHECK ERROR [6999381.436845] MACHINE CHECK ERROR [6999381.436847] MACHINE CHECK ERROR [6999381.436848] MACHINE CHECK ERROR [6999381.436852] MACHINE CHECK ERROR [6999381.436854] MACHINE CHECK ERROR [6999381.436855] MACHINE CHECK ERROR [6999381.437207] cctrli: SUP/TOR NMI handler called. cmd: 1 [6999381.437207] cctrli: SUP/TOR NMI handler called [6999381.437213] @@@cctrli: wrote 2 to scratch RR [6999381.437996] nvram_klm wrote rr=2 rr_str=(null) to nvram [6999381.437997] (1585071080.399201) (03-24-2020 17:31:20 UTC) cctrl_cmn_reset_ports_handler: send signal to bfdc begin. [6999381.438061] Sending SIGUSR1 signal to bfdc process [6999382.349250] sbridge: HANDLING MCE MEMORY ERROR [6999382.404569] CPU 9: Machine Check Exception: 5 Bank 7: be00000000010091 [6999382.430183] (1585071081.391304) (03-24-2020 17:31:21 UTC) cctrl_cmn_reset_ports_handler: send signal to bfdc end. [6999382.430185] cctrl_cmn_reset_sfp_qsfp_phy_ports/1224: cctri2: disable all nl intr notify [6999382.430194] Sending SIGUSR1 signal to port_client process [6999382.430234] (1585071081.391355) (03-24-2020 17:31:21 UTC)cctrl2 card_index=21135, link flap done. [6999382.430235] Kernel panic - not syncing: FPGA watchdog
TBD
Software issue.