Symptom

[platform] Two same devices running same version `show version ` Cisco Nexus Operating System (NX-OS) Software TAC support: http://www.cisco.com/tac Documents: http://www.cisco.com/en/US/products/ps9372/tsd_products_support_series_home.html Copyright (c) 2002-2015, Cisco Systems, Inc. All rights reserved. The copyrights to certain works contained herein are owned by other third parties and are used and distributed under license. Some parts of this software are covered under the GNU Public License. A copy of the license is available at http://www.gnu.org/licenses/gpl.html. Software BIOS: version 3.6.0 Power Sequencer Firmware: Module 1: v3.0 Module 1: v1.0 Module 2: v5.0 Microcontroller Firmware: version v1.2.0.1 QSFP Microcontroller Firmware: Module not detected CXP Microcontroller Firmware: Module not detected kickstart: version 7.2(1)N1(1) system: version 7.2(1)N1(1) BIOS compile time: 05/09/2012 kickstart image file is: bootflash:///n5000-uk9-kickstart.7.2.1.N1.1.bin kickstart compile time: 10/14/2015 19:00:00 [10/15/2015 12:48:50] system image file is: bootflash:///n5000-uk9.7.2.1.N1.1.bin system compile time: 10/14/2015 19:00:00 [10/15/2015 15:01:24] Hardware cisco Nexus5548 Chassis ("O2 32X10GE/Modular Universal Platform Supervisor") Intel(R) Xeon(R) CPU with 8253868 kB of memory. Processor Board ID FOC17237QBS Device name: VDI_S_Edge_02 bootflash: 2007040 kB [ISSUE] Following logs are continuously seen on both N5K . N5K_1 : 2018 Feb 11 01:23:21 VDI_S_Edge_01 %SYSMGR-3-VAR_SYSMGR_FULL: System Manager file storage usage is unexpectedly high at 100%. This may impact system functions. 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 4512 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 4528 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 4585 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 4714 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 4724 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 4729 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-3-CORE_OP_FAILED: Core operation failed: copy_cores_from_core_regs_table: Core files: 6 exceeds max: 5 for non-sysmgr Core not saved, will be moved to logflash if exist 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 4739 with message ntpd(non-sysmgr) crashed, core will be saved . 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-CORE_SAVE_FAILED: zip_copy_file_to_logflash: PID 5231 with message Failed to copy core.4739 to logflash. Error file error (srcerr 0x0 dsterr 0x807b001c) . 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 4768 with message ntpd(non-sysmgr) crashed, core will be saved . 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-3-CORE_OP_FAILED: Core operation failed: copy_cores_from_core_regs_table: Core files: 8 exceeds max: 5 for non-sysmgr Core not saved, will be moved to logflash if exist 2018 Feb 11 01:24:02 VDI_S_Edge_01 %SYSMGR-2-CORE_SAVE_FAILED: zip_copy_file_to_logflash: PID 5231 with message Failed to copy core.4768 to logflash. Error file error (srcerr 0x0 dsterr 0x807b001c) . N5K_2 : 2018 Feb 11 02:10:30 VDI_S_Edge_02 %SYSMGR-3-VAR_SYSMGR_FULL: System Manager file storage usage is unexpectedly high at 100%. This may impact system functions. 2018 Feb 11 02:11:48 VDI_S_Edge_02 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 6119 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 02:11:48 VDI_S_Edge_02 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 6143 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 02:11:48 VDI_S_Edge_02 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 6153 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 02:11:48 VDI_S_Edge_02 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 6452 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-3-CORE_OP_FAILED: Core operation failed: copy_cores_from_core_regs_table: Core files: 6 exceeds max: 5 for non-sysmgr Core not saved, will be moved to logflash if exist 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 6489 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-3-CORE_OP_FAILED: Core operation failed: copy_cores_from_core_regs_table: Core files: 7 exceeds max: 5 for non-sysmgr Core not saved, will be moved to logflash if exist 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 6510 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-3-CORE_OP_FAILED: Core operation failed: copy_cores_from_core_regs_table: Core files: 8 exceeds max: 5 for non-sysmgr Core not saved, will be moved to logflash if exist 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 6516 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-3-CORE_OP_FAILED: Core operation failed: copy_cores_from_core_regs_table: Core files: 9 exceeds max: 5 for non-sysmgr Core not saved, will be moved to logflash if exist 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 6525 with message non-sysmgr(non-sysmgr) crashed, core will be saved . 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-3-CORE_OP_FAILED: Core operation failed: copy_cores_from_core_regs_table: Core files: 10 exceeds max: 5 for non-sysmgr Core not saved, will be moved to logflash if exis 2018 Feb 11 02:11:49 VDI_S_Edge_02 %SYSMGR-2-LAST_CORE_BASIC_TRACE: : PID 6531 with message non-sysmgr(non-sysmgr) crashed, core will be saved . Configuration of ntp on both N5K is same , it's not Ipv6 but bug mentions "this also affects IPv4-only": ntp server 70.121.5.245 use-vrf management ntp server 70.121.5.254 use-vrf management ntp source-interface mgmt0 ntp access-group peer NTP_SRV ip access-list NTP_SRV 1 permit ip 70.121.5.254/32 any 2 permit ip 70.121.5.245/32 any The differ between the two devices is core files seen on N5K_2 but not on N5K_1 : N5K_1 : `show cores` VDC Module Instance Process-name PID Date(Year-Month-Day Time) --- ------ -------- --------------- -------- ------------------------- N5K_2 : `show cores` VDC Module Instance Process-name PID Date(Year-Month-Day Time) --- ------ -------- --------------- -------- ------------------------- 1 1 1 ntpd 3696 2018-02-11 01:37:58 1 1 1 ntpd 4107 2018-02-11 01:37:59 Customer had run "clear core" for free up the space thus unclear for if the core files really not get generated on N5K_1 . We've collected the both core files from N5K_2 shown above , but failed for decoding using all of the tools we have . [logs and core files] Show tech from N5K_1 : VDI_S_Edge_01_tech.txt file in https://largefile-prod.cloudapps.cisco.com/getLargeFile/getLargeFile.php?appId=CSOne&srId=683951156&fileName=20180211-231227735_show-tech(1%ED%98%B8%EA%B8%B0%2C2%ED%98%B8%EA%B8%B0).zip Show tech and core files from N5K_2 : VDI_S_Edge_02_tech.txt file in https://largefile-prod.cloudapps.cisco.com/getLargeFile/getLargeFile.php?appId=CSOne&srId=683951156&fileName=20180211-231227735_show-tech(1%ED%98%B8%EA%B8%B0%2C2%ED%98%B8%EA%B8%B0).zip https://largefile-prod.cloudapps.cisco.com/getLargeFile/getLargeFile.php?appId=CSOne&srId=683951156&fileName=20180212-020207174_1518280679_0x101_ntpd_log.4107.tar.gz https://largefile-prod.cloudapps.cisco.com/getLargeFile/getLargeFile.php?appId=CSOne&srId=683951156&fileName=20180212-020207113_1518280678_0x101_ntpd_log.3696.tar.gz

Conditions

unclear

Workaround

no workaround , Issue is continuously happening on the two devices

Further Problem Description