Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
The bootflash of a Nexus 3000 or 9000 series switches may be mounted in a read-only state if the model of the bootflash's SSD is a Micron M500IT drive. This can cause a number of symptoms, such as the following: 1. Syslogs from the "KERN" facility indicating bootflash, I/O, disk, or file system errors. Some examples are shown below. Note that not all of the below syslogs are required for a device to be symptomatic of this issue: switch# show logging logfile {snip} 2021 Jan 1 10:00:00 switch Jan 1 10:00:00 %KERN-3-SYSTEM_MSG: [9449604.288388] blk_update_request: I/O error, dev sda, sector 79155200 - kernel 2021 Jan 1 10:00:01 switch Jan 1 10:00:01 %KERN-3-SYSTEM_MSG: [9449605.284757] Buffer I/O error on dev sda7, logical block 0, lost sync page write - kernel 2021 Jan 1 10:00:02 switch Jan 1 10:00:02 %KERN-3-SYSTEM_MSG: [9449606.284757] EXT4-fs error (device sda3): ext4_find_entry:1457: inode #17: comm cat: reading directory lblock 0 - kernel 2021 Jan 1 10:00:03 switch Jan 1 10:00:03 %KERN-3-SYSTEM_MSG: [9449607.284757] EXT4-fs (sda3): previous I/O error to superblock detected - kernel 2021 Jan 1 10:00:04 switch Jan 1 10:00:04 %KERN-3-SYSTEM_MSG: [9449608.284757] PID 25202 with message Failed to copy core.23718 to logflash. Error file error (srcerr 0x0 dsterr 0x807b001e) . 2. Inability to copy or save files to the bootflash: switch# copy ftp://username:password@192.0.2.100/file.txt bootflash: {snip} local: /bootflash/file.txt: Permission denied Copy complete, now saving to disk (please wait)... Copy complete. switch# 3. Inability to save the device's configuration with the copy running-config startup-config command. switch# copy running-config startup-config [########################################] 100% Configuration update aborted: request was aborted switch# 4. The switch or supervisor's diagnostic test for the bootflash is in a "Failed" or "Error disabled" state. switch# show module {snip} Mod Ports Module-Type Model Status --- ----- ------------------------------------- --------------------- --------- 27 0 Supervisor Module N9K-SUP-B+ active * 28 0 Supervisor Module N9K-SUP-B+ ha-standby Mod Online Diag Status --- ------------------ 27 Fail 28 Pass switch# show diagnostic result module 27 {snip} Current bootup diagnostic level: complete Module 27: Supervisor Module (Active) Test results: (. = Pass, F = Fail, I = Incomplete, U = Untested, A = Abort, E = Error disabled) 7) BootFlash---------------------> E
This issue is observed on Nexus 3000 or 9000 series switches equipped with Micron M500 SSD drives. The show hardware internal bootflash model command can be used to verify the SSD drive model a switch is equipped with. An example of this command run on a switch with a Micron M500 SSD drive is shown below. switch# show hardware internal bootflash model ATA Micron_M500IT_MT
A reload of the switch via the reload command is required to work around this issue. After a reload of the switch, the bootflash of the switch will be in a read/write state as expected.
Although CSCvu05760 is marked as a duplicate of CSCvx21260, both defects track two separate issues related to the firmware running on the bootflash SSD. CSCvu05760 tracks an issue with the firmware where the firmware does not properly handle transient parity errors, which causes the bootflash to be re-mounted in a read-only state. CSCvx21260 tracks an issue with the firmware where a memory buffer driven by the SSD's POH overruns, which causes the bootflash to be re-mounted in a read-only state. The fix implemented in CSCvx21260 resolves the root cause of both CSCvu05760 and CSCvx21260. In other words, while CSCvu05760 tracks a different root cause than CSCvx21260, the code committed as part of CSCvx21260 fixes the root cause of both CSCvu05760 and CSCvx21260. This is why CSCvu05760 is marked as a duplicate of CSCvx21260. The symptoms and conditions described in Field Notice 72150 (which links to CSCvx21260) are not applicable to switches that encounter CSCvu05760 if the SSD's Power On Hours (POH) counter does not exceed the value defined in Field Notice 72150 (28,224 hours).