Symptom

1. NXOS Upgrading from version 6.1.3 to 6.2.6 to 2. Upgraded issued via Active Supervisor is Module 5 3. Standard ISSU Upgrade procedures ?install all kickstart bootflash:n7000-s2-kickstart.6.2.6.bin system system bootflash:n7000-s2-dk9.6.2.6.bin? 4. Upgraded failed with the following Module, Redundancy and VDC statuss aa6-nec-7010-03# show module Mod Ports Module-Type Model Status --- ----- ----------------------------------- ------------------ ---------- 1 32 10 Gbps Ethernet Module N7K-M132XP-12 powered-dn 2 48 10/100/1000 Mbps Ethernet Module N7K-M148GT-11 ok 3 48 1000 Mbps Optical Ethernet Module N7K-M148GS-11 ok 4 48 1/10 Gbps Ethernet Module N7K-F248XP-25 ok 5 0 Supervisor Module-2 N7K-SUP2E active * 6 0 Supervisor Module-2 powered-up 7 48 1/10 Gbps Ethernet Module N7K-F248XP-25 ok 8 32 10 Gbps Ethernet Module N7K-M132XP-12 ok 10 6 10/40 Gbps Ethernet Module N7K-M206FQ-23L powered-dn aa6-nec-7010-03# show redundancy status Redundancy mode --------------- administrative: HA operational: Warm This supervisor (sup-5) ----------------------- Redundancy state: Active Supervisor state: Active Internal state: Active with HA standby Other supervisor (sup-6) ------------------------ Redundancy state: Standby Supervisor state: HA standby Internal state: HA synchronization in progress System start time: Fri Feb 14 19:12:29 2014 System uptime: 26 days, 15 hours, 2 minutes, 59 seconds Kernel uptime: 26 days, 15 hours, 5 minutes, 46 seconds Active supervisor uptime: 26 days, 15 hours, 2 minutes, 59 seconds aa6-nec-7010-03# show vdc Switchwide mode is m1 f1 m1xl f2 m2xl f2e f3 vdc_id vdc_name state mac type lc ------ -------- ----- ---------- --------- ------ 1 aa6-nec-7010-03 active 00:26:51:c9:34:41 Ethernet m1 m1xl m2xl 2 aa6-nec-7010-03-fp failed 00:26:51:c9:34:42 Ethernet m1 f1 m1xl m2xl 4 aa6-nec-7010-07-fp active 00:26:51:c9:34:44 Ethernet m1 f1 m1xl m2xl 5. Tried to manual upgrade standby Supervisor 6 but failed 6. Cannot Upgrade or Downgrade at this point Log file contained many of the following messages: 2014 Feb 17 10:15:57.127 aa6-nec-7010-03 %SYSMGR-2-NON_VOLATILE_DB_INODE_FULL: System non-volatile inode storage usage is unexpectedly high at 100%.

Conditions

Seen once during ISSU between 6.1.3 to Nexus 6.2.6 on a Nexus 7010 Chassis.

Workaround

Action 1 : Get output of "show system internal flash" to check usage data for all file systems. Action 2 : Load debug plugin and check which directory/files are using the most space. You can use the following commands: du -sc - shows total space usage of a directory (including sub-directories) ls -alS | head - shows files taking up the highest space within a directory Then delete unnecessary files, clear up the space and attempt ISSU again

Further Problem Description