Symptom

You will see this syslog: %SYSMGR-2-TMP_DIR_FULL: System temporary directory usage is unexpectedly high at 100%. `show system internal flash` Filesystem 1K-blocks Used Available Use% Mounted on none 9265152 1494784 7770368 17% / /dev/loop0 65792 65792 0 100% /usr_ro aufs 9265152 1494784 7770368 17% /usr proc 0 0 0 - /proc none 0 0 0 - /sys none 204800 72640 132160 36% /var none 5120 3112 2008 61% /etc none 102400 1764 100636 2% /nxos/tmp none 81920 4 81916 1% /nxos/xlog none 81920 10552 71368 13% /nxos/dme_logs none 51200 5920 45280 12% /var/volatile/log none 5120 48 5072 1% /var/home none 307200 307200 0 100% /var/volatile/tmp <<<<<< /var/volatile/tmp will be full. /var/volatile/tmp/ will be filled with "csm_sh_run_acfg" files. CORE# run bash bash-4.3$ cd /var/volatile/tmp/ bash-4.3$ ls -lah total 11M drwxrwxrwx 35 root root 4.3K Jan 28 09:27 . drwxr-xr-x 5 root floppy 100 Apr 29 2020 .. -rw-rw-rw- 1 root root 0 Jan 27 10:58 acfg_maintenance_profile_dst_file_vdc1 -rw-rw-rw- 1 root root 0 Jan 27 10:58 acfg_maintenance_profile_src_file_vdc1 -rw-rw-rw- 1 root root 941 Jan 27 10:58 acfg_rollback_vdc_1_dst_excl_cfg_exec -rw-rw-rw- 1 root root 113 Jan 27 10:58 acfg_rollback_vdc_1_src_excl_cfg_exec -rw-rw-rw- 1 root root 5.1M Jan 24 13:16 auto_tmp_file_deletion_log.txt -rw-r--r-- 1 root root 512 May 28 2020 blkoops_pattern -rw-r--r-- 1 root root 1.2K May 28 2020 bootflash_sync.log -rw-rw-rw- 1 root root 92 May 28 2020 bootflash_virt_strg_pool_bf_vdc_1__rootfs.guestshell+.e2fsck.20200528214528 -rw-r--r-- 1 root root 1.6K May 28 2020 boot_uptime.log -rw-rw-rw- 1 root root 7 Jan 27 17:22 cfg_status.log_1 -rw-rw-rw- 1 root root 0 May 28 2020 cmp_slot_id.27 -rw-rw-r-- 1 user1 network-admin 52K Jan 25 04:02 csm_sh_run_acfg_10034.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 06:02 csm_sh_run_acfg_10204.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 00:02 csm_sh_run_acfg_11012.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 17:02 csm_sh_run_acfg_11322.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 19:02 csm_sh_run_acfg_11366.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 08:02 csm_sh_run_acfg_12447.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 18:02 csm_sh_run_acfg_12578.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 10:02 csm_sh_run_acfg_12591.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 17:02 csm_sh_run_acfg_12906.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 05:02 csm_sh_run_acfg_12907.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 04:02 csm_sh_run_acfg_13445.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 21:02 csm_sh_run_acfg_13789.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 23:02 csm_sh_run_acfg_13960.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 03:02 csm_sh_run_acfg_1401.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 12:02 csm_sh_run_acfg_14868.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 22:02 csm_sh_run_acfg_15075.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 11:02 csm_sh_run_acfg_15264.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 09:02 csm_sh_run_acfg_15295.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 05:02 csm_sh_run_acfg_1591.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 01:02 csm_sh_run_acfg_16222.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 03:02 csm_sh_run_acfg_16357.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 21:02 csm_sh_run_acfg_17185.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 14:02 csm_sh_run_acfg_17390.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 08:02 csm_sh_run_acfg_17443.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 16:02 csm_sh_run_acfg_17462.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 15:02 csm_sh_run_acfg_17864.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 14:02 csm_sh_run_acfg_18089.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 05:02 csm_sh_run_acfg_18634.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 07:02 csm_sh_run_acfg_18784.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 02:02 csm_sh_run_acfg_19071.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 01:02 csm_sh_run_acfg_19608.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 18:02 csm_sh_run_acfg_19903.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 20:02 csm_sh_run_acfg_19960.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 09:02 csm_sh_run_acfg_21039.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 19:02 csm_sh_run_acfg_21166.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 11:02 csm_sh_run_acfg_21183.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 06:02 csm_sh_run_acfg_21493.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 05:02 csm_sh_run_acfg_22040.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 22:02 csm_sh_run_acfg_22409.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 00:02 csm_sh_run_acfg_22569.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 18:02 csm_sh_run_acfg_23287.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 13:02 csm_sh_run_acfg_23537.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 23:02 csm_sh_run_acfg_23677.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 12:02 csm_sh_run_acfg_23848.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 23:02 csm_sh_run_acfg_2402.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 02:02 csm_sh_run_acfg_24817.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 04:02 csm_sh_run_acfg_24973.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 22:02 csm_sh_run_acfg_25782.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 15:02 csm_sh_run_acfg_25951.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 09:02 csm_sh_run_acfg_26032.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 17:02 csm_sh_run_acfg_26148.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 16:02 csm_sh_run_acfg_26553.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 15:02 csm_sh_run_acfg_26707.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 16:02 csm_sh_run_acfg_2710.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 06:02 csm_sh_run_acfg_27227.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 08:02 csm_sh_run_acfg_27376.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 18:02 csm_sh_run_acfg_2762.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 03:02 csm_sh_run_acfg_27659.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 02:02 csm_sh_run_acfg_28200.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 19:02 csm_sh_run_acfg_28488.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 21:02 csm_sh_run_acfg_28638.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 10:02 csm_sh_run_acfg_29632.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 20:02 csm_sh_run_acfg_29755.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 07:02 csm_sh_run_acfg_30062.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 06:02 csm_sh_run_acfg_30757.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 12:02 csm_sh_run_acfg_30795.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 23:03 csm_sh_run_acfg_31320.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 01:02 csm_sh_run_acfg_31420.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 17:02 csm_sh_run_acfg_3181.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 19:02 csm_sh_run_acfg_32142.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 14:02 csm_sh_run_acfg_32393.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 00:02 csm_sh_run_acfg_32534.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 16:02 csm_sh_run_acfg_3330.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 13:02 csm_sh_run_acfg_350.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 07:02 csm_sh_run_acfg_3846.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 09:02 csm_sh_run_acfg_3992.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 04:02 csm_sh_run_acfg_4303.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 03:02 csm_sh_run_acfg_4821.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 24 20:02 csm_sh_run_acfg_5113.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 10:02 csm_sh_run_acfg_5124.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 22:02 csm_sh_run_acfg_5258.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 11:02 csm_sh_run_acfg_6257.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 21:02 csm_sh_run_acfg_6462.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 08:02 csm_sh_run_acfg_6687.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 02:02 csm_sh_run_acfg_7791.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 20:02 csm_sh_run_acfg_8509.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 25 15:02 csm_sh_run_acfg_8732.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 26 13:02 csm_sh_run_acfg_8765.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 07:02 csm_sh_run_acfg_8848.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 28 01:02 csm_sh_run_acfg_8909.txt -rw-rw-r-- 1 user1 network-admin 52K Jan 27 14:02 csm_sh_run_acfg_9273.txt

Conditions

User runs a show command but disconnects the SSH connection before the output has been replayed to the terminal. Realistically this is only problematic when an automated user is performing this many times over an extended period.

Workaround

Wait until the output has completed before ending SSH sessions. If using an automated process to run show commands ensure that the script checks for the output to be complete before ending the SSH session.

Further Problem Description