Symptoms

An attempt to run getlogs was performed.The Avamar Utility Node's root partition "/" is at 100% capacity: df -h Filesystem Size Used Avail Use% Mounted on /dev/sda5 7.9G 7.9G 0 100% / udev 5.9G 236K 5.9G 1% /dev tmpfs 5.9G 0 5.9G 0% /dev/shm /dev/sda1 114M 80M 29M 74% /boot /dev/sda7 1.5G 217M 1.2G 16% /var /dev/sda10 256G 59G 198G 23% /data01 On investigating it was found that the /usr and /root directories are the ones which contain the most data under the root "/" partition: sudo du -xh / --max-depth=1 4.0K /var 505M /opt 4.8G /usr 556M /tmp 4.0K /bootalt 4.0K /mnt 0 /dev 12M /sbin 7.4M /bin 16K /lost+found 0 /sys 3.0K /boot 0 /proc 4.0K /media 4.0K /selinux 15M /srv 4.0K /varalt 15M /etc 174M /lib 4.0K /rootalt 84K /old_avamar 1.8G /root 11M /lib64 0 /data01 7.8G / Listing the size of the files in the /root directory we see temporary files named "/root/0.X" which would have been created after an attempt to run getlogs from within this partition: du -sh /root/* 147M /root/0.0 147M /root/0.1 143M /root/0.2 130M /root/0.3 147M /root/0.4 144M /root/0.5 154M /root/0.6 140M /root/0.7 140M /root/0.8 198M /root/0.s 8.0K /root/Ava_General_Logs.sh 4.0K /root/Raccourci vers Philippe_P.lnk 20K /root/actual_os_rpmlist.txt 872K /root/backups_20160928_104835 112K /root/backups_20160928_104835.tgz 1.4M /root/backups_20170118_092350 252K /root/backups_20170118_092350.tgz 4.0K /root/bin 16K /root/bootcheck.pl 4.0K /root/discover_nodes.sh 1.2M /root/dpnnetutilbg-work.10690 4.0K /root/dpnstat-avmvlzbck1eua 40K /root/etc 12K /root/getnodelogs 4.0K /root/hc_proactive_check.log 4.0K /root/hc_results.txt 56K /root/inst-sys 0 /root/logs.171114.152420.tar 4.0K /root/nohup.out 4.0K /root/ntp-slew.sh 260K /root/proactive_check.pl 8.0K /root/restore_stunnel 4.0K /root/serialnumber 20K /root/site_inventory_info 4.0K /root/step-tickers 64K /root/timesyncmon 16K /root/tmp

Cause

There was an attempt to run getlogs while in the /root directory and this might have failed due to error "No space left on the device".

Resolution

The temporary getnodelogs files are still present in the /root partition must be deleted manually.The following command removes the directories: rm -rf /root/0.? If the usage of the root partition "/" is still high, see the following: Avamar OS Capacity (Resolution Path)Avamar - A disk partition is full but not due to 'cur' or checkpoint overhead (RESOLUTION PATH)Avamar - how to identify large files or directories consuming a lot of disk space on an Avamar nodeAvamar root "/" Partition Full on Utility or AER Node and Not Able to RecoverAvamar - root "/" partition 100% due to tomcat logsRoot Partition Full Due to Tomcat Holding Deleted Log Files on Utility Node or Single NodeReplication logs rotation and clean upJava heap dump (.hprof) files fill up the / drive on the Avamar single node/utility node.Avamar 19.3 with SLES12.5 is generating the core dump file at /var/lib/systemd/coredump and causing /var partition 100% full

Support Cases