Loading...
Loading...
You see that the /storage/log directory is full and in critical state the appliance manager page in VCSA 6.5. You see that large Java dump files are created at /storage/log/vmware/perfcharts You see that the alerts are triggered in the vCenter Server VAMI page You notice that the /storage/log is almost full root@VCENTER [ ~ ]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/autodeploy_vg-autodeploy 9.8G 33M 9.2G 1% /storage/autodeploy /dev/mapper/seat_vg-seat 25G 3.3G 20G 15% /storage/seat /dev/mapper/db_vg-db 9.8G 1.8G 7.5G 20% /storage/db /dev/mapper/updatemgr_vg-updatemgr 99G 757M 93G 1% /storage/updatemgr /dev/mapper/log_vg-log 74G 67G 4.2G 95% /storage/log /dev/mapper/core_vg-core 50G 52M 47G 1% /storage/core /dev/sda1 120M 28M 87M 24% /boot When you browse to /storage/log/vmware/perfcharts directory, you see lots of Java dump (.hprof) files. root@VCENTER [ /storage/log/vmware/perfcharts ]# ls -l total 67433848 -rw-r--r-- 1 perfcharts users 102100 May 19 19:03 java_error23079.log -rw-r--r-- 1 perfcharts users 106892 Mar 29 23:25 java_error45346.log -rw------- 1 perfcharts users 772752973 May 18 04:57 java_pid1290.hprof -rw------- 1 perfcharts users 775950352 Apr 23 22:59 java_pid13403.hprof -rw------- 1 perfcharts users 776215829 Apr 14 08:18 java_pid14007.hprof -rw------- 1 perfcharts users 776821858 Apr 10 16:00 java_pid14105.hprof -rw------- 1 perfcharts users 776148983 Apr 24 17:43 java_pid14437.hprof -rw------- 1 perfcharts users 775075757 Apr 30 03:48 java_pid14502.hprof -rw------- 1 perfcharts users 777414024 Mar 7 21:55 java_pid1463.hprof -rw------- 1 perfcharts users 777816472 Mar 3 14:35 java_pid15043.hprof -rw------- 1 perfcharts users 773803956 May 14 20:16 java_pid16276.hprof -rw------- 1 perfcharts users 776556635 Apr 19 04:38 java_pid17986.hprof -rw------- 1 perfcharts users 774637444 May 6 03:29 java_pid18867.hprof -rw------- 1 perfcharts users 776309973 Apr 17 14:34 java_pid19069.hprof Files are getting re-generated even after removing the java files by running the rm -rf /storage/log/vmware/perfcharts/java_pid*.hprof; command
To resolve this issue upgrade to vCenter Server 6.5 U1d, available at VMware Download.Note: Should the /storage/log partition be full but the symptoms in this KB don't match your issue, please see Troubleshooting disk space related issues: vCenter Server Appliance Maintenance Series (76563) for additional issues related to insufficient disk space.
To workaround this issue: Log in to vCenter Server appliance using SSH or console and enter the Bash shellTake a backup of the /etc/vmware/service-layout.mfx file by running this command: cp /etc/vmware/service-layout.mfx /etc/vmware/service-layout.mfx.bak; Stop the vmware-perfcharts service by running this command: service-control --stop vmware-perfcharts; Open the service-layout.mfx file using a text editor by running this command: vi service-layout.mfx; Change CompressClassSize size from 94 to 64 for the row vmware-perfchartsSave and close the service-layout.mfx fileStart the vmware-perfcharts service by running this command: service-control --start vmware-perfcharts; Now increase the allocated memory of the vmware-perfcharts service and restart it by running this command: cloudvm-ram-size -C 1024 vmware-perfcharts;service-control --stop vmware-perfcharts;service-control --start vmware-perfcharts;
Troubleshooting vCenter Appliance /storage/log directory is 80% or more full
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.