OPERATIONAL DEFECT DATABASE
...
...
You see a "log disk exhaustion" warning on the vCenterChecking storage logs for disk utilization you find the vmafdd.log is extremely large. Example: sudo du -ah /storage/log/ | sort -h -r | head -n 20 7.0G /storage/log/vmware 7.0G /storage/log/ 1.4G /storage/log/vmware/vmafd/vmafdd.log 1.4G /storage/log/vmware/vmafd 768M /storage/log/vmware/lookupsvc 762M /storage/log/vmware/sso 757M /storage/log/vmware/eam Additionally you may see the following messages in the vmafdd log: 2021-02-04T08:41:15.517 [vmafdd][INFO] VecsSrvWriteCertOrCrlToDisk: cert/cl already exists as /etc/ssl/certs/d70df8d2.0, so will not write again. 2021-02-04T08:41:15.517 [vmafdd][INFO] VecsSrvWriteCertOrCrlToDisk: cert/cl already exists as /etc/ssl/certs/6349821d.0, so will not write again. 2021-02-04T08:41:15.517 [vmafdd][INFO] VecsSrvWriteCertOrCrlToDisk: cert/cl already exists as /etc/vmware-vpx/docRoot/certs/ d70df8d2.0, so will not write again. 2021-02-04T08:41:15.518 [vmafdd][INFO] VmAfdProcessCACerts: force flushing.
This happens because the logrotate.d configuration for the vmafd service expects the logs to be located in the following path: Expected path/var/log/vmware/vmafdd/vmafdd.logCurrent path/var/log/vmware/vmafd/vmafdd.log # cat /etc/logrotate.d/vmware-vmafd.lr /var/log/vmware/vmafdd/vmafdd.log <<< Here's where the log file is expected. { size 10M nodateext rotate 10 missingok notifempty compress create 644 root root postrotate /usr/sbin/killproc -L -HUP /usr/lib/vmware-vmafd/sbin/vmafdd endscript } Checking the vmafdd registry you see the incorrect path: /opt/likewise/bin/lwregshell list_values "[HKEY_THIS_MACHINE\Services\vmafd\Parameters]" root@vcsa1 [ ~ ]# /opt/likewise/bin/lwregshell list_values "[HKEY_THIS_MACHINE\Services\vmafd\Parameters]" + "CAPath" REG_SZ "/etc/ssl/certs" + "DCName" REG_SZ "vcsa1.domain.local" + "DCPort" REG_DWORD 0x000001bb (443) + "DomainGUID" REG_SZ "3dc76f89-4b73-41bc-bfff-9bcfd65f2496" + "DomainName" REG_SZ "vsphere.local" + "DomainState" REG_DWORD 0x00000001 (1) + "LDU" REG_SZ "c9a2071c-c517-4041-8d3f-3eb21207e651" + "LogFile" REG_SZ "/var/log/vmware/vmafd/vmafdd.log"
This issue is resolved in the following versions of vCenter Server. 7.0 Update 3o (7.0.3.01700)8.0 Update 1c (8.0.1.00300)
Note: Please ensure to create a fresh backup or offline snapshot (in powered off state) of the vCenter Server Appliance before implementing the steps below. In case the affected vCenter Server Appliance is part of an Enhanced Linked Mode (ELM) replication group, the backup or offline snapshots must be created for all of its replication partners as well. When restoring an ELM vCenter, all members of the ELM replication need to be restored too, otherwise there will be inconstencies in the VMDirectory LDAP database. To work around this issue, please apply the following steps: Update the registry to match the log rotate configuration path, using the following command: # /opt/likewise/bin/lwregshell set_value "[HKEY_THIS_MACHINE\Services\vmafd\Parameters]" "LogFile" "/var/log/vmware/vmafdd/vmafdd.log" Restart all services: # service-control --stop --all && service-control --start --all Afterwards you should move the old log /storage/log/vmware/vmafd/vmafdd.log out of the /storage/log partition or delete it to reduce storage utilization in the log partition.
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
