...
After configuration of remote syslog server in rsyslogd-syslog.log you see messages similar to:<YYYY-MM-DD><time>Z err rsyslogd imfile error trying to access state file for '/var/log/vmware/sso/tomcat/localhost_access.<YYYY-MM-DD>.log': Too many open files [v8.2001.0 try https://www.rsyslog.com/e/2027 ]<YYYY-MM-DD><time>Z err rsyslogd file '/var/log/vmware/sso/tomcat/localhost_access.<YYYY-MM-DD>.log': open error: Too many open files [v8.2001.0 try https://www.rsyslog.com/e/2433 ]<YYYY-MM-DD><time>Z err rsyslogd imfile: error accessing file '/var/log/vmware/sso/tomcat/localhost_access.<YYYY-MM-DD>.log': Too many open files [v8.2001.0]<YYYY-MM-DD><time>Z err rsyslogd imfile: error accessing file '/var/log/vmware/sso/tomcat/localhost_access.<YYYY-MM-DD>.log': Too many open files [v8.2001.0] Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
This is an issue due to FD overshoot of rsyslog.
This is a known issue with vCenter 7.0. To fix this issue upgrade to vCenter 7.0 U3c and newer, to download latest version of vCSA 7.0 see VMware Download Center .
Please follow any one of the below workarounds :Workaround 1 :1. Take a backup of the original postgres-archiver syslog configuration:cp /etc/vmware-syslog/vmware-services-vmware-postgres-archiver.conf /etc/vmware-syslog/vmware-services-vmware-postgres-archiver.conf.orig2. Using vi or vim edit the file :/etc/vmware-syslog/vmware-services-vmware-postgres-archiver.conf3. Remove the existing content and replace it with the below:# vmware-postgres-archiver logsinput(type="imfile"File="/var/log/vmware/vpostgres/pg_archiver.log.stdout"Tag="postgres-archiver"Severity="info"Facility="local0")input(type="imfile"File="/var/log/vmware/vpostgres/pg_archiver.log.stderr"Tag="postgres-archiver"Severity="info"Facility="local0")4. Restart rsyslog:systemctl restart rsyslog5. confirm that rsyslog is running:systemctl status rsyslog6. The output of the above command should be similar to the below:rsyslog.service - System Logging ServiceLoaded: loaded (/lib/systemd/system/rsyslog.service; enabled; vendor preset: enabled)Active: active (running) since Fri 2021-03-05 00:33:37 UTC; 32s agoDocs: man:rsyslogd(8)http://www.rsyslog.com/doc/Main PID: 38846 (rsyslogd)Tasks: 13 (limit: 9830)Memory: 5.3MCGroup: /system.slice/rsyslog.service└─38846 /usr/sbin/rsyslogd -nWorkaround 2 : Stop rsyslog using command "systemctl stop rsyslog"Delete the core files in /storage/core with pattern "core.in:imfile*"Then use the below workaround to avoid any further core dump issues.Run the below command : find /var/log/vmware/ -mtime +1 -type f -name "localhost*access*log*" | while read file; do rm "$file" ; done Create a shell script with below content : vi cleanupAccessLog.sh #!/bin/bash find /var/log/vmware/ -mtime +1 -type f -name "localhost*access*log*" | while read file; do rm "$file" ; done Provide execute permission to the script : chmod 755 cleanupAccessLog.sh Navigate to cron job folder and create a cron file to execute the script on a regular basis : cd /etc/cron.d vi cleanAccessLog.cron 0 0 * * 0 root <path of the script>/cleanupAccessLog.sh 2>&1 Post this start the Syslog service : systemctl start rsyslog