Issue
What were you trying to do that didn't work?
The performance copilot(pmcd) and haproxy-pmcd and pmproxy services are installed on virtual machine to fetch the cluster details,
It is observed that over a period of time the logs of pmcd / pmproxy are increasing to huge size in path /var/log/pcp/ which eventually is leading to /var getting to 100% full
If /var gets to 100% full, the risk is sudo to root on VM is not possible - which eventually means any critical administrative operations on server cannot be performed - this is an critical impact to SAP source system - since the SPOF (single point of faiure) SAP critical resources Message server and Database runs on these VM's
After reading https://bugzilla.redhat.com/show_bug.cgi?id=1954711 we upgraded to the following:
[root@amedbvm1 pmcd]# rpm -qa | grep pcp
pcp-pmda-hacluster-5.3.5-8.el8.x86_64
pcp-libs-5.3.5-8.el8.x86_64
pcp-conf-5.3.5-8.el8.x86_64
pcp-selinux-5.3.5-8.el8.x86_64
pcp-5.3.5-8.el8.x86_64
This seems to have resolved the issue with the pmproxy.log getting the "too many open files". but now we are seeing pmcd.log with a similar error.
Please provide the package NVR for which bug is seen:
How reproducible:
This is ongoing in multiple systems
Steps to reproduce
1. sudo yum install pcp pcp-pmda-hacluster
2. sudo systemctl start pmcd
3. sudo systemctl enable pmcd
HA cluster PMDA Install
4. cd /var/lib/pcp/pmdas/hacluster
5. sudo ./Install
***Start pmproxy
6. sudo systemctl start pmproxy
7. sudo systemctl enable pmproxy
Expected results
normal disk usage for the system
Actual results
either pmproxy.log or pmcd.log fills up the /var filesystem
