Info
Original slack thread: https://mongodb.slack.com/archives/C07T35KELLC/p1738125749813369
Max:
color: Color value is invalid looks like the hang analyzer taking core dumps is filling up 100GB in
https://jira.mongodb.org/browse/BF-36363
color: Color value is invalid . From the system logs in
https://parsley.mongodb.com/evergreen/mongodb_mongo_master_enterprise_amazon_linux2_ar[…]_16_41_34/0/system?bookmarks=0,15107&filters=100mongod%2520
color: Color value is invalid , I see the total virtual memory used by all 24 mongod processes is 99GB. The resident memory is less than 5GB. I feel like one avenue would be to investigate what allocated memory within the mongod process because the WT cache is still restricted to 1GB. This is maybe better performed by the Server team so I'll tag
color: Color value is invalid @Abdul Qadeer
color: Color value is invalid as well
color: Color value is invalid For the Correctness team, I think
https://github.com/10gen/mongo/blob/6614f6533ef5932a95dd1b9b2eda8ff280a7a548/buildscripts/resmokelib/hang_analyzer/hang_analyzer.py#L174
color: Color value is invalid needs to be looked into for why it is ineffective at preventing the disk from filling up with core dumpsFor the future, I think the Correctness team can also consider using a named pipe so the contents of the
gcore
color: Color value is invalid command can be piped into gzip/pgzip since the compressed size of the core dumps will be much smaller
