Symptom

Snort goes into D state and process is eventually killed due to OOM resulting in traffic outage

Conditions

Burst of traffic causing spikes in throughput and high connection rate that accumulates memory over time resulting in Memory growth reaching the limit without memcap pruning kicking in. Surge of IP fragmentation just prior to OOM pushes memory usage beyond limit and we are unable to prune sessions fast enough to stay below the limit.

Workaround

Reduce session memory cap to allow more headroom between pruning trigger and system limit Monitor memory usage and do a failover role change when it hits 80% cgroups limit

Further Problem Description