Symptom
- "snmpd[1101]: t11 Failed process trap" messages
- snmpd consuming high memory and CPU with a sustained high rate of traps
o see "show snmp" and "show snmp traps" to find the trap rate and per-trap type counts
Conditions
This was seen with "logging trap debug" & "snmp-server traps syslog" configured while high rate debugs are enabled. This is especially likely with the above configuration when enabling of "debug snmp trap" as it creates a feedback loop resulting in exponential growth in the rate of syslog traps.
Workaround
Change any 'logging trap debug' configuration to have syslogs of higher severity generated as traps, ie. 'logging trap informational' or 'logging trap warning'.
It is not recommended to use the configuration 'logging trap debug' as that causes EVERY syslog, including debugs, to be generated as SNMP traps via the CISCO-SYSLOG-MIB. This causes a trap storm.
Recovery:
- disable any debugs causing high rate of traps
- restart the snmpd process -- 'process restart snmpd'
- Change the SNMP trap send throttle "snmp-server trap throttle-time " configuration to the minimum of 10ms via:
"snmp-server trap throttle-time 10"
- disable any high rate traps via "no snmp-server traps "
- remove all "snmp-server host" configuration until the condition clears
Further Problem Description