Symptoms
Trap Exploder keeps crashing every other day.
Cause
This is the trap definition for the event that we found causing the crash..
fgFmTrapIfChange NOTIFICATION-TYPEOBJECTS
{ fnSysSerial, ifName, fgManIfIp, fgManIfMask, fgManIfIp6 }
STATUS currentDESCRIPTION "Trap is sent to the managing FortiManager if an interface IP is changed"::=
{ fgFmTrapPrefix 1004 }
As you can see above, the fifth varbind defines an IPV6 address, and that s what we received in the trap from the customer.
Trap was received at: 24-Jun-2016 03:29:37 AM EDT -> 80553|198.235.103.142|.1.3.6.1.4.1.12356.101.6|6|1004|.1.3.6.1.4.1.12356.100.1.1.1.0|FGT37D4615800058|.1.3.6.1.2.1.31.1.1.1.1.34|mgmt2|.1.3.6.1.4.1.12356.101.6.2.1.0|198.235.103.142|.1.3.6.1.4.1.12356.101.6.2.2.0|255.255.255.0|.1.3.6.1.4.1.12356.101.6.2.3.0|0:0:0:0:0:0:0:0|
So for all intents and purposes, the trap we received seems to have been coded exactly the way it was supposed to be, and yet it stopped the smarts process.
Resolution
We had the customer disable this particular trap, and the crashes ceased.
Notes: When the device SNMP agent sends this trap, it encodes the trap's 5th field as smiType of smiIPAddress (e.g., the type code is 0x40), because of that, smarts treats the value of this varbind as an IPv4 address, and eventually failed the PRECONDITION checking.