OPERATIONAL DEFECT DATABASE
...
...
N3500 reloads with "spm hap reset" or only SPM process crashes. A core file may or may not be generated.
Following error might be seen before the crash: %AFM-2-AFM_TCAM_UTIL_ABOVE_THRESHOLD: nat tcam utilization has gone above the threshold of 90% on ASIC id 0 which means NAT has to be configured and heavily utilized. Also, the following library in SPM is constantly increasing its Current allocated memory, for example: TYPE NAME ALLOCS BYTES CURR MAX CURR MAX ... 116 [r-xp]/isan/plugin/1/isan/lib/libspm.so 2913705 2916326 46619280 46771792 ... 116 [r-xp]/isan/plugin/1/isan/lib/libspm.so 7628055 7630322 122048880 122196488 this can be checked with "show system internal spm mem-stats detail".
None at the moment apart from removing NAT configuration or reloading the Switch to release memory. To know at what moment to reload the Switch monitor the following command: show system internal process memory | i spm which should generate similar output to this: `show system internal process memory | i spm` 3660 ? Ss 03:24:27 0 0 370684 586484 2.3 /isan/bin/spm 29971 pts/0 S+ 00:00:00 0 99 956 3012 0.0 egrep -- spm and, the second to last column is the value of Virtual Memory used by the process. In other words, it is the total memory used currently by SPM and should it reach value of >500000 - reload the Switch. Above output was captured from a Switch which was about to crash.
Simply put: this is a day one issue triggered by the amount of translations Switch is handling. If translations are being cleared in large quantities at once, SPM will not correctly release TVL allocations.
Click on a version to see all relevant bugs
Cisco Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
