Symptom

Crash after "PIM register asynchronous tunnel creation" process leaking memory in processor pool. Following messages are seen: Nov 4 16:50:16: %PLATFORM_INFRA-5-IOS_INTR_OVER_LIMIT: IOS thread disabled interrupt for 11 msec -Traceback= 1#05d1d6e039e2f341c0281d675af6d748 :10000000+7532A0 :10000000+753650 :10000000+3A26034 :10000000+5C4FDAC :10000000+5C50594 :10000000+238DB3C :10000000+5CA7F88 :10000000+5C6AFCC :10000000+5C6B080 :10000000+2395650 :10000000+FD53F8 :10000000+EE11B8 :10000000+EE38D8 :10000000+10977D8 :10000000+1081648 :10000000+10AA56C Nov 4 16:50:29: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel6, changed state to up Nov 4 16:50:29: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel7, changed state to up Nov 4 16:50:29: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel8, changed state to up These tunnel flaps are continuous and even after the crash, from the latest show tech: Nov 4 19:20:16: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel25, changed state to up Nov 4 19:22:20: %PARSER-5-CFGLOG_LOGGEDCMD: User:v786021 logged command:!exec: enable Nov 4 19:23:34: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel10, changed state to up Nov 4 19:23:34: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel11, changed state to up Nov 4 19:23:34: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel12, changed state to up Nov 4 16:57:56: %SYS-2-MALLOCFAIL: Memory allocation of 8432 bytes failed from 0x124285C8, alignment 4 Pool: Processor Free: 2895424 Cause: Memory fragmentation Alternate Pool: None Free: 0 Cause: No Alternate pool -Process= "PIM register asynchronous tunnel creation", ipl= 0, pid= 176 -Traceback= 1#05d1d6e039e2f341c0281d675af6d748 :10000000+7532A0 :10000000+753650 :10000000+2414BD8 :10000000+241BCB0 :10000000+24285CC :10000000+2BC24D0 :10000000+2BC34BC :10000000+2BC23C8 :10000000+2397B08 :10000000+239A914 :10000000+FD54E4 :10000000+EE3BC4 :10000000+EE3FB8 Nov 4 16:57:56: %TUN-3-DECAP_TINFO: failed to startup -Traceback= 1#05d1d6e039e2f341c0281d675af6d748 :10000000+7532A0 :10000000+753650 :10000000+2BC24FC :10000000+2BC34BC :10000000+2BC23C8 :10000000+2397B08 :10000000+239A914 :10000000+FD54E4 :10000000+EE3BC4 :10000000+EE3FB8 Nov 4 16:57:56: %SYS-2-NOMEMORY: No memory available for FMANRP msg malloc 396 %Software-forced reload

Conditions

When its leaking memory, following increment can be observed: Earlier: router#Show process memory 176 Process ID: 176 Process Name: PIM register asynchronous tunnel creation Total Memory Held: 7777068 bytes Processor memory Holding = 7777068 bytes pc = 0x15C42DC8, size = 2418080, count = 680 pc = 0x16E7D3AC, size = 1194928, count = 1 pc = 0x15C42D94, size = 971040, count = 680 Later: router#show process memory 176 Process ID: 176 Process Name: PIM register asynchronous tunnel creation Total Memory Held: 8180468 bytes Processor memory Holding = 8180468 bytes pc = 0x15C42DC8, size = 2602992, count = 732 pc = 0x16E7D3AC, size = 1194928, count = 1 pc = 0x15C42D94, size = 1045296, count = 732

Workaround

None

Further Problem Description