Symptom
1) In a vPC setup, layer-3 packets with TTL set to1 and destined to the peer switch (switch-B) are delayed or dropped by Switch-A.
vPC config has 'l3 peer-router' enabled.
2) Elam has Sup hit flag set and DI as Sup index.
L4 Protocol : 89
Sup hit: 1, Sup Idx : 3511
Note: Sup Idx value is variable and can have different number.
3) CoPP has huge count under ttl exception class.
class-map copp-system-p-class-exception-diag (match-any)
match exception ttl-failure
match exception mtu-failure
set cos 1
police cir 150 kbps , bc 32000 bytes
module 1 :
transmitted 187005366349 bytes;
dropped 4725576043564 bytes;
Conditions
packet must have TTL = 1
N9k Cloud Scale ASIC based switches
Workaround
Increase policer rate for ICMP exception packets.
Further Problem Description
TTL = 1 hit ICMP exception and are punted to SUP. These are then software forwarded. The forwarding stops when there are too many ICMP exceptions going to SUP and policer starts dropping these packets.
Fix Remarks
- Release 9.3(4) ACL TCAM must be recarved for the fix to work. "ing-sup" region must be configured for more than the current size of 512. Below config is an example of TCAM changes that can be applied.
hardware access-list tcam region ing-racl 2048
hardware access-list tcam region ing-sup 768
- Release 9.3(5) and onwards fix is transparent and doesn't require TCAM carving for ing-sup region. Same applies to release 7.0(3)I7(9) and any later 7.0(3)I7 release.
