Symptom
Stack of C9300 switches may stop forwarding multicast traffic for particular group(s). There is an entry in IGMP membership table and IGMP snooping groups table; the mroute entry is also valid. However, FED entries are not programmed in hardware.
1) Debugging FED - handlers are not programmed:
show platform software fed switch active ip multicast group x.x.x.x/32 source x.x.x.x
Multicast (S,G) Information
VRF : 0
Source Address :
HTM Handler : 0x0 <<<< Value should not be 0x0 in working state
SI Handler : 0x0 <<<< Value should not be 0x0 in working state
DI Handler : 0x0 <<<< Value should not be 0x0 in working state
REP RI handler : 0x0 <<<< Value should not be 0x0 in working state
Flags :
Packet count : 0
State : 146
RPF :
A
OIF :
A
F NS
[2] Snooping group in FED: handlers are not programmed
show platform software fed switch active ip igmp snooping groups (NOTE: Command may cause an unexpected reload)
Vlan:x Group:x.x.x.x
---------------------------------
Member ports :
CAPWAP ports :
Host Type Flags: 0
Failure Flags : 0
DI handle : 0x0
REP RI handle : 0x0
SI handle : 0x0
HTM handle : 0x0
[3] FED traces: set platform software trace fed switch active asic_l3m debug
show platform software trace message fed switch active
[asic_l3m] AAL-INFRA:L3M Failed to allocate REP RI
Conditions
Hardware: C9300-48P
Software: Everest-16.6.2
[1] Cat9300 has a role of IGMP Querier; IGMP snooping is enabled.
[2] There is an entry in IGMP membership table and IGMP snooping groups table; the mroute entry is also valid.
[3] TCAM resources are not overutilized.
Workaround
Reload of the stack is the only way to recover multicast hardware capacity. This is a temporary workaround as the issue may come back after reload.
Further Problem Description