Symptom
A Catalyst 9500X may fail to transmit packets with larger MTU after MACSEC is enabled on a a nearby port.
Prior to enabling MACSEC all ports are able to transmit at link MTU, however after MACSEC is enabled some port with jumbo MTU are unable to transmit at even half the size of interface MTU.
Example: hu1/0/5 has MACSEC while hu1/0/6 and hu1/0/7 do not have MACSEC. All ports inherit the system MTU of 9216.
After hu1/0/5 is configured with MACSEC, the other 2 ports fail to transmit packets and can only forward at a much smaller MTU.
OSPF flaps may be observed due to retry limit exceeded if an update packet is larger than the MTU that the interface can forward after MACSEC was enabled on neighboring port.
Conditions
9500X configured with MACSEC and nearby port without MACSEC.
Issue has been seen on 17.10 and 17.11
Workaround
Remove MACSEC. It is also seen that moving the non-MACSEC port further away from the MACSEC port works around the problem by having it on another PHY.