Symptoms
The VPLEX VS6 and VS2 are left configured on the same VLAN (post-GenU*, or for whatever other reason) Impacted VPLEX Hardware: EMC Hardware: VPLEX SeriesEMC Hardware: VPLEX VS2EMC Hardware: VPLEX VS6EMC Hardware: VPLEX-LocalEMC Hardware: VPLEX-MetroImpacted VPLEX GeoSynchrony code versions:EMC Software: GeoSynchrony 5.XEMC Software: GeoSynchrony 6.X*Note: GenU is a Generational Hardware Upgrade of the VS2 to a VS6.
High frequency of LLDP (Link Layer Discover Protocol) multicasts are seen on the XG00 orts of all directors on one cluster. The XG01 ports on the the same directors of this one cluster, and all directors XG ports at peer cluster, do not have the LLDP multicast storm issue.
Network captures (TCP dumps) from the VPLEX IP WAN-COM ports show very high frequencies of LLDP frames (almost at the order of 1000 frames every minute).
End Users IP network is affected due to an LLDP multicast storm.
Possible reboot of the VPLEX directors due to the way the NIC (Network Interface Card) on the VPLEX handles the LLDP multicast storm.
Cause
Link Layer Discovery Protocol (LLDP) is a layer-2 standard protocol which defines messages encapsulated in Ethernet frames to allow devices to announce basic device information to other (neighboring) devices in the LAN (Local Area Network). LLDP is a vendor-independent protocol (not proprietary to any vendor).
This is achieved through periodic transmissions by the Network Interface Card (NIC) every 30 seconds (by default).
The VPLEX VS6 cluster utilizes the Broadcom Emulex OneConnect OCe14000 Series NIC for it's IP WAN-COM module.
The VPLEX VS2 cluster utilizes the QLogic Heatwave EP8112 NIC for it's IP WAN-COM module.
Both the Broadcom and Qlogic NICs support LLDP as the chipsets support iSCSI, even though VPLEX does not support iSCSI.
TCP dumps captured during the incident reveal that the LLDP frames are being streamed at a very high frequency and not per the default value of 30 seconds. This is because whenever a NIC receives an LLDP frame with different settings, it is applying and processing the settings and re-transmitting the frame with the new settings. The LLDP multicast frames from all the NICs are forwarded to all other NICs within the same VLAN causing the LLDP multicast storm.
Resolution
Resolution:This issue was fixed in GeoSynchrony 6.2 P4 and later where the LLDP functionality on the VPLEX is disabled.If an end user wishes to disable LLDP on their switch ports the VPLEX is connected to they would need to engage the switch vendor for the steps to disable LLDP forwarding on the switch-ports to which the VPLEX WAN-COM ports are connected to.If you are running on a pre-6.2 P4 GeoSynchrony version it is recommended that you upgrade to at least GeoSynchrony 6.2 P7 that is the current target code, or to upgrade to the latest GeoSynchrony code release. You will need an active service contract to up0grade your VPLEX. If you do not have an active service contract reach out to your Dell Sales Representative to discuss with them.
