Loading...
Loading...
Document VersionRelease DateDetails309/12/2019Updated Scope section with any Mellanox/Broadcom/QLogic adapters supporting the RoCE feature are affected.209/18/2018Updated Resolution section with additional information.111/16/2017Original Document Release.Broadcom and QLogic Ethernet adapters with RoCE support do not load the RoCE module when the Mellanox Ethernet + RoCE or the MLNX_OFED driver is already installed. Broadcom and QLogic RoCE supported adapters depend on the native OS "ib_core" kernel module. Once the MLNX_OFED or the Mellanox Ethernet + RoCE driver is installed that updates the "ib_core" kernel module, it prevents other vendor's RoCE drivers to load, as shown in the following example:Native "ib_core" kernel module:[root@localhost ~]# modinfo ib_corefilename: /lib/modules/3.10.0-693.el7.x86_64/kernel/drivers/infiniband/core/ib_core.kolicense: Dual BSD/GPLdescription: core kernel InfiniBand APIauthor: Roland Dreierrhelversion: 7.4srcversion: 2B35581D5A8815FA8684260depends:intree: Yvermagic: 3.10.0-693.el7.x86_64 SMP mod_unload modversionssigner: Red Hat Enterprise Linux kernel signing keysig_key: 4F:FD:D6:3C:93:7E:B4:A7:A1:14:BC:5E:89:1A:CB:DE:50:20:65:21sig_hashalgo: sha256parm: send_queue_size:Size of send queue in number of work requests (int)parm: recv_queue_size:Size of receive queue in number of work requests (int)parm: force_mr:Force usage of MRs for RDMA READ/WRITE operations (bool)Mellanox Ethernet + RoCE - updated "ib_core" kernel module:[root@localhost Mellanox-RoCE]# modinfo ib_corefilename: /lib/modules/3.10.0-693.el7.x86_64/extra/mlnx-ofa_kernel/drivers/infiniband/core/ib_core.kolicense: Dual BSD/GPLdescription: core kernel InfiniBand APIauthor: Roland Dreierrhelversion: 7.4srcversion: 88498DC1AE00B29161E536Cdepends: mlx_compatvermagic: 3.10.0-693.el7.x86_64 SMP mod_unload modversionssigner: Hewlett-Packard Company: HP UEFI Secure Boot 2013 DB keysig_key: 1D:7C:F2:C2:B9:26:73:F6:9C:8E:E1:EC:70:63:96:7A:B9:B6:2B:ECsig_hashalgo: sha256parm: send_queue_size:Size of send queue in number of work requests (int)parm: recv_queue_size:Size of receive queue in number of work requests (int)parm: roce_v1_noncompat_gid:Default GID auto configuration (Default: yes) (bool)parm: force_mr:Force usage of MRs for RDMA READ/WRITE operations (bool)Qlogic driver does not load after "ib_core" update is installed as follows:[root@localhost Qlogic]# modinfo qedrfilename: /lib/modules/3.10.0-693.el7.x86_64/extra/qlgc-fastlinq/qedr.koversion: 8.33.0.0license: Dual BSD/GPLauthor: QLogic Corporationdescription: QLogic 40G/100G ROCE Driverrhelversion: 7.4srcversion: 91BE17AF97BBFDD84C3FCF2depends: qede,qed,ib_corevermagic: 3.10.0-693.el7.x86_64 SMP mod_unload modversionssigner: Hewlett-Packard Company: HP UEFI Secure Boot 2013 DB keysig_key: 1D:7C:F2:C2:B9:26:73:F6:9C:8E:E1:EC:70:63:96:7A:B9:B6:2B:ECsig_hashalgo: sha256parm: debug: Default debug msglevel (uint)parm: delayed_ack: iWARP: Delayed Ack: 0 - Disabled 1 - Enabled. Default: Disabled (uint)parm: timestamp: iWARP: Timestamp: 0 - Disabled 1 - Enabled. Default: Enabled (uint)parm: rcv_wnd_size: iWARP: Receive Window Size in K. Minimum is 64K. Default is set according to device configuration (uint)parm: crc_needed: iWARP: CRC needed 0 - Disabled 1 - Enabled. Default:Enabled (uint)parm: peer2peer: iWARP: Support peer2peer ULPs 0 - Disabled 1 - Enabled. Default:Enabled (uint)parm: mpa_enhanced: iWARP: MPA Enhanced mode. Default:1 (uint)parm: rtr_type: iWARP: RDMAP opcode to use for the RTR message: BITMAP 1: RDMA_SEND 2: RDMA_WRITE 4: RDMA_READ. Default: 7 (uint)parm: insert_udp_src_port: Insert a non-zero UDP source port for RoCEv2 packets that is unique per QP. 0 - Disabled, 1 - Enabled. Default:Enabled) (uint)parm: wq_multiplier: When creating a WQ the actual number of WQE created will be multiplied by this number (default is 3). (uint)[root@localhost Qlogic]# modprobe qedrmodprobe: ERROR: could not insert 'qedr': Invalid argument"/var/log/messages" outputOct 25 16:40:28 localhost kernel: qed_init calledOct 25 16:40:28 localhost kernel: QLogic FastLinQ 4xxxx Core Module qed 8.33.0.0Oct 25 16:40:28 localhost kernel: qedr: disagrees about version of symbol ib_umem_releaseOct 25 16:40:28 localhost kernel: qedr: Unknown symbol ib_umem_release (err -22)Oct 25 16:40:28 localhost kernel: qedr: disagrees about version of symbol ib_modify_qp_is_okOct 25 16:40:28 localhost kernel: qedr: Unknown symbol ib_modify_qp_is_ok (err -22)Broadcom driver does not load after ib_core updates from native OS version as follows:[root@localhost Broadcom]# modinfo bnxt_refilename: /lib/modules/3.10.0-693.el7.x86_64/extra/bnxt_en/drivers/infiniband/hw/bnxt_re/bnxt_re.koversion: 20.8.0.6license: Dual BSD/GPLdescription: Broadcom NetXtreme RoCE Driverauthor: Eddie Wai <eddie.wai@broadcom.com>rhelversion: 7.4srcversion: 4AA42989AAA09F07E785E15depends: ib_corevermagic: 3.10.0-693.el7.x86_64 SMP mod_unload modversionssigner: Hewlett-Packard Company: HP UEFI Secure Boot 2013 DB keysig_key: 1D:7C:F2:C2:B9:26:73:F6:9C:8E:E1:EC:70:63:96:7A:B9:B6:2B:ECsig_hashalgo: sha256parm: restrict_mrs: Restrict the no. of MRs 0 = 256K , 1 = 64K (uint)[root@localhost Broadcom]# modprobe bnxt_remodprobe: ERROR: could not insert 'bnxt_re': Invalid argument"/var/log/messages" outputOct 25 16:44:19 localhost kernel: bnxt_re: disagrees about version of symbol ib_umem_releaseOct 25 16:44:19 localhost kernel: bnxt_re: Unknown symbol ib_umem_release (err -22)Oct 25 16:44:19 localhost kernel: bnxt_re: disagrees about version of symbol ib_modify_qp_is_okOct 25 16:44:19 localhost kernel: bnxt_re: Unknown symbol ib_modify_qp_is_ok (err -22)Oct 25 16:44:19 localhost kernel: bnxt_re: disagrees about version of symbol ib_sg_to_pages
Any HPE ProLiant or HPE Synergy server running Linux and configured with both Mellanox IB/Ethernet and Qlogic OR Broadcom Ethernet RoCE supported adapters and installed with the following drivers:Mellanox-OFED or Mellanox Ethernet+RoCE (mlnx-ofa_kernel)ANDBroadcom NetXtreme-E RoCE driver -OR- QLogic FastLinQ RoCE driver.The following Operating Systems are affected:Red Hat Enterprise Linux (all supported versions).SUSE Linux Enterprise Server (all supported versions).The following adapters are affected:Mellanox network adapters:HPE Ethernet 25Gb 2-port 640SFP28 Adapter (HPE Part Number: 817753-B21)HPE Ethernet 25Gb 2-port 640FLR-SFP28 Adapter (HPE Part Number: 817749-B21)HPE InfiniBand EDR/Ethernet 100Gb 1-port 840QSFP28 Adapter (HPE Part Number: 825110-B21)HPE InfiniBand EDR/Ethernet 100Gb 2-port 840QSFP28 Adapter (HPE Part Number: 825111-B21)HP Ethernet 10Gb 2-port 546SFP+ Adapter (HP Part Number: 779793-B21)HP Ethernet 10Gb 2-port 546FLR-SFP+ Adapter (HP Part Number: 779799-B21)HP InfiniBand QDR/Ethernet 10Gb 2-port 544+M Adapter (HP Part Number: 764282-B21)HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+M Adapter (HP Part Number: 764283-B21)HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+QSFP Adapter (HP Part Number: 764284-B21)HP InfiniBand FDR/Ethernet 10Gb/40Gb 2-port 544+FLR-QSFP Adapter (HP Part Number: 764285-B21)HP InfiniBand QDR/Ethernet 10Gb 2-port 544+FLR-QSFP Adapter (HP Part Number: 764286-B21)HP InfiniBand QDR/EN 10Gb Dual Port 544M Adapter (HP Part Number: 644160-B21)HP InfiniBand FDR/EN 10/40Gb Dual Port 544M Adapter (HP Part Number: 644161-B21/644161-B22)HP InfiniBand FDR/Ethernet 10/40Gb 2-port 544QSFP Adapter (HP Part Number: 649281-B21)HP InfiniBand FDR/Ethernet 10/40Gb 2-port 544FLR-QSFP Adapter (HP Part Number: 649282-B21)HP InfiniBand QDR/Ethernet 10Gb 2-port 544FLR-QSFP Adapter (HP Part Number: 649283-B21)HP Infiniband QDR/Ethernet 10Gb 2P 544i Adapter (HP ProLiant SL4545 G7 Server and HP ProLiant SL4540 Gen8 Server LOM)HP Infiniband FDR 2-port 545QSFP Adapter (HP Part number: 702211-B21)HP Infiniband FDR 2P 545FLR-QSFP Adapter (HP Part number: 702212-B21)HP InfiniBand FDR 2-port 545M Adapter (HP Part number: 702213-B21)Broadcom Ethernet adapters:HPE Ethernet 10/25Gb 2-port 631SFP28 AdapterHPE Ethernet 10/25Gb 2-port 631FLR-SFP28 AdapterHPE Ethernet 10Gb 2-port 535T AdapterHPE Ethernet 10Gb 2-port 535FLR-T AdapterQLogic Ethernet adapters:HPE Ethernet 10Gb 2-port 521T AdapterHPE FlexFabric 10Gb 2-port 522FLR-T Converged Network AdapterHPE Ethernet 10/25Gb 2-port 621SFP28 AdapterHPE FlexFabric 10/25Gb 2-port 622FLR-SFP28 Converged Network AdapterHPE Synergy 6810C 25/50Gb Ethernet AdapterHPE Ethernet 4x25Gb 1-port 620QSFP28 AdapterNote: The list of affected adapters is current as of the Release Date of this Customer Advisory Revision 2 (2018-09-18). Any Mellanox/Broadcom/QLogic adapters supporting the RoCE feature are affected.
A QLogic OR Broadcom RoCE driver and a Mellanox OFED/Ethernet + RoCE driver cannot both be installed on the same HPE ProLiant or HPE Synergy server if both Mellanox and QLogic or Broadcom RoCE supported Ethernet adapters are to be used on the same node.As a workaround, for newer Linux Operating Systems for example, Red Hat Enterprise Linux 7.x or SUSE Linux Enterprise Server 12.x/15.x, use the inbox Mellanox drivers directly. However, for older Operating Systems (Red Hat Enterprise Linux 6.x or SUSE Linux Enterprise Server 11.x), the inbox Mellanox driver may not function with the latest ConnectX-4/ConnectX-5 adapters.RECEIVE PROACTIVE UPDATES: Receive support alerts (such as Customer Advisories), as well as updates on drivers, software, firmware, and customer replaceable components, proactively via e-mail through HPE Subscriber's Choice. Sign up for Subscriber's Choice at the following URL:Proactive Updates Subscription Form.NAVIGATION TIP: For hints on navigating HPE.com to locate the latest drivers, patches, and other support software downloads for ProLiant servers and Options, refer to theNavigation Tips document.SEARCH TIP: For hints on locating similar documents on HPE.com, refer to theSearch Tips document.To search for additional advisories related to Linux, use the following search string:+Advisory +ProLiant -"Software and Drivers" +Linux
Operating Systems Affected:Not Applicable
Click on a version to see all relevant bugs
Hewlett Packard Enterprise Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.