Symptoms

Upgrade to VNX File OE 8.1.9.236 Running any server_X related commands on the affected Data Mover fail with [RPC_CLIENT_CONN.c_create] errorAny Client access to that affected Data Mover fails, for example, SMB shares access, or NFS exports, or both may not be accessible, is failing, or the access is very slow.Outbound IP Connections for services are impacted, for example DNS, SMB, HTTP, Replication, so on From sys.log.txt: Dec 4 04:06:00 2020:DART:CEPP:ERROR:12:Slot 2:::1607051158:Error on CEPP server # of pool cepapool: HTTP_ERROR, httpStatus: 1006 Connection Disconnected. Dec 4 04:06:10 2020:DART:CEPP:ERROR:12:Slot 2:::1607051164:Error on CEPP server # of pool cepapool: HTTP_ERROR, httpStatus: 1006 Connection Disconnected. Dec 4 04:07:20 2020:DART:REP:WARNING:22:Slot 2:::1607051235:Source=242_CKM#_2007_252_CKM#_2007(alias=production_fs-to-DR), transferring. Data connection down. Source is retrying. From server_2.log: 2020-12-04 09:06:25: 26045448192: RCPT: 6: OpenStream failed 2020-12-04 09:07:45: 26045448192: RCPT: 6: OpenStream failed 2020-12-04 09:09:05: 26045448192: RCPT: 6: OpenStream failed 2020-12-04 09:30:21: 13157007360: SMB: 3:[VDM1] dns_DCLookup::process:: Unable to get any DC from DNS for domain example.com 2020-12-04 09:31:21: 13157007360: SMB: 3:[VDM1] dns_DCLookup::process:: Unable to get any DC from DNS for domain example.com 2020-12-04 09:33:21: 13157007360: SMB: 3: dns_DCLookup::process:: Unable to get any DC from DNS for domain example.com

Cause

Data Mover became unresponsive to any server_X-related commands after consuming all available 65535 TCP connections and failing to close inactive TCP connections. The following versions of VNX File OE are known to have this issue: 8.1.9.1848.1.9.232008.1.9.236

Resolution

WorkaroundEither a Data Mover reboot or Data Mover failover and failback temporarily resolves this issue. FixA fix is available from Dell Support in data mover patch 8.1.21.23604. Support installs this as a single file, and it is loaded by failing over and back the data mover or by rebooting the data mover. The issue is also addressed in VNX File OE Service Pack 6 for 8.1 (8.1.21.256). Contact Dell Technical Support or your Authorized Service Representative, quoting this Knowledge Base Article to upgrade the VNX OE.

Support Cases