Symptoms

When running "showmount -e" against NFS server, there is error, e.g: isiloncluster-1# showmount -e localhostRPC: Unable to receiveshowmount: can't do exports rpcisiloncluster-1# Check the node /var/log/nfs.log file, there are errors: ...2024-09-04T15:11:20.982330-06:00 <30.4> isioncluster-1(id4) nfs[5201]: [nfs] rendezvous_request(453) Connection limit hit count: 404, max: 4042024-09-04T15:11:21.030873-06:00 <30.4> isioncluster-1(id4) nfs[5201]: [nfs] rendezvous_request(453) Connection limit hit count: 404, max: 4042024-09-04T15:11:22.004448-06:00 <30.4> isioncluster-1(id4) nfs[5201]: [nfs] rendezvous_request(453) Connection limit hit count: 404, max: 404... The clients exhibited a problem with connectivity or network issue even right after restarting the NFS service no change occurred.These messages started to log right after the upgrade was completed.

Cause

During nfs server initializing it logs the Physical Memory and VMEM Limit in the /var/log/nfs.log, e,g: ...2024-09-04T15:00:33.123909-06:00 <30.5> isioncluster-1(id4) nfs[4410]: [nfs] NfsInitialize(612) Initializing...2024-09-04T15:00:33.142086-06:00 <30.5> isioncluster-1(id4) nfs[4410]: [nfs] Physical Memory: 169675530242024-09-04T15:00:33.142617-06:00 <30.5> isioncluster-1(id4) nfs[4410]: [nfs] VMEM Limit: 42418872322024-09-04T15:00:33.800550-06:00 <30.4> isioncluster-1(id4) nfs[4410]: [nfs] NfsTuneNodeConfig(2783) RPC Max Connections: 4042024-09-04T15:00:33.800612-06:00 <30.4> isioncluster-1(id4) nfs[4410]: [nfs] NfsTuneNodeConfig(2811) SchedulerLimitNfsKernelSizeMB: 63, SchedulerLimitNfsUserSizeMB 162024-09-04T15:00:33.966847-06:00 <30.5> isioncluster-1(id4) nfs[4410]: [nfs] NfsInitialize(798) Limits bytesin: 4294967296, bytesout: 4294967296, workitems: 0,kernelbytes: 848377651, userbytes: 848377446, connections: 0, openfiles: 02024-09-04T15:00:34.953730-06:00 <30.4> isioncluster-1(id4) nfs[4410]: [nfs] Config backend for nfs_scheduler_config is now synchronized2024-09-04T15:00:35.328750-06:00 <30.4> isioncluster-1 syslogd: last message repeated 1 times2024-09-04T15:00:43.140745-06:00 <30.5> isioncluster-1(id4) nfs[4410]: [nfs] NfsInitialize(1493) Initialization complete: status: 0x0 (STATUS_SUCCESS)2024-09-04T15:00:47.633530-06:00 <30.4> isioncluster-1(id4) nfs[4410]: [nfs] Config backend for nfs_config_zone_1 is now synchronized2024-09-04T15:00:47.853666-06:00 <30.4> isioncluster-1(id4) nfs[4410]: [nfs] Config backend for nfs_config is now synchronized2024-09-04T15:00:47.905511-06:00 <30.5> isioncluster-1(id4) nfs[4410]: [nfs] Server now ready to process requests... The Node Physical Memory: 16967553024, VMEM Limit: 4241887232The RPC Max Connections is 404, and this is relevant to the default registry.Services.lwio.Parameters.Drivers.nfs.RpcPercentVMemMBConnections is 10. 

Resolution

Remedy/Workaround: # isi_gconfig registry.Services.lwio.Parameters.Drivers.nfs.RpcPercentVMemMBConnections=0 After changing the value OneFS_NFS must be restarted: # isi_for_array -sX '/usr/likewise/bin/lwsm restart onefs_nfs; sleep 30' Or disable/enable NFS: # isi services nfs disable# isi services nfs enable NOTE: Existing nfs connection will be interrupted for a short time. Resolution: Increase the A200 node physical memory to 64G.

Support Cases