...
Timeout error messages sporadically appearing in the VMware vROps User InterfaceAdapters status is “No Data Receiving” In the /usr/lib/vmware-vcops/user/log/gemfire_vRealize Ops Analytics_*.log, you see entries similar to: [severe 2017/06/18 00:19:11.158 UTC <locator request thread[2349]> tid=0x176f] SSL Error in connecting to peer /X.X.X.X[35,509]. javax.net.ssl.SSLException: Received close_notify during handshake at sun.security.ssl.Alerts.getSSLException(Alerts.java:208) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949) at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1901) at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:2002) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1125) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) at com.gemstone.gemfire.internal.SocketCreator.configureServerSSLSocket(SocketCreator.java:1077) at com.gemstone.org.jgroups.stack.tcpserver.TcpServer$3.run(TcpServer.java:303) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) In the /usr/lib/vmware-vcops/user/log/gemfire_vRealize Ops Collector_*.log, you see entries similar to: [warning 2017/06/18 00:10:35.370 UTC vRealize Ops Collector-05f930-fac-49-8bb7-1d97 <poolLoadConditioningMonitor-CollectorAnalyticsPool> tid=0x4d] Could not connect to: X.X.X.X:10000 java.net.SocketException: Connection reset at java.net.SocketInputStream.read(SocketInputStream.java:209) at java.net.SocketInputStream.read(SocketInputStream.java:141) at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) at sun.security.ssl.InputRecord.read(InputRecord.java:503) at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:973) at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1375) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1403) at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1387) at com.gemstone.gemfire.internal.SocketCreator.configureClientSSLSocket(SocketCreator.java:1141) at com.gemstone.gemfire.internal.SocketCreator.connect(SocketCreator.java:987) at com.gemstone.gemfire.internal.SocketCreator.connect(SocketCreator.java:945) at com.gemstone.gemfire.internal.SocketCreator.connectForClient(SocketCreator.java:914) at com.gemstone.gemfire.cache.client.internal.ConnectionImpl.connect(ConnectionImpl.java:97) at com.gemstone.gemfire.cache.client.internal.ConnectionFactoryImpl.createClientToServerConnection(ConnectionFactoryImpl.java:134) at com.gemstone.gemfire.cache.client.internal.pooling.ConnectionManagerImpl.createLifetimeReplacementConnection(ConnectionManagerImpl.java:991) at com.gemstone.gemfire.cache.client.internal.pooling.ConnectionManagerImpl$ConnectionMap.checkLifetimes(ConnectionManagerImpl.java:1524) at com.gemstone.gemfire.cache.client.internal.pooling.ConnectionManagerImpl$LifetimeExpireConnectionsTask.run(ConnectionManagerImpl.java:842) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Repeated exception error for com.gemstone.gemfire.cache.client.NoAvailableServersExceptionNetwork instability in terms of number of dropped packets in the analytics nodes communicationSYN_RECV sockets existence on the vROps nodes
This is occurs by 3rd party Gemfire/GEODE component not handling correctly its internal SSL Socket creating an operation flow.
This is a known issue affecting vRealize Operations Manager 6.6 and earlier versions.The issue is fixed in vRealize Operation Manager 6.6.1, To download, Download VMware vRealize Operations Manager 6.6.1 In vRealize Operations Manager 6.6.1 the resiliency to networking issues has been improved and few flows suffering of the same problem have been covered and resolved.
If you are unable to upgrade to vRealize Operations Manager 6.6.1, perform the workaround by restarting vROps cluster as below.From, vROps Admin UI > Take the vROPS cluster Offline and then bring Online. Note: Ensure to verify and resolve if there are any network issues.