Loading...
Loading...
There are no hosts that are listed on Hosts page in VxRail and the physical view is in "loading status" on each node: /var/log/mystic/ web.log : 2021-08-28T00:26:51.574+0000 INFO [myScheduler-5] com.vce.commons.domainowner.node.NodeRepository NodeRepository.getAllNodes:946 - GraphQL query all nodes hardware info, host sn = [XXXX, XXXX, XXXX, … XXXX, XXXX, XXXX] ======> Node Service Tags 2021-08-28T00:26:55.471+0000 INFO [myScheduler-5] com.emc.mystic.manager.cluster.data.ApplianceDataQueryClient ApplianceDataQueryClient.getAllNodes:80 - return nodes size = 18 2021-08-28T00:26:55.471+0000 INFO [myScheduler-5] com.vce.commons.domainowner.cluster.VCRepository VCRepository.getClusterTypeString:264 - get cluster type from vc … …. … 2021-08-28T00:26:55.545+0000 INFO [myScheduler-5] com.vce.commons.core.services.NodeModelChecker NodeModelChecker.isC6420:59 - start to check whether VxRail P570F is a c6420 node 2021-08-28T00:26:55.545+0000 ERROR [myScheduler-5] com.emc.mystic.manager.cluster.data.ApplianceDataRoot ApplianceDataRoot.fetchData:127 - failed to fetch data. java.lang.NullPointerException: null at com.emc.mystic.manager.cluster.data.ApplianceDataRoot.buildPowerSupply(ApplianceDataRoot.java:478) ~[classes/:?] at com.emc.mystic.manager.cluster.data.ApplianceDataRoot.buildAppliances(ApplianceDataRoot.java:269) ~[classes/:?] at com.emc.mystic.manager.cluster.data.ApplianceDataRoot.fetchData(ApplianceDataRoot.java:124) [classes/:?] at jdk.internal.reflect.GeneratedMethodAccessor477.invoke(Unknown Source) ~[?:?] at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?] at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?] at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84) [spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE] at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) [spring-context-5.2.4.RELEASE.jar:5.2.4.RELEASE] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305) [?:?] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) [?:?] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?] at java.lang.Thread.run(Thread.java:834) [?:?] /var/log/microservice_log/ short.term.log , do-host can query nodes from node.node table succesfuly: 2021-08-30-18:21:54 microservice.configservice "INFO:configservice.lib.etcd_client:/configservice/cluster_moid exists" 2021-08-30-18:21:54 microservice.configservice "INFO:configservice.lib.etcd_client:read from etcd: /configservice/cluster_moid = domain-c9" 2021-08-30-18:21:54 microservice.do-host "[infra_db_util.db_util.database_connector] 2021-08-30 18:21:54,172 DEBUG 24923:DummyThread-904 [database_connector.py:148] query_row(): execute query [" 2021-08-30-18:21:54 microservice.do-host "SELECT sn FROM node.node WHERE uuid=%(uuid)s" 2021-08-30-18:21:54 microservice.do-host "] with params=[\{'uuid': 'XXXXXXXXXXXXXXXXXXXXXXXXX'}]" 2021-08-30-18:21:54 microservice.do-host "[infra_db_util.db_util.database_connector] 2021-08-30 18:21:54,173 DEBUG 24923:DummyThread-904 [database_connector.py:148] query_row(): execute query [" The node table has all nodes' information: psql -U postgres vxrail -c "select * from node.node";
This is caused by the iDRAC on the server. There are missing components when querying node information.
To resolve this issue, run the below command on VxRail Manager as root user: curl --unix-socket /var/lib/vxrail/nginx/socket/nginx.sock -H "Content-Type: application/json" http://127.0.0.1/rest/vxm/internal/do/v1/host/query -d '{"query":"{configuredHosts{name,hardware{sn,chassis{psus{slot,name,sn,baseline{sn,slot,isMissing}}}}}}"}' |jq Normal node's output is "isMissing": false, as below: { "name": "Host Name", "hardware": { "sn": "XXX", "chassis": { "psus": [ { "slot": 1, "name": "Power Supply 1", "sn": "XXXXXXXXXXX", "baseline": { "sn": "XXXXXXXXXXXXX", "slot": "1", "isMissing": false } }, { "slot": 2, "name": "Power Supply 2", "sn": "XXXXXXXXXX", "baseline": { "sn": "XXXXXXXXX", "slot": "2", "isMissing": false } Issue Node: "name": "Host Name", "hardware": { "sn": "XXXXXX", "chassis": { "psus": [ { "slot": 1, "name": "Power Supply 1", "sn": "XXXXX", "baseline": { "sn": "XXXXXX", "slot": "1", "isMissing": true } }, { "slot": 1, "name": "Power Supply 1", "sn": "XXXXXXXXXXXXX", "baseline": { "sn": null, "slot": null, "isMissing": true } Find all nodes with an "isMissing": true status. Reset the iDRAC of all nodes with the "isMissing": true status. Resetting the iDRAC causes the iDRAC restart and the iDRAC UI is inaccessible during the restart. SSH to the host and run: 8.0.300 and above: /opt/vxrail/tools/ipmitool mc reset cold 7.x - 8.0.240: /opt/vxrail/tools/ipmitool mc reset cold Wait 5 minutes for the iDRAC to restart. Rerun the command, ensure all nodes are fine. curl --unix-socket /var/lib/vxrail/nginx/socket/nginx.sock -H "Content-Type: application/json" http://127.0.0.1/rest/vxm/internal/do/v1/host/query -d '{"query":"{configuredHosts{name,hardware{sn,chassis{psus{slot,name,sn,baseline{sn,slot,isMissing}}}}}}"}' |jq Wait 15 minutes after resetting the iDRAC and then refresh vCenter UI. (VxRail Manager Plugin needs enough time to load data)
Click on a version to see all relevant bugs
Dell Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.