Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Upgrade will fail with the following message on UIUnexpected error while upgrading upgrade unit: Install of offline bundle failed on host 5ab65127-234e-4634-98cc-7f28ab2cefab with error : [LiveInstallationError] Error in running ['/etc/init.d/nsx-datapath', 'start', 'upgrade']: Return code: 1 Output: ioctl failed: No such file or directory ioctl failed: No such file or directory start upgrade begin Exception: Traceback (most recent call last): File "/etc/init.d/nsx-datapath", line 1397, in <module> PostLoad() File "/etc/init.d/nsx-datapath", line 1023, in PostLoad shell = True) File "/build/mts/release/bora-13885523/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 676, in __init__ File "/build/mts/release/bora-13885523/bora/build/esx/release/vmvisor/sys-boot/lib64/python3.5/subprocess.py", line 1228, in _execute_child OSError: [Errno 28] No space left on device It is not safe to continue. Please reboot the host immediately to discard the unfinished update. Please refer to the log file for more details..- Host nsx-syslog will show following message, not able to connect to manager.<180>1 2019-12-14T19:10:37.898Z prom-05056a0ad9.nsbucqesystem.test NSX 2110786 - [nsx@6876 comp="nsx-esx" subcomp="nsx-proxy" s2comp="nsx-net" tid="2110789" level="WARN"] StreamConnection[935 Connecting to ssl://20.20.0.14:1234 sid:935] Couldn't connect to 'ssl://20.20.0.14:1234' (error: 335544539-short read)<180>1 2019-12-14T19:10:37.898Z prom-05056a0ad9.nsbucqesystem.test NSX 2110786 - [nsx@6876 comp="nsx-esx" subcomp="nsx-proxy" s2comp="nsx-net" tid="2110789" level="WARN"] StreamConnection[935 Error to ssl://20.20.0.14:1234 sid:-1] Error 335544539-short read<180>1 2019-12-14T19:10:37.898Z prom-05056a0ad9.nsbucqesystem.test NSX 2110786 - [nsx@6876 comp="nsx-esx" subcomp="nsx-proxy" s2comp="nsx-rpc" tid="2110789" level="WARN"] RpcConnection[935 Connecting to ssl://20.20.0.14:1234] Couldn't connect to ssl://20.20.0.14:1234 (error: 335544539-short read<180>1 2019-12-14T19:10:37.898Z prom-05056a0ad9.nsbucqesystem.test NSX 2110786 - [nsx@6876 comp="nsx-esx" subcomp="nsx-proxy" s2comp="nsx-rpc" tid="2110789" level="WARN"] RpcTransport[0] Unable to connect to ssl://20.20.0.14:1234: 335544539-short rea
The upgrade fails because of a certificate mismatch between NSX Manager and the host. Host certificate and host private keys are not updated properly
Host Upgrade from NSX-T 2.5.0 to NSX-T 2.5.1 may fail.
This is known issue and is planned to be fixed in future release.
Follow the steps below to recover the failed host 1. Manually migrate the VMs residing on the impacted host to a good host 2. Reboot the impacted host after all the VMs are moved out. 3. Exit host from Transport Node maintenance mode using TN API / UI. Use the API below to perform the operationNote: This is not the same as exiting host from ESX maintenance modePOST https://<nsx-mgr>/api/v1/transport-nodes/<<Transport Node UUID>>/?action=exit_maintenance_mode 4. Push the host certificate to the NSX Manager node. A. Obtain the NSX Manager thumbprint by logging into one of the MP nodes and execute the command below NSX-MANAGER:> get certificate api thumbprint B. Once the thumbprint of the NSX manager is obtained, perform the host certificate push operation from the host using the command below host-1# nsxcli -c push host-certificate <NSX Manager-IP/hostname> username admin thumbprint <thumbprint> password <NSX Manager password> Example: [host-1]# nsxcli -c push host-certificate 20.20.0.12 username admin thumbprint 53537ebe011198dea035837c7c75a3cd66b08743d12278bd2f2643862b74fc65 password ABCD!! Host certificate was pushed to management plane successfully 5. Retry the host upgrade after successfully pushing the host certificate to the NSX Manager