Launching soon: The world's first vendor agnostic bug scrubLearn more & join waitlist
OPERATIONAL DEFECT DATABASE
...
...
Action-based extensibility (ABX) actions and deployments start to fail.Running kubectl get nodes fails with the error error: You must be logged in to the server (Unauthorized) Note: kubectl restarts after this error Running kubectl get pods --all-namespaces fails with the error Unable to connect to the server: x509: certificate has expired or is not yet valid Running journalctl -u kubelet contains entries similar to Jan 26 11:46:40 applianceFQDN.vmware.com kubelet[5669]: F0126 11:46:40.942105 5669 server.go:266] failed to run Kubelet: unable to load bootstrap kubeconfig: stat /etc/kubernetes/bootstrap-kubelet.conf: no such file or directory Jan 26 11:46:40 applianceFQDN.vmware.com kubelet[5669]: E0126 11:46:40.941998 5669 bootstrap.go:264] Part of the existing bootstrap client certificate is expired: 2021-01-16 17:13:45 +0000 UTC
This issue is caused by the kubelet service certificate expiring after one year.
This issue will impact vRealize Automation & vRealize Orchestrator 8.0.1 and 8.1.
This issue is resolved in vRealize Automation & vRealize Orchestrator 8.2 and above.
To workaround this issue: Single VA Deployment Take a snapshot of the vRA VM.Locate an etcd backup at /data/etcd-backup/ and copy the selected backup to /rootReset Kubernetes by running vracli cluster leaveRestore the etcd backup in /root by using the /opt/scripts/recover_etcd.sh command. Example: /opt/scripts/recover_etcd.sh --confirm /root/backup-123456789.db Extract VA config from etcd with kubectl get vaconfig -o yaml | tee > /root/vaconfig.yaml Reset Kubernetes once again using vracli cluster leave Run to Install the VA config kubectl apply -f /root/vaconfig.yaml --force Run vracli license to confirm that VA config is installed properly.Run /opt/scripts/deploy.sh Clustered VAs Deployment with 3 Nodes Take a snapshots of all 3 nodes.Let's call one of the nodes a primary node. On the primary node, locate a etcd backup at /data/etcd-backup/ and preserved in /root.Reset each node with vracli cluster leave On the primary node, restore the etcd backup taken at /root using the /opt/scripts/recover_etcd.sh command Example: /opt/scripts/recover_etcd.sh --confirm /root/backup-123456789.db Extract VA config from etcd with kubectl get vaconfig -o yaml | tee > /root/vaconfig.yaml Reset the node once again with vracli cluster leave Install VA config with kubectl apply -f /root/vaconfig.yaml --force Run vracli license to confirm that VA config is installed properly. Note: vracli license is not applicable for vRO and CExP installations. Join the other 2 nodes in the cluster by running the following command on each vracli cluster join [primary-node] --preservedata Run /opt/scripts/deploy.sh from the primary node
https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-certs/https://github.com/kubernetes/kubeadm/issues/1753