Loading...
Loading...
For an ACP4OCP cluster running on 03.01.00.00 release, if the nodes in cluster use the bond mode 4 "802.3ad" for management traffic, day2 node addition will fail at step "Approve CSR".
During ACP4OCP node addition, NetworkManager in CoreOS configures the hostname of the new node via reverse DNS lookup. But if the nodes in cluster use the bond mode 4 (also known as "802.3ad" or LACP) for management traffic, the DNS lookup will fail because the network connection between the node and DNS server may not be ready when the system tries to set the hostname for the new node, leading the new node failed to be added into the cluster due to missing hostname.
To add the failed node into the cluster, perform the following steps: SSH to access the failed node and change the user to root(sudo -s) Run " hostnamectl hostname xxx " to set the hostname of the node to the expected value. Reboot the node. Retry the failed day2 node addition job from the DELL APEX Cloud Platform plugin UI. To prevent the next node addition failure, perform the following steps: 1. Log in to the cluster, use the following template to create a yaml file "machineconfig.yaml" apiVersion: machineconfiguration.openshift.io/v1 kind: MachineConfig metadata: labels: machineconfiguration.openshift.io/role: worker name: 99-worker-set-host-name spec: config: ignition: version: 3.4.0 storage: files: - contents: source: data:text/plain;charset=utf-8;base64,IyEvYmluL2Jhc2gKCkxPR19GSUxFPS92YXIvbG9nL2FjcC1zZXQtaG9zdG5hbWUubG9nCgpleGVjICY+PiA+KHRlZSAtaWEgIiRMT0dfRklMRSIpCgp3aGlsZSB0cnVlOyBkbwogICAgaG9zdG5hbWU9JChjYXQgL2V0Yy9ob3N0bmFtZSkKICAgIGlmIFsgLXogIiRob3N0bmFtZSIgXTsgdGhlbgogICAgICAgICBlY2hvICJob3N0bmFtZSBpcyBudWxsLCB0cnkgdG8gc2V0IHRoZSBob3N0bmFtZSIKICAgIGVsaWYgISBlY2hvICRob3N0bmFtZSB8IGdyZXAgLXEgImxvY2FsaG9zdCI7IHRoZW4KICAgICAgICBlY2hvICJob3N0bmFtZSBoYXMgYmVlbiBzZXQgdG8gJGhvc3RuYW1lIHN1Y2Nlc3NmdWxseSIKICAgICAgICBicmVhawogICAgZmkKCiAgICBuYW1lc2VydmVycz0kKGNhdCAvZXRjL3Jlc29sdi5jb25mIHwgZ3JlcCBebmFtZXNlcnZlcikKICAgIGlmIFsgLXogIiRuYW1lc2VydmVycyIgXTsgdGhlbgogICAgICAgIGVjaG8gIm5vIG5hbWVzZXJ2ZXIgaXMgY29uZmlndXJlZCwgcmV0cnkuLi4iCiAgICBlbHNlCiAgICAgICAgbmFtZXNlcnZlcnM9JChlY2hvICIkbmFtZXNlcnZlcnMiIHwgYXdrICd7cHJpbnQgJDJ9JykKICAgICAgICBob3N0X2FkZHJlc3M9IiIKICAgICAgICBleHRfbmFtZV9zZXJ2ZXI9IiIKICAgICAgICBmb3IgbmFtZXNlcnZlciBpbiAkbmFtZXNlcnZlcnM7IGRvCiAgICAgICAgICAgIG1hdGNoZWRfYWRkcmVzcz0kKGlwIGFkZHJlc3MgfCBncmVwICJpbmV0ICRuYW1lc2VydmVyIikKICAgICAgICAgICAgcmV0X2NvZGU9JD8KICAgICAgICAgICAgaWYgWyAkcmV0X2NvZGUgLW5lIDAgXSB8fCBbIC16ICIkbWF0Y2hlZF9hZGRyZXNzIiBdOyB0aGVuCiAgICAgICAgICAgICAgICBleHRfbmFtZV9zZXJ2ZXI9JG5hbWVzZXJ2ZXIKICAgICAgICAgICAgZWxzZQogICAgICAgICAgICAgICAgaG9zdF9hZGRyZXNzPSRuYW1lc2VydmVyCiAgICAgICAgICAgIGZpCiAgICAgICAgZG9uZQogICAgICAgIGlmIFsgIiRob3N0X2FkZHJlc3MiID0gIiIgXTsgdGhlbgogICAgICAgICAgICBlY2hvICJob3N0IGFkZHJlc3MgaXMgbm90IGZvdW5kIGluIHJlc29sdi5jb25mLCByZXRyeS4uLiIKICAgICAgICBlbHNlCiAgICAgICAgICAgIGVjaG8gImhvc3QgYWRkcmVzczogJGhvc3RfYWRkcmVzcywgbmFtZXNlcnZlcjogJGV4dF9uYW1lX3NlcnZlciIKICAgICAgICAgICAgcmVjb3Jkcz0kKG5zbG9va3VwICIkaG9zdF9hZGRyZXNzIiAiJGV4dF9uYW1lX3NlcnZlciIpCiAgICAgICAgICAgIGVjaG8gIm5zbG9va3VwIHJlc3VsdHM6ICRyZWNvcmRzIgogICAgICAgICAgICByZXRfY29kZT0kPwogICAgICAgICAgICBpZiBbICRyZXRfY29kZSAtZXEgMCBdOyB0aGVuCiAgICAgICAgICAgICAgICByZWNvcmQ9JChlY2hvICIkcmVjb3JkcyIgfCBoZWFkIC1uIDEgfCBhd2sgJ3twcmludCAkNH0nKQogICAgICAgICAgICAgICAgbGFzdF9jaGFyPSR7cmVjb3JkOiAtMX0KICAgICAgICAgICAgICAgIGlmIFsgIiRsYXN0X2NoYXIiID0gIi4iIF07IHRoZW4KICAgICAgICAgICAgICAgICAgICByZWNvcmQ9JHtyZWNvcmQlP30KICAgICAgICAgICAgICAgIGZpCiAgICAgICAgICAgICAgICBlY2hvICJzZXQgaG9zdG5hbWUgdG8gJHJlY29yZCIKICAgICAgICAgICAgICAgIGhvc3RuYW1lY3RsIGhvc3RuYW1lICIkcmVjb3JkIgogICAgICAgICAgICBlbHNlCiAgICAgICAgICAgICAgICBlY2hvICJuc2xvb2t1cCAkaG9zdGFkZHJlc3MgZnJvbSAkZXh0X25hbWVfc2VydmVyIGZhaWxlZCIKICAgICAgICAgICAgZmkKICAgICAgICBmaQogICAgZmkKICAgIHNsZWVwIDEKZG9uZQo= mode: 493 overwrite: true path: /usr/local/bin/acp-set-hostname.sh systemd: units: - contents: | [Unit] Description=Try to set node with a non-localhost hostname Before=kubelet-dependencies.target [Service] Type=oneshot RemainAfterExit=yes User=root ExecStart=/usr/local/bin/acp-set-hostname.sh # Wait up to 10min for the node to get a non-localhost name TimeoutSec=600 [Install] WantedBy=kubelet-dependencies.target enabled: true name: acp-set-hostname.service 2. Apply the yaml file to the cluster via command " oc apply -f machineconfig.yaml " 3. Wait the worker machineconfigpool "UPDATED" status become true, for example: mystic@mystic-vm:~$ oc get mcp worker NAME CONFIG UPDATED UPDATING DEGRADED MACHINECOUNT READYMACHINECOUNT UPDATEDMACHINECOUNT DEGRADEDMACHINECOUNT AGE worker rendered-worker-2ca5d42df01165197d1676f931d938c6 True False False 0 0 0 0 34h
Click on a version to see all relevant bugs
Dell Integration
Learn more about where this data comes from
BugZero Plan
Streamline upgrades with automated vendor bug scrubs
BugZero Prevent
Wish you caught this bug sooner? Get proactive today.