Issue

What were you trying to do that didn't work? using "leapp" utility Description of problem: RHEL Enterprise Upgrade 8.9 to 9.3 fails - /boot device with UUID not found There's a KB about: Leapp can fail when /boot is a multipath: https://access.redhat.com/solutions/6962799 After replacing boot-UUID /boot in fstab with /dev/md126 - Upgrade did fail again. Version-Release number of selected component (if applicable): leapp version 0.16.0 How reproducible: Steps to Reproduce: 1. on RHEL Enterprise 8.9 - run "leapp upgrade" leapp will report: reboot now 2. reboot 3. upgrade will fail - landing in emergency cli Actual results: no workaround seems to work Expected results: RHEL upgrade procedure (pyhton scripts) should properly handle situations, where system running Multipath configurations (md), because this is common used case for many systems. Additional info: 1rst boot upon reboot request of leapp: [ 9.829615] localhost upgrade[1214]: mount: /boot: can't find UUID=xxxx.xxxx.xxxx. [ 9.838088] localhost kernel: XFS (dm-8): Mounting V5 Filesystem [ 9.994909] localhost kernel: XFS (dm-8): Ending clean mount [ 10.004975] localhost kernel: xfs filesystem being mounted at /home supports timestamps until 2038 (0x7fffffff) [ 32.928596] localhost upgrade[1223]: ==> Processing phase `InitRamStart` [ 32.928596] localhost upgrade[1223]: ====> * remove_upgrade_boot_entry [ 32.928596] localhost upgrade[1223]: Remove boot entry for Leapp provided initramfs. [ 33.185882] localhost upgrade[1622]: Process Process-176: [ 33.185882] localhost upgrade[1622]: Traceback (most recent call last): [ 33.185882] localhost upgrade[1622]: File "/usr/lib64/python3.6/multiprocessing/process.py", line 258, in _bootstrap [ 33.185882] localhost upgrade[1622]: self.run() [ 33.185882] localhost upgrade[1622]: File "/usr/lib64/python3.6/multiprocessing/process.py", line 93, in run [ 33.185882] localhost upgrade[1622]: self._target(*self._args, **self._kwargs) [ 33.185882] localhost upgrade[1622]: File "/usr/lib/python3.6/site-packages/leapp/repository/actor_definition.py", line 74, in _do_run [ 33.185882] localhost upgrade[1622]: actor_instance.run(*args, **kwargs) [ 33.185882] localhost upgrade[1622]: File "/usr/lib/python3.6/site-packages/leapp/actors/_init_.py", line 289, in run [ 33.185882] localhost upgrade[1622]: self.process(*args) [ 33.185882] localhost upgrade[1622]: File "/etc/leapp/repos.d/system_upgrade/common/actors/removeupgradebootentry/actor.py", line 20, in process [ 33.185882] localhost upgrade[1622]: remove_boot_entry() [ 33.185882] localhost upgrade[1622]: File "/etc/leapp/repos.d/system_upgrade/common/actors/removeupgradebootentry/libraries/removeupgradebootentry.py", line 30, in remove_boot_entry [ 33.185882] localhost upgrade[1622]: '--remove-kernel={0}'.format(kernel_filepath) [ 33.185882] localhost upgrade[1622]: File "/usr/lib/python3.6/site-packages/leapp/libraries/stdlib/_init_.py", line 192, in run [ 33.185882] localhost upgrade[1622]: result=result [ 33.185882] localhost upgrade[1622]: leapp.libraries.stdlib.CalledProcessError: Command ['/usr/sbin/grubby', '--remove-kernel=/boot/vmlinuz-upgrade.x86_64'] failed with exit code 1. 2nd boot - replacing uuid of fstab /boot with md-device (/dev/md126): [ 9.912370] localhost systemd[1]: Finished dracut pre-pivot and cleanup hook. [ 9.960504] localhost kernel: xfs filesystem being remounted at /sysroot supports timestamps until 2038 (0x7fffffff) [ 9.956914] localhost upgrade[1192]: Found file /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed [ 9.957712] localhost upgrade[1192]: Error: Leapp previously failed and cannot continue, returning back to emergency shell [ 9.957712] localhost upgrade[1192]: Please file a support case with /sysroot/var/log/leapp/leapp-upgrade.log attached [ 9.957712] localhost upgrade[1192]: To rerun the upgrade upon exiting the dracut shell remove the /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed file [ 9.959836] localhost upgrade[1099]: writing logs to disk and rebooting [ 10.345738] localhost upgrade[1198]: mount: /boot: special device /dev/md126 does not exist. [ 10.350740] localhost kernel: /dev/md126: Can't open blockdev [ 10.352841] localhost kernel: XFS (dm-8): Mounting V5 Filesystem [ 10.527509] localhost kernel: XFS (dm-8): Ending clean mount [ 10.530779] localhost kernel: xfs filesystem being mounted at /home supports timestamps until 2038 (0x7fffffff) [ 10.536318] localhost kernel: XFS (dm-8): Unmounting Filesystem [ 10.547479] localhost upgrade[1099]: /bin/upgrade: line 19: /sysroot/var/tmp/system-upgrade.state: Read-only file system [ 10.548229] localhost systemd[1]: upgrade.service: Main process exited, code=exited, status=1/FAILURE output from leapp-upgrade.log: Dec 30 12:07:34 localhost systemd[1]: Reached target Local Encrypted Volumes. Dec 30 12:07:34 localhost systemd[1]: Reached target Path Units. Dec 30 12:07:34 localhost systemd[1]: Starting Wait for udev To Complete Device Initialization... Dec 30 12:07:34 localhost systemd[1]: sys-module-configfs.device: Failed to enqueue SYSTEMD_WANTS= job, ignoring: Unit modprobe@configfs.service not found. Dec 30 12:07:34 localhost udevadm[1536]: systemd-udev-settle.service is deprecated. Please fix multipathd-configure.service not to pull it in. Dec 30 12:07:34 localhost systemd[1]: Finished Wait for udev To Complete Device Initialization. Dec 30 12:07:34 localhost systemd[1]: Device-Mapper Multipath Default Configuration was skipped because of an unmet condition check (ConditionPathExists=!/etc/multipath.conf). Dec 30 12:07:34 localhost systemd[1]: Starting Device-Mapper Multipath Device Controller... Dec 30 12:07:34 localhost systemd[1]: Finished dracut initqueue hook. Dec 30 12:07:34 localhost systemd[1]: Reached target Preparation for Remote File Systems. Dec 30 12:07:34 localhost systemd[1]: Reached target Remote Encrypted Volumes. Dec 30 12:07:34 localhost systemd[1]: Reached target Remote File Systems. Dec 30 12:07:34 localhost systemd[1]: Starting dracut pre-mount hook... Dec 30 12:07:34 localhost multipathd[1578]: -------start up------- Dec 30 12:07:34 localhost multipathd[1578]: read /etc/multipath.conf Dec 30 12:07:34 localhost multipathd[1578]: path checkers start up Dec 30 12:07:34 localhost systemd[1]: sys-devices-virtual-block-md127.device: Failed to enqueue SYSTEMD_WANTS= job, ignoring: Unit mdmonitor.service not found. Dec 30 12:07:34 localhost systemd[1]: Finished dracut pre-mount hook. Dec 30 12:07:34 localhost systemd[1]: Started Device-Mapper Multipath Device Controller. Dec 30 12:07:34 localhost systemd[1]: Reached target Initrd Root File System. Dec 30 12:07:34 localhost systemd[1]: Reached target Preparation for Local File Systems. Dec 30 12:07:34 localhost systemd[1]: Reached target Local File Systems. Dec 30 12:07:34 localhost systemd[1]: Reached target System Initialization. Dec 30 12:07:34 localhost systemd[1]: Reached target Basic System. Dec 30 12:07:34 localhost systemd[1]: Reached target System Upgrade. Dec 30 12:07:34 localhost systemd[1]: dracut mount hook was skipped because no trigger condition checks were met. Dec 30 12:07:34 localhost systemd[1]: Starting dracut pre-pivot and cleanup hook... Dec 30 12:07:34 localhost systemd[1]: Started Hardware RNG Entropy Gatherer Daemon. Dec 30 12:07:34 localhost systemd[1]: Starting System Upgrade... Dec 30 12:07:34 localhost upgrade[1619]: starting upgrade hook Dec 30 12:07:34 localhost rngd[1617]: Disabling 7: PKCS11 Entropy generator (pkcs11) Dec 30 12:07:34 localhost rngd[1617]: Disabling 5: NIST Network Entropy Beacon (nist) Dec 30 12:07:34 localhost rngd[1617]: Disabling 9: Qrypt quantum entropy beacon (qrypt) Dec 30 12:07:34 localhost rngd[1617]: Initializing available sources Dec 30 12:07:34 localhost rngd[1617]: [hwrng ]: Initialization Failed Dec 30 12:07:34 localhost rngd[1617]: [rdrand]: Initialization Failed Dec 30 12:07:34 localhost rngd[1617]: [jitter]: JITTER timeout set to 5 sec Dec 30 12:07:34 localhost upgrade[1629]: 7 logical volume(s) in volume group "cl_vg1" now active Dec 30 12:07:34 localhost kernel: xfs filesystem being remounted at /sysroot supports timestamps until 2038 (0x7fffffff) Dec 30 12:07:34 localhost upgrade[1658]: Found file /sysroot/root/tmp_leapp_py3/.leapp_upgrade_failed Dec 30 12:07:34 localhost upgrade[1658]: Error: Leapp previously failed and cannot continue, returning back to emergency shell Multipath.conf: --------------- after the upgrade failed the 1rst time, i did run this command: "mpathconf --enable" cat /etc/multipath.conf device-mapper-multipath configuration file For a complete list of the default configuration values, run either: # multipath -t or # multipathd show config For a list of configuration options with descriptions, see the multipath.conf man page. defaults { user_friendly_names yes find_multipaths yes enable_foreign "^$" } blacklist_exceptions { property "(SCSI_IDENT_|ID_WWN)" } blacklist { }  

Release Notes

No. of Comments

Resolution

Won't Do