Symptom
- Upgrade has failed. The user tries to roll back to the previous version but this will fail too. See below error:
# upgrade_rollback.sh
Restoring the system to pre-upgrade state...
removed '/tmp/upgrade.lock/UPGRADE'
removed '/tmp/upgrade.lock/LSM'
removed '/tmp/upgrade.lock/UUID'
removed '/tmp/upgrade.lock/main_upgrade_script.log'
removed '/tmp/upgrade.lock/PID'
removed '/tmp/upgrade.lock/AQ_UUID'
removed '/tmp/upgrade.lock/status_log'
removed directory '/tmp/upgrade.lock'
logdir is /var/log/sf/Cisco_Firepower_Mgmt_Center_Upgrade-7.0.1
[211026 21:40:07] Pre-upgrade version is 6.7.0.2
[211026 21:40:07] 1635284407 Oct 26 21:40 UTC Restoring database...
[211026 21:40:07] Retrieved aq_uuid: 462d505e-369b-11ec-bcbb-b96c3ede8271 message: Update Installation Failed : [82%] Fatal error: Error running script 800_post/1035_670_update_adi_conf.pl. and state: 4 from new-mysql.
[211026 21:40:07] Stopped ActionQueueScrape.
[211026 21:40:07] Stopping mysql
waiting for mysql to stop (500 ms)...
waiting for mysql to stop (1000 ms)...
...
waiting for mysql to stop (59500 ms)...
waiting for mysql to stop (60000 ms)...
[211026 21:41:08] Failed to stop mysql (max waiting time reached).
[211026 21:41:08] Rollback failed. Could not stop mysqld.
Conditions
This issue will be seen when one opens a new ssh session if any upgrade failure after 1027_ldap_external_auth_fix.pl.
Workaround
If any existing ssh sessions are open, rollback will work as expected.
This issue will not be seen if user logins to the device via console and upgrade rollback will work fine from the console session.
Further Problem Description