...
ISA3000 stuck in boot loop after reboot
ISA-3000-4C-X running FTD 7.0.1 and patch applied to fix the shutdown feature After patch has been applied, and device has been restarted as part of the patch install no issues are seen. Subsequent reboots of the device after the patch is on the device cause boot loop condition
Investigating if cold-reboot will resolve the boot loop issue
The following output is seen after the device restarts, and the boot loop condition is present Cisco Firepower Extensible Operating System (FX-OS) v2.10.1 (build 175) Cisco ISA-3000-4C-X Threat Defense v7.0.1 (build 84) > shutdown This command will shutdown the system. Continue? Please enter 'YES' or 'NO': YES INIT: INIT: Sending pStopping Cisco ISA-3000-4C-X Threat Defense......ok Shutting down sfifd... [ OK ] Clearing static routes Unconfiguring default route [ OK ] Unconfiguring address on br1 [ OK ] Unconfiguring IPv6 [ OK ] Downing interface [ OK ] Stopping nscd... [ OK ] Turning off swapfile /ngfw/Volume/.swaptwo Stopping system log daemon... [ OK ] Stopping Threat Defense ... Stopping system message bus: dbus. [ OK ] Un-mounting disk partitions ... Device var is still in use. Stopping OpenBSD Secure Shell server: sshd stopped /usr/sbin/sshd (pid 4521) done. Stopping Advanced Configuration and Power Interface daemon: stopped /usr/sbin/acpid (pid 4525) acpid. Stopping system message bus: dbus. Stopping DHCP server: dhcpd3no /usr/sbin/dhcpd found; none killed . Stopping LXC containers: Stopping internet superserver: xinetd. Stopping random number generator daemon. Deconfiguring network interfaces... ifdown: interface br1 not configured done. mkdir: cannot create directory '/var/log': File exists SSP-Security-Module is shutting down ... Sending ALL processes the TERM signal ... acpid: exiting Sending ALL processes the KILL signal ... Deactivating swap... Unmounting local filesystems... mount: /: mount point is busy. grep: error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory Firepower Threat Defense stopped. It is safe to power off now. To restart the device, you must Power cycle the device. dfgdg Rom image verified correctly Cisco Systems ROMMON, Version 1.0.5, RELEASE SOFTWARE Copyright (c) 1994-2019 by Cisco Systems, Inc. Compiled Tue 05/07/2019 11:53:04.90 by builders Current image running: Boot ROM1 Last reset cause: PowerOn DIMM Slot 0 : Present Platform ISA-3000-4C with 8192 Mbytes of main memory MAC Address: 68:9e:0b:9b:90:01 Use BREAK or ESC to interrupt boot. Use SPACE to begin boot immediately. Boot in 10 seconds. Boot in 9 seconds. Boot in 8 seconds. Boot in 7 seconds. Boot in 6 seconds. Boot in 5 seconds. Boot in 4 seconds. Boot in 3 seconds. Boot in 2 seconds. Boot in 1 second. Located '.boot_string' @ cluster 194127. # Attempt autoboot: "boot disk0:/os.img" Located 'os.img' @ cluster 172693. ############################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################################### LFBFF signature verified. INIT: version 2.88 booting Starting udev Configuring network interfaces... done. Starting random number generator daemon. Detected PID ISA-3000-4C. Found device serial number FCH2443L042. Found USB flash drive /dev/sdc Found hard drive(s): /dev/sda fsck from util-linux 2.32.1 fsck.fat 4.1 (2017-01-24) /dev/sdc1: 54 files, 21002/1800351 clusters ============================================== Use ESC to interrupt boot and launch boot CLI. Use SPACE to launch Cisco FTD immediately. Cisco FTD launch in 30 seconds ... Cisco FTD launch in 29 seconds ... Cisco FTD launch in 28 seconds ... Running on kenton Mounting disk partitions ... Failed to open /dev/md0p4 2022-02-15 16:07:49 Starting Cisco Firepower Threat Defense ... 2022-02-15 16:07:49 Found USB flash drive /dev/sdc 2022-02-15 16:07:49 Found hard drive(s): /dev/sda 2022-02-15 16:07:49 Running [mkdir -p /mnt/disk0] ... success 2022-02-15 16:07:50 Running [mkdir -p /mnt/disk1] ... success 2022-02-15 16:07:50 Running [cp -f /mnt/disk0/.private/mdadm.conf /etc/mdadm.conf] ... success 2022-02-15 16:07:50 Running [mdadm --assemble --scan -c /mnt/disk0/.private/mdadm.conf] ... success 2022-02-15 16:07:50 mdadm: /dev/md0 has been started with 1 drive (out of 2). 2022-02-15 16:07:50 Running [mkdir -p /ngfw] ... success 2022-02-15 16:07:50 Running [rm -rf /etc/rc.d] ... success 2022-02-15 16:07:50 Running [rm -rf /etc/sf] ... success 2022-02-15 16:07:50 Running [rm -rf /etc/sysconfig] ... success 2022-02-15 16:07:50 Running [rm -rf /root] ... success 2022-02-15 16:07:50 Running [rm -rf /usr/local/sf] ... success 2022-02-15 16:07:50 Running [rm -rf /var/lib/rpm] ... success 2022-02-15 16:07:50 Running [rm -rf /var/sf] ... success 2022-02-15 16:07:50 Running [rm -rf /Volume] ... success 2022-02-15 16:07:50 Running [rm -rf /asa] ... success 2022-02-15 16:07:50 Running [/usr/sbin/diskkey -o /dev/md0p1:swap] ... success 2022-02-15 16:07:53 Running [/usr/sbin/diskkey -o /dev/md0p2:root] ... success 2022-02-15 16:07:57 Running [/usr/sbin/diskkey -o /dev/md0p4:var] ... failed 2022-02-15 16:08:05 Failed to generate admin ping. 0xfc-TAM_LIB_ERR_READ_CHECKSUM_FAILURE 2022-02-15 16:08:05 Failed to initialize the hardware. 2022-02-15 16:08:05 Failed to open /dev/md0p4 Un-mounting disk partitions ... Entering boot CLI ... Detected PID ISA-3000-4C. Found device serial number FCH2443L042. Found USB flash drive /dev/sdc Found hard drive(s): /dev/sda /dev/dm-1 fsck from util-linux 2.32.1 fsck.fat 4.1 (2017-01-24) /dev/sdc1: 54 files, 21002/1800351 clusters ============================================== Use ESC to interrupt boot and launch boot CLI. Use SPACE to launch Cisco FTD immediately. Cisco FTD launch in 30 seconds ... Cisco FTD launch in 29 seconds ... Cisco FTD launch in 28 seconds ... Cisco FTD launch in 27 seconds ... Cisco FTD launch in 26 seconds ... Cisco FTD launch in 25 seconds ... Cisco FTD launch in 24 seconds ... Cisco FTD launch in 23 seconds ... Cisco FTD launch in 22 seconds ... Cisco FTD launch in 21 seconds ... Cisco FTD launch in 20 seconds ... Cisco FTD launch in 19 seconds ... Cisco FTD launch in 18 seconds ... Cisco FTD launch in 17 seconds ... Block device not found. Continue with /dev/sda. Configuring packages on first boot.... (This m INIT: Entering runlevel: 3 Starting system message bus: dbus. Starting OpenBSD Secure Shell server: sshd generating ssh rsa key... generating ssh dsa key... generating ssh ecdsa key... generating ssh ed25519 key... done. Starting Advanced Configuration and Power Interface daemon: acpid. acpid: starting up with netlink and the input layer acpid: 1 rule loaded acpid: waiting for events: event logging is off Starting DHCP server: . Starting LXC autoboot containers: Starting syslog-ng:. Starting internet superserver: xinetd. Not UCS or 3RU Platform Starting ntpd: done /etc/rc3.d/S97start_csp.sh: line 14: /opt/cisco/platform/logs/ssp-rm.log: No such file or directory /etc/rc3.d/S97start_csp.sh: line 34: /opt/ssp-mgmt-scripts/ssp_rm_utils.sh: No such file or directory /etc/rc3.d/S97start_csp.sh: line 51: /opt/ssp-mgmt-scripts/ssp_rm_utils.sh: No such file or directory xfs_quota: cannot setup path for mount /opt/cisco/csp: No such device or address xfs_quota: cannot find mount point for path `/opt/cisco/csp/cores': No such file or directory xfs_quota: cannot setup path for mount /opt/cisco/csp: No such device or address xfs_quota: cannot find mount point for path `/opt/cisco/csp/cores': No such file or directory Start_CSP_RcInit in Progress........... /etc/rc3.d/S97start_csp.sh: line 131: /opt/cisco/platform/logs/ssp-rm.log: No such file or directory INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_system_maintenance_mode.sh" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init"cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: Id "heim" respawning too fast: disabled for 5 minutes INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: Id "auth" respawning too fast: disabled for 5 minutes INIT: Id "ssp" respawning too fast: disabled for 5 minutes INIT: Id "ssp0" respawning too fast: disabled for 5 minutes INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: Id "auth" respawning too fast: disabled for 5 minutes INIT: Id "heim" respawning too fast: disabled for 5 minutes INIT: Id "ssp" respawning too fast: disabled for 5 minutes INIT: Id "ssp0" respawning too fast: disabled for 5 minutes INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/auth_agent_init" INIT: cannot execute "/opt/ssp-mgmt-scripts/ssp_heimdall_init" INIT: Id "auth" respawning too fast: disabled for 5 minutes INIT: Id "heim" respawning too fast: disabled for 5 minutes