Symptoms

Code upgradeUpgrade fail due to 0x40820011 timeout & SRG extraction failedPre-checks error:"Extracting "bios" version from image bootflash:/m9500-sf2ek9-mz.6.2.11c.bin.[# ] 0% -- FAIL.Return code 0x40820011 (Job timedout).Pre-upgrade check failed. Return code 0x40930012 (SRG extraction failed)." %TACACS-6-TACACS_ACCOUNTING_MESSAGE: update:.x.x.x.x@pts/0: selvarar_sto: install all system bootflash:/m9500-sf2ek9-mz.6.2.11c.bin kickstart bootflash:/m9500-sf2ek9-kickstart-mz.6.2.11c.bin (FAILURE) xx.x.x@pts/0 %TACACS-6-TACACS_ACCOUNTING_MESSAGE: update: : : Install all failed. %USER-5-SYSTEM_MSG: > System upgrade aborted. - new_installer %SYSMGR-3-SERVICE_TERMINATED: Service "installer" (PID 24366) has finished with error code SYSMGR_EXITCODE_FAILURE_NOCALLHOME (20). The internal install logs detail an SRG extraction failure with the respective error-id: `show system internal log install details`srv_installer_job_done: procjob callback failed. (0x40820011)process_srg_extraction_done: SRG extraction failed (0x40820011).notify_job_status_change: Sending job status notification.installer_state_change: old_state: INSTALLER_STATE_SRG_EXTRACTION_WAIT, new_state: INSTALLER_STATE_ABORTINGinstaller_state_change: State changed from INSTALLER_STATE_SRG_EXTRACTION_WAIT to INSTALLER_STATE_ABORTING.notify_global_status_change: release config mode lockunlock_config_mode: Releasing config mode lockbind_mts_vdc: bind done for default vdc: 9unbind_mts_vdc: unbind succeeded: q(9)notify_global_status_change: Sending global status notification.notify_global_status_change: Getting vdc node info.notify_global_status_change: Sending global status notification to all vdcs.fsm_action_srv_installer_aborting: Waiting for 10 secs before exiting..process_install_start: got install request: abort_installation (0), install_operation (1).process_install_start: failure_reason: 0x40930012abort_installation_stub: We are in state "INSTALLER_STATE_ABORTING" and got a request for abort_and_exit, going to state DONE. installer_state_change: old_state: INSTALLER_STATE_ABORTING, new_state: INSTALLER_STATE_DONEinstaller_state_change: State changed from INSTALLER_STATE_ABORTING to INSTALLER_STATE_DONE.unlock_all_modules: unlocking all modules show system error-id 0x40930012 Error Facility: installerError Description: SRG extraction failed MTS messages queued on both the Active and Standby Supervisors: Activeshow system internal mts buffer summarynode sapno recv_q pers_q npers_q log_qsup 3 0 0 0 11show system internal mts buffer detailNode/Sap/queue Age(ms) SrcNode SrcSAP DstNode DstSAP OPC MsgId MsgSize RRToken Offsetsup/3/log 1472169795 0x501 44352 0x501 3 2482 0x5422c2c4 83 0x5422c2c4 0xaa4104sup/3/log 1472139758 0x501 44352 0x501 3 2482 0x5422c30d 75 0x5422c30d 0xaa4304sup/3/log 1472109738 0x501 44352 0x501 3 2482 0x5422c351 91 0x5422c351 0xaa4404sup/3/log 63330004 0x501 1010 0x501 3 2482 0x555a5675 89 0x555a5675 0xaa4504sup/3/log 63149991 0x501 1010 0x501 3 2482 0x555a65b5 89 0x555a65b5 0xaa4704sup/3/log 62969959 0x501 1010 0x501 3 2482 0x555a71c7 89 0x555a71c7 0xaa4204sup/3/log 62789929 0x501 1010 0x501 3 2482 0x555a7726 89 0x555a7726 0xaa4004sup/3/log 62031170 0x501 1010 0x501 3 2482 0x555a9bc8 89 0x555a9bc8 0xaa4d04sup/3/log 61851151 0x501 1010 0x501 3 2482 0x555aa163 89 0x555aa163 0xaa4604sup/3/log 61671119 0x501 1010 0x501 3 2482 0x555ab0fa 89 0x555ab0fa 0xaa4804sup/3/log 61491089 0x501 1010 0x501 3 2482 0x555ab3a4 89 0x555ab3a4 0xaa4904 Standbyshow system internal mts buffer summarynode sapno recv_q pers_q npers_q log_qsup 6 2 0 0 0sup 3 0 0 2 0show system internal mts buffer detailNode/Sap/queue Age(ms) SrcNode SrcSAP DstNode DstSAP OPC MsgId MsgSize RRToken Offsetsup/6/recv 64452983 0x601 59525 0x601 0 2520 0xf236c6 0 0xf236c6 0xa80304sup/6/recv 11663 0x601 60353 0x601 0 2520 0xf28112 0 0xf28112 0xa80404sup/3/nper 5230061892 0x501 6 0x601 3 1362 0x50edca7d 272 0x50edca7d 0xa80004sup/3/nper 1472184783 0x501 44353 0x601 3 2483 0x5422c2c7 71 0x5422c2c7 0xa80204

Cause

The BIOS upgrade daemon is having issues communicating with System Manager & also there are sync issues causing this ISSU failure.

Resolution

Check for MTS messages using the below commands: show system internal mts buffer detail show system internal mts buffer summary In case they are queued on the supervisors we can clear them using the below steps1)Power cycle the Standby-SUP to clear the MTS messages on Standby Switch#reload module x wait for standby-SUP to boot to ha-standby mode2) Perform System Switchover to clear stuck MTS messages on the current Active-SUP Switch#system switchover wait for previous active to boot to ha-standby mode3)Retry ISSUNote: You may need to copy the kickstart/system files from sup-1 to sup-2 or from ftp server to sup-2If the problem still exits, please contact your EMC Customer Service representative or the EMC Customer Support Center for technical assistance and upload the switch logs to the service request, so that support staff can check them and provide you with a feedback. Please quote the solution ID.

Support Cases