Symptom
Upgrading to IOS image larger than 512MB on codesigning capable routers (ASR 1001-X, ASR1002-X) will fail with below error messages:
Using FLASH based Keys of type = PRIMARY KEY STORAGE
Using FLASH based Keys of type = ROLLOVER KEY STORAGE
Signature verification failed for key# 2
Using FLASH based Keys of type = ROLLOVER KEY STORAGE
Signature verification failed for key# 3
Failed to validate digital signature
Using FLASH based Keys of type = BACKUP KEY STORAGE
Using FLASH based Keys of type = ROLLOVER KEY STORAGE
Signature verification failed for key# 2
Using FLASH based Keys of type = ROLLOVER KEY STORAGE
Signature verification failed for key# 3
Failed to validate digital signature
RSA Signed REVOCATION Image Signature Verification Failed.
signature verification FAILED...
boot: error executing "boot bootflash:asr1002x-universalk9.16.04.01.SPA.bin"
autoboot: boot failed, restarting...
Conditions
ASR1k product line: limited to ASR1001-X, ASR1002-X when loading IOS-XE images > 512M (IOSXE 16.x and later)
ISR4k product line: This error is seen on ISR 4k routers as well when upgrading to 16.6.3 from 16.3.6
Workaround
Upgrade ROMMON or run an IOS-XE version with image size < 512M
ASR1k product line: ROMMONs 15.5(3r)S1 and later contain this fix.
ISR 4k product line: ROMMON upgrade also failed on 16.3.6 release. So, we had to downgrade the IOS to 3.16 release first. Now upgrade the rommon to 16.7(4r) latest rommon release and then upgrade the IOS to 16.6 release
Further Problem Description
Bug is addressed through a rommon fix. Upgrade rommon to the latest/recommended release on cisco.com to resolve the issue.