Symptom

TimingIC-A and TimingIC-B FPD upgrade is not included in FPD "upgrade all" and "fpd auto-upgrade" because these FPD are not guaranteed to work post a firmware upgrade if they are not immediately reloaded. However, this has significant negative consequences: 1) additional manual FPD upgrade is required after the XR+FPD upgrade; 2) overall upgrade time is extended by at least ~20 min; 3) upgrade MoP is more complicated. This is an enhancement request to improve the FPD upgrade orchestration of TimingIC-A and TimingIC-B FPD. TimingIC-A and/or TimingIC-B upgrade may fail with the following error: LC/0/2/CPU0:2022 Sep 27 01:39:24.193 : zllc[192]: %PLATFORM-PLL-3-UPGRADE_FAIL : Firmware upgrade failed for device TimingIC-A. Err-code: 0x7de Failed to load some files (words loaded: uPath 0, fwPath 0, fw2Path 0)

Conditions

Observed during FPD upgrade from initial FPD release (shipped with IOS XR release 7.1.x) to FPD version 7.212 (shipped with IOS XR release 7.3.x). Upgrade fails because a local copy of a file from line card disk into FPGA is failing.

Workaround

Further Problem Description

TimingIC-A and TimingIC-B FPGA don't have a primary and secondary partition. Therefore, FPD upgrade is performed directly into the only partition. Additionally, as TimingIC FPGAs provide clock to all components on the line card (including PHY), link flap may be observed during the upgrade. TimingIC-A and TimingIC-B FPD upgrade should be performed one after another, immediatelly followed by a single reload. As some customers upgrade FPDs during normal business hours and then reload the cards during a MW, such customers could observe link flaps if TimingIC-A and TimingIC-B FPD upgrade were included in "upgrade hw-module location all" command. Consequently, TimingIC-A and TimingIC-B FPD upgrade is also not included in "fpd auto-upgrade". This enhancement is closed because the issue was unreproducible and other analysis did not show possible ways to improve the behaviour.