Symptoms

Gen6 nodes may report spurious Battery Backup Unit (BBU) failures similar to the following: Battery Test Failure: Replace the battery backup unit in chassis <serial number> slot <number> as soon as possible.

Cause

Issues with both the OneFS battery test code, and the Battery Charge Controller (BCC) firmware that can cause these errors to be reported.

Resolution

If the node dial home is for OneFS 9.x or later, and explicitly states: has indicated cell End Of Life. Replace the battery backup unit as soon as possible. See Isilon: How to Check the Battery Wear Threshold for a Gen6 Cluster for additional information.The underlying causes for most battery test failures are resolved in OneFS 8.1.0.4 and later, and Node Firmware Package version 10.1.6 and later (DEbcc/EPbcc version 00.71). To resolve this issue, upgrade to these software versions, in that order, as soon as possible. If after a battery or riser replacement the DEbcc/EPbcc firmware reports 00.00 the node firmware 10.3.3 must be upgraded on this node. The following steps are required to restart the battery after the firmware is at 01.20: Check the BBU logs for a "Persistent fault" message. This indicates a test failure state that cannot be cleared in the field. Run the following command on the affected node: OneFS_v8.x.x and older: # isi_hwmon -b | grep "Battery 1 Status" OneFS_v9.x.x and newer: # isi_hwmon -b BatteryMonitoring If the battery reports a Persistent Fault condition, gather and upload logs using the isi_gather_info command, contact Dell Isilon Technical Support and reference this KB article. If OneFS version 8.1 x and older it is necessary to initiate a manual battery test by performing the following commands: # isi services isi_hwmon disable # mv /var/log/nvram.xml /var/log/nvram.xml.old # isi services isi_hwmon enable If OneFS version 8.2.x and newer, it is necessary to initiate a manual battery test by performing the following command: # isi_hwmon -s If OneFS version is 8.2.x or newer, the nvram.xml is located in the /etc/ifs directory. Clear the battery test alert and unset the node read-only state so the upgrade can proceed. Check 'isi event events list' to get the event ID for the HW_INFINITY_BATTERY_BACKUP_FAULT event. Then run the following commands: # isi event modify <eventid> --resolved true # /usr/bin/isi_hwtools/isi_read_only --unset=system-status-not-good Upgrade OneFS to version 8.1.0.4 or later. Instructions for upgrading OneFS are found in the OneFS Release Notes on the Dell Support website. Update node firmware using Node Firmware Package 10.1.6 or later. Instructions to upgrade node firmware are found in the Node Firmware Package Release Notes on the Dell Support website. Once the system is upgraded, no additional battery replacement alerts should occur.If a OneFS upgrade to 8.1.0.4 or later is not a current option, or if the system generates more battery failure alerts after upgrading, contact Dell Isilon Technical Support for assistance, and reference this KB article.

Support Cases