Symptoms
SyncIQ's Policy and Reports pages in the OneFS WebUI (Web Administration Interface) load slowly or do not load at all. SyncIQ CLI commands are also slow.
Cause
Many failed SyncIQ reports could exceed the 2000 max reports default and lead to slow performance in the OneFS WebUI (Web Administration Interface).
Resolution
Cancel any running job and disable the SyncIQ service on the cluster:
Cancel all running SyncIQ jobs:
isi sync jobs cancel --all
Verify that the jobs are canceled:
isi sync jobs reports list
If the command is unresponsive, check and ensure that there are no entries in the following path:
ls -alh /ifs/.ifsvar/modules/tsm/sched/run
Disable the SyncIQ service:
isi services -a isi_migrate disable
Verify that all processes have stopped:
isi_for_array -sX ps -auwwx | grep migr | grep -v grep
Perform the following procedure to clear reports:
If a backup is needed, make a backup of the reports directory (This may take time).
cp -r /ifs/.ifsvar/modules/tsm/sched/reports /ifs/data/Isilon_Support/synciqreports_backup
Delete reports job to remove the reports directory:
find /ifs/.ifsvar/modules/tsm/sched/reports -type f -name "report-*.gc" -exec rm -f {} \;
Alternatively, delete all but the last 3 days worth of reports, for example:
find /ifs/.ifsvar/modules/tsm/sched/reports -type f -name "report-*.gc" -Btime +3d -exec rm -f {} \;
After deletion, rebuild the reporting database:
python -c 'import isi.fs.siq;isi.fs.siq.SyncIQUtils().rebuild_reportdb();'
Restart the SyncIQ service and verify it is running:
Re-enable the sync service:
isi services -a isi_migrate enable
Verify that the service is running on each node:
isi_for_array -sX ps -auwwx | grep migr | grep -v grep
Retry WebUI and CLI commands to ensure it is functional again.
NOTE: Policies configured with "Skip When Source Unmodified: Yes", using this procedure can cause improper reporting behavior. The job_id for a skipped job will not iterate causing reports to build up again. The workaround to this behavior is to manually run a job to success.
