Symptoms

Apache HTTP logging can lead to premature boot drive failure due to excessive writes.

Cause

By default, Apache is configured to log all HTTP access and HTTP errors events to the following logs: /var/log/apache2/webui_httpd_error.log /var/log/apache2/webui_httpd_access.log Depending on the workflow, this logging can lead to significant wear on the boot drives causing premature failure.

Resolution

Disable or decrease the amount of logging to the webui_httpd_error.log and webui_httpd_access.log by modifying the syslogd log level.1. Back up the existing syslog configuration. cp /etc/mcp/templates/syslog.conf /ifs/data/Isilon_Support/syslog.conf cp /etc/mcp/templates/syslog.conf /ifs/data/Isilon_Support/syslog.conf.old <-Copy of original 2. Modify syslog.conf with vi, nano, or similar text editor. nano /ifs/data/Isilon_Support/syslog.conf 3. Locate the section !httpd it appears as follows: !httpd local2.* /var/log/apache2/webui_httpd_error.log local3.* /var/log/apache2/webui_httpd_access.log 4. Comment out the following lines with a statement including the SR number (if any) and why this was modified for so it will be clear in the future why this change was made. #COMMENT OUT FOR SR TO REDUCE APACHE LOGGING #!httpd #local2.* /var/log/apache2/webui_httpd_error.log #local3.* /var/log/apache2/webui_httpd_access.log 5. Add the following lines with a statement including the SR number (if any) and indicating the log level is set to none. #LOG LEVEL CHANGED TO NONE FOR SR !httpd local2.none /var/log/apache2/webui_httpd_error.log local3.none /var/log/apache2/webui_httpd_access.log #LOG LEVEL CHANGED TO NONE FOR SR !httpd local2.none /var/log/apache2/webui_httpd_error.log local3.none /var/log/apache2/webui_httpd_access.log 4. Copy the edited file to all nodes. isi_for_array 'cp /ifs/data/Isilon_Support/syslog.conf /etc/mcp/templates/syslog.conf' 5. Verify that the new file has been copied successfully to all nodes. Isilon02-1# md5 /ifs/data/Isilon_Support/syslog.conf MD5 (/ifs/data/Isilon_Support/syslog.conf) = f0626c10f6a88dabeb13183255debba4 Isilon02-1# isi_for_array -s 'md5 /etc/mcp/templates/syslog.conf' Isilon02-1: MD5 (/etc/mcp/templates/syslog.conf) = f0626c10f6a88dabeb13183255debba4 Isilon02-2: MD5 (/etc/mcp/templates/syslog.conf) = f0626c10f6a88dabeb13183255debba4 Isilon02-3: MD5 (/etc/mcp/templates/syslog.conf) = f0626c10f6a88dabeb13183255debba4 6. Once copied restart the syslog for change to take effect. isi_for_array 'killall -HUP syslogd' 7. Confirm logging to the access and error logs have stopped and are no longer spamming or rolling over frequently. tail -f /var/log/apache2/webui_httpd_access.log tail -f /var/log/apache2/webui_httpd_error.log

Support Cases