OPERATIONAL DEFECT DATABASE
...
...
Processes using temporary scratch disk space on ESXi 4.1 Update 1 hosts fail: Scanning or remediating an ESXi 4.1 Update 1 host with VMware Update Manager fails. Update Manager reports the error:The host returns esxupdate error codes: 10. Check the Update Manager log files and esxupdate log files for more details In the esxupdate.log file located at /var/log/vmware , you see an error similar to:FileIOError: ('/var/tmp/cache/metadata-1539263447', "Cannot create dir /var/tmp/cache/metadata-1539263447: [Errno 17] File exists: '/var/tmp'") Installing third-party drivers or packages on an ESXi 4.1 Update 1 host using the vihostupdate command-line utility fails. The vihostupdate command-line utility reports the error:Error encountered:Description - I/O Error (17) on file /var/tmp: [Errno 17] File exists: '/var/tmp'Message - Unable to create, write or read a file as expected.I/O Error (17) on file /var/tmp: [Errno 17] File exists: '/var/tmp' Generating a Diagnostic Data log bundle from an ESXi 4.1 Update 1 host using the vSphere Client fails. The vSphere Client reports the error:Log Bundle has not been generated Generating a Diagnostic Data log bundle from an ESXi 4.1 Update 1 host using the vm-support command-line utility fails. The vm-support command-line utility reports the error:/sbin/vm-support: cd: line 2448: can't cd to /var/tmpCould not set working directory to '/var/tmp'. Performing a backup of an ESXi 4.1 Update 1 host using the vicfg-cfgbackup command-line utility fails. The vicfg-cfgbackup command-line utility reports the error:Saving firmware configuration to ...GET https://hostname/downloads/configBundle-hostname.tgz unsuccessful : 404 Not Found Performing a vMotion fails at 82%. The Console tab fails with the error:Unable to connect to the MKS: There is no VMware process running for the config file /vmfs/volumes/DatastoreName/DirectoryName/VirtualMachineName.vmx Resetting the virtual machine fails at 95%. The host reports the error:Unable to apply DRS resource settings on the host. The operation is not allowed in the current state.. This can significantly reduce the effectiveness of DRS. Attempting to manually inspect or utilize the /var/tmp directory fails: Open a console to the ESXi host. For more information, see Using Tech Support Mode in ESXi 4.1 (1017910). Attempt to change to the /var/tmp directory using the command:cd /var/tmpIf this command fails with "-ash: cd: can't cd to /var/tmp", the system is potentially affected.
This article describes a specific issue specific to ESXi 4.1 Update 1, in which the /sbin/tmpwatch.sh cleanup script may remove temporary directories relating to the scratch location. If you experience one of the above symptoms and manually inspecting the temporary scratch directory fails, consult the sections below. If any of the above work succeeds, then your issue is not related to this article. Search for your symptoms further or Open a Support Request.
This issue occurs when the temporary directory /var/tmp is unavailable. It may have been deleted by the /sbin/tmpwatch.sh cleanup script due to an issue which affects ESXi 4.1 Update 1 hosts which have a non-persistent, ramdisk-backed scratch location. Operations such as installing patches and collecting log bundles rely on temporarily writing data to a scratch location on the ESXi host. These temporary files are placed in the /var/tmp directory, which points to the /scratch directory, which may be backed by a a ramdisk or persistent storage location. If the /var/tmp and /scratch directories are not available, these operations will fail. Possible scratch locations include: A VMFS Datastore's .locker directory. For example: /vmfs/volumes/UUID/.locker/ A FAT16 volume on the boot or other local media. For example: /vmfs/volumes/UUID/ A Ramdisk. For example: /tmp/scratch/ ESXi 4.x periodically deletes temporary files contained in the /tmp directory that are older than 10 days. Under some circumstances, the /sbin/tmpwatch.sh cleanup script may remove the /tmp/scratch directory and its contents, leaving the /var/tmp and /scratch symlinks with no target. In this situation, operations attempting to place new temporary files in these directories will fail. For more information on ESXi and the Scratch location, see the About the Scratch Partition section of the ESXi Installable and vCenter Server Setup Guide 4.1, the VMware ESXi Chronicles blog entry Ops changes part 5 - Scratch partition and Creating a persistent scratch location for ESXi (1033696). Solution This issue is resolved in the patch release for ESXi. For more information, see VMware ESXi 4.1, Patch Release ESXi410-201104001 (1035111). Workaround To workaround this issue, when you do not want to apply the patch, try one of these two options: This issue does not affect hosts which have persistent scratch space configured. To workaround this issue, configure the ESXi 4.1 Update 1 hosts with persistent storage. To implement this workaround, see Creating a persistent scratch location for ESXi (1033696). Recreate the missing scratch directory on the ramdisk. To prevent it from being deleted again, create a symbolic link within the scratch directory. Note: The target of the symlink does not have to exist. The symlink itself will not be deleted by the /sbin/tmpwatch.sh cleanup script, nor the directory containing it. To implement this workaround: Open a console to the ESXi host. For more information, see Using Tech Support Mode in ESXi 4.1 (1017910). Create the missing directories using the command:/sbin/configLocker Create a symbolic link within the scratch directory to prevent it from being deleted by the /sbin/tmpwatch.sh script:ln -fs /scratch/kb1037190 /scratch/.keep Open the file /etc/rc.local in a text editor. For more information, see Editing configuration files in VMware ESX (1017022). Add the same command from Step 3 to the end of the file on its own line:ln -fs /scratch/kb1037190 /scratch/.keep Save the file. Back up the configuration change to the bootbank so it persists across reboots using the command:auto-backup.sh Note: In ESXi Embedded, this issue may also may occur when the temporary directory /var/tmp is pointing to a location which is unexpected by the process that you are attempting to run, such as collecting logs and installing patches. The correct location should be /tmp/scratch/var/tmp . To create this directory, run this command from the ESXi host:# mkdir -p /tmp/scratch/var/tmp
To be alerted when this document is updated, click the Subscribe to Article link in the Actions boxError when running vm-support if insufficient disk space
Click on a version to see all relevant bugs
VMware Integration
Learn more about where this data comes from
Bug Scrub Advisor
Streamline upgrades with automated vendor bug scrubs
BugZero Enterprise
Wish you caught this bug sooner? Get proactive today.
