14 March 2013

Mysterious cab files fill-up temp folder

Our Labtech RMM recently spit out a warning that a client's 2008R2 server had their disk space filling up fast (Note they are not a premium customer):

Upon closer inspection I found that every hour an unknow process would attempt to write a .cab file of approx 60MB to the Windows temp folder. Checking with Process Explorer I found that it was makecab.exe writing these files. Makecab was invoked by services.exe, so that was a bit of a dead end. I looked through the list of Windows scheduled tasks, but did not find anything that was supposedly run every hour.

I tried to rename the cab files created, adding a .cab extension, but they turned out corrupt. I then used Sysinternals ProcessExplorer to find the source of the cab file, which was tricky, as it would only take a few minutes to actually create the file. The indication was that the source of the cab file (eg the file being compressed) was from the c:\windows\logs\cbs folder.

I learned a bit more about Windows Resource Protection (WRP), which stops programs overwriting essential system files. It keeps its log files in this folder. The SFC.exe program writes the details of each verification operation and of each repair operation to the CBS.log file. The CBS.persist.log is generated when the CBS gets to be around 50Mb in size. CBS.log is copied to cbs.persist.log and a new cbs.log file is started. A bit of Google foo and we determine that the cbs logs would only be useful for serious troubleshooting issues. If the system is running fine, we can delete this file. SFC.exe will create a new one, next time it is run.
So why do we have this mysterious process writing a file here?? It appears that on this server the SFC archive process has not been running for a long tome. On the 9th Feb the system was restarted and the log got recycled. Due to the long period of not being archived the log file grew to 4GB in size. Now the fun begins: Every hour, the archive process tries to create a new .cab file. I now speculate that the file size is larger than  what is supported and the process fails, hence resulting in a partial .cab file that sits in the temp folder, rather than a complete .cab file in the CBS log folder.
I have deleted the offending .cab file and most of the other ones too, just keeping a few recent ones in case we need them. No more mysteries!