locked
How I *Know* WHS isn't corrupting my share files..... RRS feed

  • Question

  • The threads about file corruption made me think a little about how I would see when a corruption happens...

     

    BTW, I use WHS for file and machine backups only, plus occasional remote access.  That role is so important, I have a WHS dedicated to just that purpose, and I don't touch or mess with that.  If I want to do anything else, I load another WHS into a virtual machine and play with that.  I don't edit files off the shares directly, I have scheduled tasks on each machine that copy files from the machines to the WHS share, via the Hobocopy utility.  First generation of WHS - use the silk booties - Duh!

     

    Do you have online backup going somehow (I use the JungleDisk add-in)?  If not, get it going.  My CRC checker runs several hours before the JungleDisk backup runs, so I can stop JungleDiskt in time if the CRC checker finds corruption, and get my original files back off JungleDisk.

     

    I've been running WHS for over a year now, and there are no problems, here is how I know....

     

    Download a CRC check utility like fsum, from here:

    http://www.slavasoft.com/fsum/

     

    Download a command line emailer, like vmailer from here:

    http://virdi-software.com/vmailer/desc.shtml

     

    Notice how fsum has a switch to build CRC values for all files in a directory, and store those into a specific .sfv file.

    There is also a fsum switch to compare the current file CRC values against the values you stored in that .sfv file.

     

    Once a day, for each share directory and subdirectory you have, run a .bat file that:

     

    1.  Uses fsum to compare the current CRC values against what is in the directory .sfv value, and pipe any results to a file.

         Skip step 1 if the .sfv file doesn't exist yet.

    2.  Use fsum to regenerate all CRC values into that .sfv file (for tomorrow's run), to account for adds or deletes.

    3.  Email those results to the email address of your choice, using vmailer.

     

    Repeat the above steps for EACH directory in each share you want to verify.

     

    Once the report comes, you need to figure out if a CRC value change is a legitimate file change, or a corruption.  The report can't tell, it just knows a specific file now has a different CRC value than it did yesterday.

     

    If someone wants to build this into a add-on, please do....

     

    Yes, there was a reason I put the solution in pseudo-code instead of giving you the full code directly.  If you look at the tech-support angle, you'll figure out why.  I do want to give something back, though.

     

    DDanster

     

     

     

     

     

     

     

    Wednesday, March 12, 2008 8:49 PM

Answers

  • Hey great post!  There has been few posts about how users can check to see if their files are being affected by KB 946676. Those users should give this a try.

    Wednesday, March 12, 2008 10:35 PM
    Moderator

All replies

  • Hey great post!  There has been few posts about how users can check to see if their files are being affected by KB 946676. Those users should give this a try.

    Wednesday, March 12, 2008 10:35 PM
    Moderator
  • Any chance you would like to share your code?

    Are you using a scheduled task to kick it off?

    Where are the CRC check files kept?  I do not want to fill up my shares with the check files.

     

     

    Wednesday, March 19, 2008 6:56 PM
  • The problem is that the file corruption doesn't just occur at an arbitrary time (i.e. this bug doesn't cause the file to change "on its own"). The file become corrupted at the same time as a legitimate file change. It's only when you intentionally modify the file that the wrong data gets written. There is really no way to detect this sort of corruption with CRCs (or any other automated method). The only solution is to save to a local (non-WHS) drive, then copy to the WHS.

     

    --Doug

    Wednesday, March 19, 2008 7:54 PM