none
Tired of file conflicts. Tired of DEMigrator going insane. How to rid myself of them, move to raid? RRS feed

  • Question

  • I know, DEMigrator.exe is checking the health of my system, moving duplicated files around, etc.  I am just tired of hearing my HDDs run constantly, for weeks now, going insance on read/writing.  Most likely cause I have dozens of projects of 1000s of files, all in SVN - so that's  most likely > 500,000 files at the very very least.

    I am also tired of random file conflicts that are showing up for no reason.  Yes, I had a previous issue with my 2nd sata onboard controller, and switched to the primary sata onboard controller.  Since then I have turned off duplication (waited a day).  Deleted the files (waited a day).  And finally, re-copied the files from my backup device to ensure a clean copy.  Waited a day, then enabled Duplication on the folders  But yet, I still get an "ALERT! File(s) in conflicted state" once or twice a day.

    So my question (and really an assurance of a theory) is, can I switch to a hardware raid solution, remove duplication, and be done with all of this insanity?

    I have a few highend adaptec pcie cards just laying here (need to ebay).  I tried the WHS Duplication route and liked the extra HDD space, yes.  But, for what DEMigrator is doing to my HDDs by constantly making them go insane, I think I would get more life out of my HDDs if I put them onto some spin-down adaptec controllers on some raid6 containers.  Loosing space, but getting more assurance then these file conflict issues and hdd insanity is giving me.




    http://eduncan911.com
    Thursday, October 23, 2008 11:08 PM

Answers

  • I'm going to propose a different source, and solution, for your disk thrashing: Windows Desktop Search. If you have your svn tree in a share, Windows Home Server will use Windows Desktop Search to try to index it for faster searching. (Windows Desktop Search is the engine behind the search feature in Remote Access.) If that's the cause, the solution is to get your repository out of your WHS shares, I'm afraid.
    I'm not on the WHS team, I just post a lot. :)
    • Marked as answer by eduncan911.com Friday, February 19, 2010 9:34 PM
    Saturday, October 25, 2008 1:43 PM
    Moderator

All replies

  • eduncan911.com said:

    I know, DEMigrator.exe is checking the health of my system, moving duplicated files around, etc.  I am just tired of hearing my HDDs run constantly, for weeks now, going insance on read/writing.  Most likely cause I have dozens of projects of 1000s of files, all in SVN - so that's  most likely > 500,000 files at the very very least.

    I am also tired of random file conflicts that are showing up for no reason.  Yes, I had a previous issue with my 2nd sata onboard controller, and switched to the primary sata onboard controller.  Since then I have turned off duplication (waited a day).  Deleted the files (waited a day).  And finally, re-copied the files from my backup device to ensure a clean copy.  Waited a day, then enabled Duplication on the folders  But yet, I still get an "ALERT! File(s) in conflicted state" once or twice a day.

    So my question (and really an assurance of a theory) is, can I switch to a hardware raid solution, remove duplication, and be done with all of this insanity?

    I have a few highend adaptec pcie cards just laying here (need to ebay).  I tried the WHS Duplication route and liked the extra HDD space, yes.  But, for what DEMigrator is doing to my HDDs by constantly making them go insane, I think I would get more life out of my HDDs if I put them onto some spin-down adaptec controllers on some raid6 containers.  Loosing space, but getting more assurance then these file conflict issues and hdd insanity is giving me.




    http://eduncan911.com




    Doing that is unsupported.  However, if you search through this forum, you will see other users have gotten it to work.
    Friday, October 24, 2008 1:45 AM
    Moderator
  • Do you have power pack 1 installed?  This update really decreased the amount of hard drive and demigrator activity.
    athlon 3400, 2gb ram, 9 drives totaling about 3.5 tbs.
    Friday, October 24, 2008 2:39 AM

  • I've got a RAID5 array as one of my volumes.  Oddly enough, I've been thinking about reverting the array back into the individual disks.  Setting up the RAID5 was just an experiment.

    I've got a post of creating the array around somewhere.  It's using the software RAID on my motherboard (ICH10R).  I even put some benchmarkings results up (well, providing the images didn't get deleted already).
    Friday, October 24, 2008 3:11 AM
  • So a thing about RAID.  Mirroring might be something worth trying to do on the system drive, although a server reinstall is the suggested way to go.  As far as RAID 0 (striping) it's useless for WHS in my opinion.  99% of the time i'm transferring stuff to and from WHS via a gigabit network.  I've tweaked it and get anywhere from 50-80 meg a second.  This is handled very easily by my sata drives without raid.  Adding a strip raid array would be worthless because the network is the bottleneck, not the drive speeds.  not to mention the increased chance of disk failure.

    I have however upgraded my gaming pc to raid 0 arrays with velociraptor drives.  I figured it gets backed up nightly on WHS so if there are any issues I have those.  The computer is extremely fast.

    athlon 3400, 2gb ram, 9 drives totaling about 3.5 tbs.
    Friday, October 24, 2008 3:16 AM
  • Thanks everyone for the feedback.  I can handle the raid setup.  ;)

    As for the question if I have PP1 installed - yes.  This is a new server as of about a month ago, and Windows Update forces PP1 to be installed now.  I tried to manually install, and it asked to re-install (which I did anyhow, just in case).  This was a month ago.


    Simple question: Will removing Duplication stop the DEMigrator from going insane?

    http://eduncan911.com
    Friday, October 24, 2008 4:28 AM
  • Are you positive it's duplication that's causing the disk thrashing?  I have quite a few files duplicated (thousands) and don't experience the disk thrashing that you are.  My disks power down and spin back up about once an hour to do whatever (chkdsk presumably).  Albeit, I only have three disk volumes at this time.

    Are you positive you don't have a bad disk & chkdsk is trying to fix those errors by moving stuff around - causing your file conflicts?  Although you should get a warning about a failing drive in those cases.
    Friday, October 24, 2008 7:13 AM
  • eduncan911.com said:

    Thanks everyone for the feedback.  I can handle the raid setup.  ;)

    As for the question if I have PP1 installed - yes.  This is a new server as of about a month ago, and Windows Update forces PP1 to be installed now.  I tried to manually install, and it asked to re-install (which I did anyhow, just in case).  This was a month ago.


    Simple question: Will removing Duplication stop the DEMigrator from going insane?


    http://eduncan911.com



    I haven't experienced any constant activity on my server.  You might want to run chkdsk /r on each drive.  As for your question, removing Duplication will not stop DE.  The only way to keep DE completely out of the equation is to only use one logical hard drive in the server (either one physical drive or hardware RAID).
    Friday, October 24, 2008 10:07 PM
    Moderator
  • I'm going to propose a different source, and solution, for your disk thrashing: Windows Desktop Search. If you have your svn tree in a share, Windows Home Server will use Windows Desktop Search to try to index it for faster searching. (Windows Desktop Search is the engine behind the search feature in Remote Access.) If that's the cause, the solution is to get your repository out of your WHS shares, I'm afraid.
    I'm not on the WHS team, I just post a lot. :)
    • Marked as answer by eduncan911.com Friday, February 19, 2010 9:34 PM
    Saturday, October 25, 2008 1:43 PM
    Moderator
  • Ken Warren said:

    I'm going to propose a different source, and solution, for your disk thrashing: Windows Desktop Search. If you have your svn tree in a share, Windows Home Server will use Windows Desktop Search to try to index it for faster searching. (Windows Desktop Search is the engine behind the search feature in Remote Access.) If that's the cause, the solution is to get your repository out of your WHS shares, I'm afraid.


    I'm not on the WHS team, I just post a lot. :)


    Why dont you just stop the indexing service - i always do it as i never user the search facility, i think as all MS search facilities are whether software or web unfortunately, its ____!!  Google Desktop search does a much better (quieter?) job and you can set it up so that any PC on your network can seach the index using a web browser.
    Saturday, October 25, 2008 5:44 PM
  • Mike, Windows Desktop Search is the engine hooked into the Remote Access web site. There's no way to search when you're accessing your shares from afar if you turn it off...
    I'm not on the WHS team, I just post a lot. :)
    Saturday, October 25, 2008 10:26 PM
    Moderator
  • Ken Warren said:

    I'm going to propose a different source, and solution, for your disk thrashing: Windows Desktop Search. If you have your svn tree in a share, Windows Home Server will use Windows Desktop Search to try to index it for faster searching. (Windows Desktop Search is the engine behind the search feature in Remote Access.) If that's the cause, the solution is to get your repository out of your WHS shares, I'm afraid.


    I'm not on the WHS team, I just post a lot. :)



    Ken:

    That is an excellent idea!  I will kill the indexing service (I have never searched) and see how that does for a while.

    -E
    http://eduncan911.com
    Saturday, November 15, 2008 4:29 AM