locked
Duplication : Is my storage working properly or is it using compression? RRS feed

  • Question

  • I have a concern.  I have copied 280 GB or so to my WHS shares.  About 230 GB has duplication turned ON.  Total should account for 280 GB or so in use.

    The backups of the two laptops I have total 35 GB  (Vista x86) and 50 GB (Vista x64).  I know about the SIS technology WHS is suppose to be using, but I have one 32bit and one 64bit being backed-up, therefore there shouldn't be much of SIS in use since the majority of the files are different.  This should account for another 85 GB in use.

    Now, that's a total of ~360 GB that should be in use on my box before duplication.


     


    As you can see in these photo above, I have a total of 1.36 TB of storage, but only 280 GB is in use?

    In addition, the next photo shows the actual data in use by the shares.





     


    So I am confused.  Is duplication working or not?  The information i read is that Duplication effectively double the amount of space used.  If that is the case, I should only have ~750 GB free.  And that is before the Backups of the Vista machines is taken into account.

    Also note that I copied all of this data last Saturday and the DEMigrator ran like crazy for about a day.  Now, the machine sits nicely, idle, and even spins down the HDDs (when I am not there). 


    Thanks in advance!
    -E
    Eric A. Duncan http://eduncan911.com
    Tuesday, October 7, 2008 5:45 PM

Answers

  • eduncan911.com said:

    I have a concern.  I have copied 280 GB or so to my WHS shares.  About 230 GB has duplication turned ON.  Total should account for 280 GB or so in use.

    The backups of the two laptops I have total 35 GB  (Vista x86) and 50 GB (Vista x64).  I know about the SIS technology WHS is suppose to be using, but I have one 32bit and one 64bit being backed-up, therefore there shouldn't be much of SIS in use since the majority of the files are different.  This should account for another 85 GB in use.

    Now, that's a total of ~360 GB that should be in use on my box before duplication.

    <snip graphic>


    So I am confused.  Is duplication working or not?  The information i read is that Duplication effectively double the amount of space used.  If that is the case, I should only have ~750 GB free.  And that is before the Backups of the Vista machines is taken into account.

    Also note that I copied all of this data last Saturday and the DEMigrator ran like crazy for about a day.  Now, the machine sits nicely, idle, and even spins down the HDDs (when I am not there). 


    Thanks in advance!
    -E


    Eric A. Duncan http://eduncan911.com



    Hello,

    Since you are backing up a x64 machine, I'm going to assume you're running PP1. Can you verify that the version of DE you have installed on the server is 6.0.1800.8? You can do this in the console under settings>resources.

    If the version is not .8, please run Windows Update on your server.

    If you have version .8, please reboot your server or turn duplication off and then back on for the folders you have set for duplication.

    If after you try both of these steps, your server still indicates balanced storage, please submit a bug here. If you are not part of the Windows Home Server program, please join on Connect.

    Thank you
    Lara Jones [MSFT] Windows Home Server Team
    Tuesday, October 7, 2008 7:45 PM
    Moderator
  • eduncan911.com said:

    As you can see in the image below, the suggestions did not resolve the issue.  I am still showing a very small amount of "Duplication".



    Eric,
    Judging from here, to me this does not look right. I would advise you to make sure you have backups of any valuable data on your server. Please also report this problem to Connect here as Lara suggested. Part of this report will be valuable system information/logging that enables MS to determine if there exists a problem and how to resolve it.


    No home server like Home Server
    Wednesday, October 8, 2008 7:29 PM
    Moderator

All replies

  • eduncan911.com said:

    I have a concern.  I have copied 280 GB or so to my WHS shares.  About 230 GB has duplication turned ON.  Total should account for 280 GB or so in use.

    The backups of the two laptops I have total 35 GB  (Vista x86) and 50 GB (Vista x64).  I know about the SIS technology WHS is suppose to be using, but I have one 32bit and one 64bit being backed-up, therefore there shouldn't be much of SIS in use since the majority of the files are different.  This should account for another 85 GB in use.

    Now, that's a total of ~360 GB that should be in use on my box before duplication.

    <snip graphic>


    So I am confused.  Is duplication working or not?  The information i read is that Duplication effectively double the amount of space used.  If that is the case, I should only have ~750 GB free.  And that is before the Backups of the Vista machines is taken into account.

    Also note that I copied all of this data last Saturday and the DEMigrator ran like crazy for about a day.  Now, the machine sits nicely, idle, and even spins down the HDDs (when I am not there). 


    Thanks in advance!
    -E


    Eric A. Duncan http://eduncan911.com



    Hello,

    Since you are backing up a x64 machine, I'm going to assume you're running PP1. Can you verify that the version of DE you have installed on the server is 6.0.1800.8? You can do this in the console under settings>resources.

    If the version is not .8, please run Windows Update on your server.

    If you have version .8, please reboot your server or turn duplication off and then back on for the folders you have set for duplication.

    If after you try both of these steps, your server still indicates balanced storage, please submit a bug here. If you are not part of the Windows Home Server program, please join on Connect.

    Thank you
    Lara Jones [MSFT] Windows Home Server Team
    Tuesday, October 7, 2008 7:45 PM
    Moderator
  • It was a new install Friday, Oct 3rd.  And it has all updates (just checked again), as well as Power Pack 1 (via Windows Updates, I did not manually install it).  DEMigrator version is:

    Windows Home Server Drive Extender: 6.0.1800.8

    I have rebooted the server, and I am attempting to turn off/on duplication on the folders - which I assume I will now have to wait for the DEMigrator.exe to chew up my HDDs for a few days to "balance" the system (I don't like how it really eats away at the HDDs).

    Is that normal practice for copying new files to a duplication folder?!?!  To reboot the server and/or cycle the duplication settings for a folder?  I had all windows updates installed before starting to copy the data.


    Eric A. Duncan http://eduncan911.com
    Wednesday, October 8, 2008 3:32 PM
  • Also note that I explicitly used the \\SERVER\SHARES to copy and move all of the data (yes, i moved a lot around).  I did not use the "D" drive (man, that would have been so much faster).  So I was not by-passing anything.  If this works, then why didn't DE see the changes and handle it?

    Eric A. Duncan http://eduncan911.com
    Wednesday, October 8, 2008 4:50 PM
  • As you can see in the image below, the suggestions did not resolve the issue.  I am still showing a very small amount of "Duplication".

    I also turned Duplication on the Videos shared folder!  Just to see if it would make a big difference (see screenshots above for the 44 GB in use there).  As you can see below, it did not make a big difference for "Duplication".

    Are we sure that WHS isn't using some type of compression?  I did submit a bug per your recommendation as well.


    NOTE: I moved over about 5 GB or so more data, but into a non-Duplication folder.  That is why the numbers are a little different than my previous screenshots.



    Eric A. Duncan http://eduncan911.com
    Wednesday, October 8, 2008 6:45 PM
  • eduncan911.com said:

    As you can see in the image below, the suggestions did not resolve the issue.  I am still showing a very small amount of "Duplication".



    Eric,
    Judging from here, to me this does not look right. I would advise you to make sure you have backups of any valuable data on your server. Please also report this problem to Connect here as Lara suggested. Part of this report will be valuable system information/logging that enables MS to determine if there exists a problem and how to resolve it.


    No home server like Home Server
    Wednesday, October 8, 2008 7:29 PM
    Moderator
  • Thank you Theo.

    Yes, I had already logged the issue now.  As for collecting the additional information you suggested, how would I go about that?

    For backing up my data, I have it on another HDD - just in case.  :)
    Eric A. Duncan http://eduncan911.com
    Wednesday, October 8, 2008 8:06 PM
  • eduncan911.com said:

    Yes, I had already logged the issue now.  As for collecting the additional information you suggested, how would I go about that?



    Eric,
    How to prepare and send this information is explained on Connect. This link explains how to "Obtain a CAB Number using Windows Home Server Toolkit".  Running the procedures explained on your server will automatically send information on your server state (log file, configuration, error events, etc.) to Microsoft and assigning a unique CAB number.  Make sure to a note of these number(s)

    You can then add the CAB numbers obtained to your current report. You might also want to add a link to this thread or maybe some screen prints or other info that you think that could be helpful.

    Good luck,
    Theo.

    No home server like Home Server
    Wednesday, October 8, 2008 8:29 PM
    Moderator
  • Theo van Elsberg said:

    eduncan911.com said:

    As you can see in the image below, the suggestions did not resolve the issue.  I am still showing a very small amount of "Duplication".



    Eric,
    Judging from here, to me this does not look right. I would advise you to make sure you have backups of any valuable data on your server. Please also report this problem to Connect here as Lara suggested. Part of this report will be valuable system information/logging that enables MS to determine if there exists a problem and how to resolve it.


    No home server like Home Server



    FWIW, I agree as well.  I just tallied up my duplicated shares and the amount (110gb) was the same as what was listed in the Server Storage pie chart.

    Of course my system is different than yours.  I only have two drives (system drive & raid array), so duplication occurs on the system drive.

    Is there a possibility the pie chart only lists what is duplicated on the system drive?
    Wednesday, October 8, 2008 8:46 PM
  • Lliam said:


    FWIW, I agree as well.  I just tallied up my duplicated shares and the amount (110gb) was the same as what was listed in the Server Storage pie chart.

    Of course my system is different than yours.  I only have two drives (system drive & raid array), so duplication occurs on the system drive.

    Is there a possibility the pie chart only lists what is duplicated on the system drive?



    Lliam has a good point/question.  What do you think Theo (or others)?

    Fyi, overnight it now shows "51 GB" as what has been Duplicated in the pie-chart.  I did copy 15 GB or so of new data - but it was straight to a non-duplicated folder.


    Eric A. Duncan http://eduncan911.com
    Thursday, October 9, 2008 1:05 PM
  • *bump*
    Eric A. Duncan http://eduncan911.com
    Saturday, October 11, 2008 12:59 AM
  • eduncan911.com said:

    ...] What do you think Theo (or others)? [...


    Hi Eric,
    I think your server is not running as designed. As it is unknown what causes this, I think you have two options:
    1. Send your CAB-reports to MS and wait to see if they can help you find out what is wrong.
    2. Do a new server installation and reload your current data.

    Personally I would go for both ....
    Theo.


    No home server like Home Server
    Saturday, October 11, 2008 3:36 PM
    Moderator
  • Hello all!

    I stumbled upon the issue.  It is a hardware issue.  Funny how WHS did not instruct me otherwise.  This topic is now closed.

    Perhaps WHS could detect about extremely slow disk I/O (see below) and display a warning.  Maybe a "system performance test" on a periodic basis.  It would have pointed me in the right direction.


    (for those that care)

    My mobo has 8 sata ports.  There are 4 sata ports on the Intel ICH7 (with 2 IDE as well), and there are 4 ports on an additional Marvell SATA Controller onboard (Asus P5WD2-E Premium mobo).

    Turns out that the Marvell drivers are horrible.  I spent the weekend installing 3rd party, direct downloads, XP/Vista/2003 versions - nada.

    What lead me down this road was when I connected my other 500gb HDD you see in the screenshots above (not added to the pool) to the Marvell controller.  I wanted more throughput, and thought about getting it off of the "System Driver"'s controller.  When I did this, and started to read/write to that drive - I immediately started seeing sever READ and WRITE delays.  We're talking 100 KB/s, after it bursts for 1 second - and delays any write/read for 5 to 10s.

    Why is this a concern? 

    Intel Southbridge ICH7 Controller

    • 500 GB System Drive (Pri Master)
    • DVD r/w (Pri Master)

    Marvell SATA Controller

    • 500 GB "Pooled" HDD
    • 500 GB "Pooled" HDD
    • 500 GB "Scratch / non-Pooled" HDD


    Since my "storage pool" extended into this Marvell SATA controller, with the sever read/writing issues, the DEMigrator NEVER got to finish (I noticed it WAS slowly increasing, very slowly over the last 10 days - about 500 MB a day).  No matter how much I let it idle for the almost two weeks I've had it up and going.


    Verified Fix
    So for now, I have moved all 4 HDDs to the Intel ICH7 Southbridged controller.  Within an hour, my "Duplication" gave me an file-out-of-sync issue, which makes sense with the previous read/write issues.  Fixed it, and let it go for another hour for another file sync issue.  Fixed it.

    Now, my Duplication folder size matches exactly the size of the folders I have marked as "Duplicate".

    Thank you all for your help.  I will close the bug in the Connect area.


    Eric A. Duncan http://eduncan911.com
    Tuesday, October 14, 2008 7:50 PM
  • Eric,

    Great to hear that you found the problem (and thanks for reporting this back to us!)
    Theo.
    No home server like Home Server
    Saturday, October 18, 2008 8:09 PM
    Moderator