locked
Why does system take more than 20GB on server storage tab pie chart? RRS feed

  • Question

  • First of all, if the answer to this specific question is posted somewhere, please point me to it.  I had many hits when searching on variations of system storage tab wrong large etc, but nothing that addressed my specific situation.

    In my sever storage tab, on the pie chart, a whopping 310GB is allocated to system.  How can this be?

    I believe in the past this did not show more than 20GB.

    My system partition is just 20GB. This is a HUGE discrepancy.  What else would be included in "system"?

    I recently did a complete restore re-install of my WHS after replacing a faulty system drive.

    Some threads with regards to size calculation errors (in the FAQ) referred to a system size calculation that never completes. however, this is not a problem in my scenario.
    Wednesday, July 22, 2009 9:26 PM

All replies

  • First of all, if the answer to this specific question is posted somewhere, please point me to it.  I had many hits when searching on variations of system storage tab wrong large etc, but nothing that addressed my specific situation.

    In my sever storage tab, on the pie chart, a whopping 310GB is allocated to system.  How can this be?

    I believe in the past this did not show more than 20GB.

    My system partition is just 20GB. This is a HUGE discrepancy.  What else would be included in "system"?

    I recently did a complete restore re-install of my WHS after replacing a faulty system drive.

    Some threads with regards to size calculation errors (in the FAQ) referred to a system size calculation that never completes. however, this is not a problem in my scenario.

    Any files stored on any hard drive that is part of the storage pool, but not in a WHS share, ends up in SYSTEM.  For example, if you are using your server to download stuff and not saving it to a WHS share, it will show as SYSTEM.  Also, Shadow Copies (which doesn't work correctly on WHS) is counted as SYSTEM files.
    • Proposed as answer by kariya21Moderator Wednesday, July 22, 2009 9:36 PM
    • Marked as answer by dpkform Thursday, July 23, 2009 2:06 AM
    • Unmarked as answer by dpkform Thursday, July 23, 2009 4:11 AM
    • Unproposed as answer by dpkform Thursday, July 23, 2009 4:12 AM
    Wednesday, July 22, 2009 9:34 PM
    Moderator
  • thanks Kariya

    okay - so if I understand this correctly - files downloaded directly to my "d" partition, and not into a share, could count towards this system total

    a quick check of my "d" partition shows not nearly enough files to contribute to a 310GB system total

    I guess that leaves shadow copies..... is there a way to determine what amount of space shadow copies are using and to delete them, and prevent them from re-curring on WHS - especially if as you say they don't work correctly in WHS...

    edit: nevermind - now that I just did a search on "shadow copies" the very first result took me to a post where you already answered my question for someone else...... too bad this post did not show up when I searched......

    http://social.microsoft.com/Forums/en-US/whssoftware/thread/8dcc1f5d-7000-4030-858a-667f3d0c4904


    Thursday, July 23, 2009 2:10 AM
  • hmm - I have been doing a little more digging..... things still do not add up..... there still seems to be a lot missing

    I have turned on folder options to show all files, including hidden and operating system files.....

    so checking the drive properties, shadow copy tab, I have on all volumes combined, just under 2GB of shadow copies

    All the files combined on my "c" partition take up 6.95GB on disk.

    I have not added any files directly to my D drive that I can see.

    Using the normal "properties" tab for drive D says I have only 877GB in use.

    The total storage size for all my drives is 1.78 TB - this still does not even come close to adding up.

    ?????

    My PC backup database was corrupted after my system hard drive crashed, and I restored a former PC backup , backup from an external backup drive using the BDBB add-in.  Hmm - could a corrupted, "deleted" yet not yet "cleaned up" former backup database, account for space that shows as unused in file properties, yet shows up in the system total on the server storage tab?

    edit: nope - my server just finished a manually initiated database backup cleanup (let it run for a couple hours) - and now...... my system use has not decreased at all - in fact it slightly INCREASED from 310GB to 318 GB?????

    Kariya - hope you don't mind I unchecked your reply as answer -  because this question now does not seem resolved... let's see if we can get to the cause...

    any further thoughts?
    Thursday, July 23, 2009 2:32 AM
  • The generic answer is that the "system" wedge in the pie chart is "All used disk space not covered by one of the other wedges". Microsoft hasn't revealed the exact algorithm to calculate this for yourself, but if you go through manually and identify all of the items that have their own wedge, and subtract that from the disk space used on all the drives your server has in the storage pool (ignore D:\Shares and D:\Folders if you have multiple drives...), probably you'll come up with the source of the discrepancy. Also realize that the System partition takes up 20 GB of "system" space, whether all of the system partition is full or not.

    In addition, add-ins that use significant additional storage for various purposes could conceivably inflate the "system" wedge. So it's possible that e.g. turning on duplication of the backup database (not supported inbox in Windows Home Server) will cause the second copy to inflate "system" usage. (I don't know if this is the case; I duplicated my backup database a couple of years ago as an experiment, but it's not that way now on any of my servers...)

    tl;dr version: you're worrying about a detail that probably makes absolutely no difference in the functionality of Windows Home Server, and intetests you only because you want to tweak and explore. :)

    I'm not on the WHS team, I just post a lot. :)
    Thursday, July 23, 2009 3:01 PM
    Moderator
  • This part of WHS is (IMHO) flawed by design. 

    When trying to copy files from a client PC on WHS RTM install (no Power packs) you could run into disk space error because the operating system reported only free space on D drive. So even if when there was more than enough space on the data disks the client system would not copy anything larger then the free space reported for D drive. This was then "fixed" (I think already in PP1) in such a way that all of free space is now reported to clients. As a result the properties tab for D drive now displays some very odd values. 

    Also in console it doesn't report all of D drive. I think it reports total of D:\DE, D:\shares, D:\folders and shadow copies.

    I think the disk management Add-in from Sam Wood is currently the only way to get correct values. It will also show the amount of space used by shadow copies.
    Thursday, July 23, 2009 3:05 PM
    Moderator
  • This part of WHS is (IMHO) flawed by design. 

    When trying to copy files from a client PC on WHS RTM install (no Power packs) you could run into disk space error because the operating system reported only free space on D drive. So even if when there was more than enough space on the data disks the client system would not copy anything larger then the free space reported for D drive. This was then "fixed" (I think already in PP1) in such a way that all of free space is now reported to clients. As a result the properties tab for D drive now displays some very odd values. 

    Also in console it doesn't report all of D drive. I think it reports total of D:\DE, D:\shares, D:\folders and shadow copies.

    I think the disk management Add-in from Sam Wood is currently the only way to get correct values. It will also show the amount of space used by shadow copies.

    Thanks brubber, ken, kariya,

    Still - things just don't add up from anything I have heard so far, unless I am misunderstanding something

    1) I DO have Sam's disk management Add-in installed. It reports the same total free space available, even summing it drive by drive, the free space total is the same as reported by WHS storage tab

    2) My shares take up 360GB - this is the same value reported by WHS storage tab, and by RDP into WHS, opening "D" drive, and selecting "properties" of the share folder

    3) Duplication is reported by WHS storage tab to take up 355GB, and this makes sense, as I have ALL but one miscellaneous share folder set to be duplicated

    4) I did a manual restore reinstall just a couple days ago, after having to replace a failed 500GB system drive, and the system then immediately updated itself all the way to PP2, before I began using WHS again, and transferring files

    5) Although I used Ken's command to clear up old shadow copies, a few new shadow copies have crept back as expected, and currently both Sam's disk management add-in, and the shadow copies tab of the  "d" drive properties from an RDP desktop login, both report 1.7GB total allocated to shadow copies (which are disabled on the property tab, but I have not used the regedit command to forcefully disable shadow copies)

    6) My backup database was corrupted after the re-install, it could not be recovered by backup database repair.

    7) The old backup database which I restored with BDBB from an external backup, shows on the external backup drive as 215 GB.  WHS storage tab reports the restored backup database as taking up 215GB.  I have NOT performed any new backups, since restoring the BDBB.  I did perform a manual backup cleanup function after restoring the BDB using BDBB.

    Is it possible that the original BDB was NOT properly deleted, even though BDBB claimed it would be, when I restored my BDB? If so, how would I check?

    8) My system drive as Ken suggested will count as 20GB, even though I currently only have 7GB of data on it.

    9) The total of the drives I have allocated to the storage pool (does not include my 1TB backup drive) is 500GB + 640GB +500 GB + 160GB + 160GB = 1.78 TB, which is the total storage value shown on the WHS storage tab.

    10) Looking manually into the "D" folder, here is a list of all root folders (folder properties are set to show hidden files, and operating system files)
         "folders" folder reports 215GB - I assume it contains the PC backups
         "shares" folder reports 360GB
         three folders which cryptic numbers and letters names, report a total of 8MB (0.008 GB)
         "system volume information" of course reports "empty"
         "DE" folder reports 311GB - this seems like an odd number - it is less than what would be the size of duplicated shares..... perhaps here lies the clue?
                inside the DE folder, the "shares" folder reports 300GB useage
                inside the DE folder, the "folders" folder reports 11GB
         recycler is the only other root folder on the D drive, it reports less than 1GB (785 MB) in use

    Does this provide any insight?  Without reasonably accurate free space storage reporting, it is impossible to know when new drives need to be added, or if the space on existing drives is somehow being wasted.

    Many thanks again for your helpful analysis.  I'll save the WHS rants for now - I'm beginning to accept that WHS is a useful, but extremely flawed product :-)



    Friday, July 24, 2009 4:56 AM
  • I agree figures don't add up. Looking in detail on my own system I also get different values depending on the method used. Actually I think the Add-in from Sam also doesn't get the correct value. Personally I think things got messed up with PP1. Before that figures would always add-up in a logical and understandable manner.

    I disagree WHS is extremely flawed. For most users the system works as it should and I still like it a lot, even though there's issues like this one that should (IMHO) be solved. Perhaps you can file a bug report on Windows Home Server Connect Feedback site and post link overhere so other can vote on it.
    Friday, July 24, 2009 9:30 AM
    Moderator
  • This part of WHS is (IMHO) flawed by design. 

    When trying to copy files from a client PC on WHS RTM install (no Power packs) you could run into disk space error because the operating system reported only free space on D drive. So even if when there was more than enough space on the data disks the client system would not copy anything larger then the free space reported for D drive. This was then "fixed" (I think already in PP1) in such a way that all of free space is now reported to clients. As a result the properties tab for D drive now displays some very odd values. 

    Also in console it doesn't report all of D drive. I think it reports total of D:\DE, D:\shares, D:\folders and shadow copies.

    I think the disk management Add-in from Sam Wood is currently the only way to get correct values. It will also show the amount of space used by shadow copies.

    Thanks brubber, ken, kariya,

    Still - things just don't add up from anything I have heard so far, unless I am misunderstanding something

    1) I DO have Sam's disk management Add-in installed. It reports the same total free space available, even summing it drive by drive, the free space total is the same as reported by WHS storage tab

    2) My shares take up 360GB - this is the same value reported by WHS storage tab, and by RDP into WHS, opening "D" drive, and selecting "properties" of the share folder

    3) Duplication is reported by WHS storage tab to take up 355GB, and this makes sense, as I have ALL but one miscellaneous share folder set to be duplicated

    4) I did a manual restore reinstall just a couple days ago, after having to replace a failed 500GB system drive, and the system then immediately updated itself all the way to PP2, before I began using WHS again, and transferring files

    5) Although I used Ken's command to clear up old shadow copies, a few new shadow copies have crept back as expected, and currently both Sam's disk management add-in, and the shadow copies tab of the  "d" drive properties from an RDP desktop login, both report 1.7GB total allocated to shadow copies (which are disabled on the property tab, but I have not used the regedit command to forcefully disable shadow copies)

    6) My backup database was corrupted after the re-install, it could not be recovered by backup database repair.

    7) The old backup database which I restored with BDBB from an external backup, shows on the external backup drive as 215 GB.  WHS storage tab reports the restored backup database as taking up 215GB.  I have NOT performed any new backups, since restoring the BDBB.  I did perform a manual backup cleanup function after restoring the BDB using BDBB.

    Is it possible that the original BDB was NOT properly deleted, even though BDBB claimed it would be, when I restored my BDB? If so, how would I check?

    It's possible.  You could try contacting the author here and see if he can provide any insight.

    8) My system drive as Ken suggested will count as 20GB, even though I currently only have 7GB of data on it.

    9) The total of the drives I have allocated to the storage pool (does not include my 1TB backup drive) is 500GB + 640GB +500 GB + 160GB + 160GB = 1.78 TB, which is the total storage value shown on the WHS storage tab.

    10) Looking manually into the "D" folder, here is a list of all root folders (folder properties are set to show hidden files, and operating system files)
         "folders" folder reports 215GB - I assume it contains the PC backups
         "shares" folder reports 360GB
         three folders which cryptic numbers and letters names, report a total of 8MB (0.008 GB)
         "system volume information" of course reports "empty"
         "DE" folder reports 311GB - this seems like an odd number - it is less than what would be the size of duplicated shares..... perhaps here lies the clue?
                inside the DE folder, the "shares" folder reports 300GB useage
                inside the DE folder, the "folders" folder reports 11GB
         recycler is the only other root folder on the D drive, it reports less than 1GB (785 MB) in use

    Does this provide any insight?  Without reasonably accurate free space storage reporting, it is impossible to know when new drives need to be added, or if the space on existing drives is somehow being wasted.

    Many thanks again for your helpful analysis.  I'll save the WHS rants for now - I'm beginning to accept that WHS is a useful, but extremely flawed product :-)




    As Ken said, it's not really an issue.  Mine is at 27 GB (which is exactly what it should be with Shadows running).  However, if you really want to track it down, I believe it can be done.  First, how much does it show in SYSTEM now?  Second, are you downloading/copying anything to your server anywhere other than a share that WHS is aware of (meaning a share that shows up in the Console)?  How many hard drives do you have in your server?  How much is each drive using for Shadow Copies (not just the D partition)?  Are you still using the BDBB add-in and keeping the backup somewhere on your server?
    Friday, July 24, 2009 11:48 PM
    Moderator
  • As Ken said, it's not really an issue.  Mine is at 27 GB (which is exactly what it should be with Shadows running).  However, if you really want to track it down, I believe it can be done.  First, how much does it show in SYSTEM now?  Second, are you downloading/copying anything to your server anywhere other than a share that WHS is aware of (meaning a share that shows up in the Console)?  How many hard drives do you have in your server?  How much is each drive using for Shadow Copies (not just the D partition)?  Are you still using the BDBB add-in and keeping the backup somewhere on your server?
    Thanks again to everyone who has been chipping into this discussion.

    IMHO 300GB of unaccounted for storage space, that should be available for use is a HUGE issue, just one small notch below data corruption.  We're not talking about a couple GB here - nearly 300 GB, with no apparent work-around to recover that missing space, because there is still no suitable explanation for why that space is missing.  In  tech support criteria, is this arguably just one step below a severity 1 bug.

    If you check my last post, it is very detailed, and answers all your questions.  It is impossible to account for the massive discrepancy, and I have itemized this drive by drive, even folder by folder.  My folder options are set to shown hidden and operating system files.

    I have filed report ID number 477086  on Microsoft connect.
    Saturday, July 25, 2009 5:27 AM
  • I'm afraid you have not itemized drive by drive. You have 5 physical drives in your server's storage pool, but you have looked (per your previous posts) at only the two partitions on the system drive. As I said previously, if you have more than a single drive in your server, you need to disregard anything you find in D:\Shares and D:\Folders. Those are only tombstones; your actual files are elsewhere (on other physical drives).

    The Windows Home Server team has, obviously, spent no time on trying to make it easy to find out where/how space on any given drive is being used, or to calculate (by hand) the usage that the pie chart displays. I honestly see no reason for them to do so; by design all access to your server should be through the published shares (you have a shortcut Shared Folders on Server on the desktop of every computer joined to your server), the Windows Home Server Console, and the Remote Access web site. Anything you do or see through Remote Desktop or physical console is unsupported, and you shouldn't expect that the Windows Home Server team will have taken care that everything will look just like any other computer.

    All that said, I agree with you that better information on what is consuming "system" space would occasionally be useful, along with tools for recovering that space. This belongs in the console and probably will not happen in an update to V1 of Windows Home Server (too much work).

    I'm not on the WHS team, I just post a lot. :)
    Saturday, July 25, 2009 1:34 PM
    Moderator
  • I'm afraid you have not itemized drive by drive. You have 5 physical drives in your server's storage pool, but you have looked (per your previous posts) at only the two partitions on the system drive. As I said previously, if you have more than a single drive in your server, you need to disregard anything you find in D:\Shares and D:\Folders. Those are only tombstones; your actual files are elsewhere (on other physical drives).

    The Windows Home Server team has, obviously, spent no time on trying to make it easy to find out where/how space on any given drive is being used, or to calculate (by hand) the usage that the pie chart displays. I honestly see no reason for them to do so; by design all access to your server should be through the published shares (you have a shortcut Shared Folders on Server on the desktop of every computer joined to your server), the Windows Home Server Console, and the Remote Access web site. Anything you do or see through Remote Desktop or physical console is unsupported, and you shouldn't expect that the Windows Home Server team will have taken care that everything will look just like any other computer.

    All that said, I agree with you that better information on what is consuming "system" space would occasionally be useful, along with tools for recovering that space. This belongs in the console and probably will not happen in an update to V1 of Windows Home Server (too much work).

    I'm not on the WHS team, I just post a lot. :)

    The problem here Ken, is that before my WHS re-install, my system usage was understandably reported at a steady 20GB.  Suddenly, it jumps to 310GB, and my available free space takes a nosedive, despite the fact I replaced the defective 500GB drive with another 500GB drive.  Given that I actually lost a few non-duplicated files on the defective drive, the end result should be that my total free space, should actually have increased.

    So while I think I understand what you are saying, that the "D" partition folders themselves can not be relied on to calculate free and available space, it is the only method available to try an understand what went wrong.  I went into each and every folder in the "D" drive simply to ensure that no manuall created folder was in those folders. All data on my server has been placed there through WHS shares, or through WHS PC backups. There is no unsupported use occurring.

    I did one more thing.  As a user with full access to every WHS shared folder, I went into each WHS shared folder, selected all, then properties, to get a total size of data on disk in each folder.  When I summed this up, it was exactly the same amount as WHS was reporting for folder shares.  Duplication is also correctly reported, when I deduct the size of the only folder not set to be duplicated.  I am trusting the amount of space reported for PC backups, as it also matches the size of my PC backups folder on my external backup drive.  That leaves the only other valid use as system - which should of course be shown as 20GB of reserved space. Yet nearly 300GB of what should be available free space is completely unaccounted for.  That means at some point WHS will force me to purchase a new drive, when I should still have 300GB of available storage.

    If you or another person that a moderator can vouch for wants to do a little digging on my system, I would be willing to temporarily grant remote RDP access to my WHS. I'm completely at a loss, as to what to do next, and it seems my trying to explain this on forms is not providing any useful troubleshooting information.








    • Edited by dpkform Sunday, July 26, 2009 9:22 AM add more info WHS shares
    Sunday, July 26, 2009 8:46 AM
  • Just an update, I have done a clean install of WHS, after deleting the partitions on my WHS drives.  After restoring all the files, and my computer backups using BDBB, and turning duplication back on, my system use on the storage tab continues to be correctly reported as 20GB + any shadow copies.

    In fact, I re-installed twice.  The first re-install I had 3 drives, and once I restored my PC backups,  that data spilled onto the system drive.

    I wiped things again, replaced the system drive with a 320GB drive from a 1.5 year old PC, and added an additional 500GB drive.  So 4 x 500GB storage and 1 x 320GB system. I decided to keep data off the system drive, by providing enough storage on data drives.

    After both clean installs, the system usage and other totals continue to be reported correctly.

    I am now wondering if the original mess was triggered by the failure of my former system hard dive.  In the recent past WHS would lock up, data inaccessible, and console could not connect.  I would then log in directly (I have the WHS machine attached to the second connector on the monitor for my desktop PC). I would simply reboot - and then everything would be fine for a while.  Things then got worse. When shutting down I would get messages about files that could not be written and data that may have been lost. I was not too worried as my important files all had duplication turned on, and I had external backup. But I quickly realized my system drive was bad. I did a "system restore" install with a new hard drive.  The server once again ran reliably, but I later noticed that "system" use was reporting around 300GB.

    As mentioned above, once I wiped all the data, did a clean install of WHS, and then added back my data from my external backup, all was then well.

    So perhaps this is a case of server re-installation not working correctly, after the former system drive caused errors and was eventually replaced? Maybe there is some sort of bug with server re installation?
    Friday, August 7, 2009 6:11 AM