none
Where did my 524GB go? WHS > Server storage > System -- WHS hard drives are LEAKING!!!!

    Diskusi Umum

  • Hello, I have a HP x485.

    I have 2.5GB of hdds (2 hard drives of 1TB and 1 hdd of 500GB)

    - Shared Folders (889GB)

     

    - Duplication (405GB)

    - PC Backups (686GB)

    - System (544GB)

    - Free Space (38GB)

     

    WindirStat report the precise size of each folder

    - d: Shares (818.1GB)

         folders (686GB)

    DE (462.7GB)

    many small folders but tiny space taken....

    - c: 13.9GB

     

    but I can't figure out where the 500+GB from System are? System should be 20GB or less. May be there is a bug in the reporting but the problem is the WHS keeps thinking it has only 38GB available and keep complaining all the time, and it shouldn't as it seems it should have 524GB + 38GB available?

    I try different things like vsadmin delete shadows /all

    but that didn't solve the issue...

    Any one as an idea on how to solve this?

    thanks for your help

    Regards

    Wil.

    Note: How can I post a screenshot on this thread?

     

    • Diedit oleh wil70 07 Desember 2010 0:23
    29 Oktober 2010 23:50

Semua Balasan

  • "System" is the sum of 20 GB (C-Partition) plus everything outside of shares, folders and /DE/ at all disks.

    I use "TreeSize Free" to catch it ...

    30 Oktober 2010 6:40
  • Thanks, pitty I can't paste a picture

     

    I started "TreeSize Free", and basically I got

    +818.6 GB Shares

    +691.6 GB folders

    +488.9 GB DE

    +All the rest is very small like 0.1GB total

    So the total is 1,999.3GB

     

    and my server storage is complaining because I have only 15GB left

    like:

    +Share folders (888 GB)

    +Duplication (405 GB)

    +PC Backups (691 GB)

    +System (563 GB)

    +Free Space (15 GB)

    So the total is 2.5TB

     

    So I'm wondering where are those 563GB and if they are free space why WHS doesn't recognize them and complain that only 15GB is left?

     

    thanks for your help

     

    Wil

    02 Nopember 2010 19:19
  • Did your server crash at any time in the past and you had to perform a recovery? Or did you try to copy data manually within the storage pool locally on the server?  In this case you may have orphaned data on the D: drive on the server under folders\{1618D36B-F4E7-4360-B070-A32070519DC9}.

    If so, verify, if you have the data still somewhere else and clean it out in that folder.

    Or do you run any 3rd party application on the server, which may have stored data on it's own outside of the WHS folder structure?

    Best greetings from Germany
    Olaf

    02 Nopember 2010 22:26
    Moderator
  • You should install the Disk Management for Windows Home Server (Tentacle Software, there is a free trial) there you can see how much of the first HDD is really used.

    Most of the files in shares and folders are tombstones (pointers). They are very small (1 KByte) but reports the real size of the original file.

    03 Nopember 2010 7:15
  • New idea: if you start the Microsoft defrag tool   (only start, not doing any defragmentation) you can see the real empty space of each partition
    03 Nopember 2010 21:52
  • The folders\{1618D36B-F4E7-4360-B070-A32070519DC9} is empty. So I guess it means I have not orphan files? Is there another way to detect orphan? Is there a way after that to fix them?
    I do not have 3rd paty tools that save files outside the WHS file structure.

    Is there a difference when I RDP on the server between moving files around using this path d:\Shares\Music
    versus this path \\Music... that should work the same way isn't it?

    Defrag report (free/capactiy):
    c: 4.72/20GB
    d: 100M/679GB
    d2 1.94/932GB
    d3: 2.72/932GB

    So it seems the hard drives are full (10GB left), which is what the WHS is reporting, but it also report only 2TB of (share, duplicate, backup) and I'm not sure where are the other 500GB reported in system.


    2 things:
    a) can it be usefull to transfert all the share files on a external hard drive, erase every file from share then copy them again?
    or
    b) I have 3 hdd / 4 bay in my WHS, can it be usefull to add a hard drive to the whs and ask to remove 1 of them, then do this again with the other drives, can WHS rebuild or fix this issue? thought I won't be able to do it with the drive that has the OS so that may not work.

     

    05 Nopember 2010 15:54
  • Does having backups of computer I unregister can cause having 544GB reported in System?

    I guess they are reported in  PC Backups (686GB), isnt'it?

     

    thanks for your help

    w

    05 Nopember 2010 15:56
  • Some days ago we discovered a strange behaviour of the WHS if all disks reached the "full" margin ... in this case the WHS stops creating Tombstones but puts the original data to d:\shares\...

    We do not check if this data is counted against System ...

    We found two ways returning the WHS to normal behaviour: turning off duplication to "enough" Shares   or  deleting files (btw. if you decomiss a client, you should delete the backup using the console if you don't need it anay more)

    After that give some time to the WHS and those files will be converted to normal tombstones-based beahviour. Then add more disk space  and change the shares back to duplication.

    06 Nopember 2010 10:21
  • Hello,

    I added an extra 1.5TB + I remove all the duplication, I remove some backups and started the backup cleanup few times,
    now I have 

    - Total space 3.87TB
    - Share (880GB)
    - Backup (536GB)
    - System (1.2TB)
    - Free (1.3TB)

    Should report 2.4TB for Free and 20GB for System.

     

    The folders in d drive are as follow (tested with explorer, windir and TreeSize):

    - Shares (808.6GB)
    - Folders (535.6GB) (everything is mostly in the folder that look like this {000....47AB4}
    - DE\Share (227.3GB
    - DE\folders (145.2GB)(everything is mostly in the folder that look like this {000....47AB4}

    So if I sum up all of this, this is 1.7TB

    Disk defrag (I never defrag the server) report 1.3GB free

    So 1.3GB free reported by defrag + 1.7TB filled is 3TB, still missing 870GB somewhere?

    Disk Managment addin report:

    - Capacity: 3958GB 
    - Used: 2663GB
    - Shadows:       0GB
    - Free: 1295GB

    Conclusion: so many inconcistencies and the issue is WHS rely on what it reported Free, anyhow it seems all the clean up and duplication that I removed (I mean via the WHS console) went into the System. 

    I let the server run for few weeks but nothing seems to change. 

    I know re-installing everything will probably work but that defy the purpose of this WHS HP server.

    Is there a safe rebuild, repair capability?

    How do I fix this? 

     

    Thanks for your help

    Regards

    W.

     

     

    02 Desember 2010 17:41
  • Help, has anybody an idea on how to solve this issue?

    I believe if you remove your share folder duplication you should start seeing the issue appearing (basically all the duplicate go on system and stay there), 

    Thanks

    W

    06 Desember 2010 23:39
  • Are you looking for space advertised when you purchased the server?

    2 TB of storage capacity is only 1818 GiB of usable windows data and then formatting (File System) eats some.  Also some Microsoft OSs create a 100 MB system partition.  And your Page file (RAM temp files saved on hard disk, also called virtual memory) and Shadow Copies may be eating the rest.

    08 Desember 2010 20:02
  • yeah I did clean anything that I can, but simply removing the duplication from a big folder (like photos) will just increase the system folder (instead of really cleaning the hard drive and adding it as free space)

    Concerning the size I have 

    2 * 1TB + 1 * 1.5TB + 1* 750GB = 4.25TB

    Sure, in reality this is 2 * 931GB + 1 * 1.36TB + 718GB = ~ 3.87TB (which is what WHS Console report)

    The issue is not the size of the hard drive, but the size of the System folder should be 20GB only instead of 1.2TB right now, and I should have 2.5TB FREE instead of 1.3TB.

    I think there is a deep bug in WHS that msft need to find, and this bug seem to start when you remove the duplication.

    Thanks for your help

     

    W.

    09 Desember 2010 0:50
  • Hello - I can't believe no body has a solution, microsoft people?,

    I guess I have only one alternative after having spend so much time, I need to backup my data, re-install the OS and re-sell this WHS server.  Too bad, WHS was a nice idea but the implementation seems to have issues.

     

    w.

    15 Desember 2010 16:49
  • Your problem isn't an issue with Windows Home Server per se, I think. If it were, many people (probably most) would have a similar issue if they turn off duplication for something. It could be related to Windows Home Server, however, if you have flaky drivers, or if you've made alterations to your server from the server desktop, rather than through the console.

    Have you installed anything on your server? Add-ins? Software installed from the server desktop (I know you've installed a couple of small utilities to get folder sizes, but anything else)? Have you changed anything at all from the server desktop or software installed from there?

    Can you tell us a little about your hardware?


    I'm not on the WHS team, I just post a lot. :)
    15 Desember 2010 17:55
    Moderator
  • I'm having a very similar issue at the moment. I have 4 x 2TB (1.82 TB) and 1TB system drive. My "system" files is showing 1.6TB!!! I have 1.1TB shared folders with duplication (2.2TB) + 800GB PC backups, so only 3TB of used space, but only shows 3.7TB free, when total system storage is actually ~ 8.2TB of  useable storage, so should show about 5TB free.
    06 Januari 2011 3:03
  • I too have a similar issue.  I currently have four 2TB hard drives in my WHS.  The Console reports a 860GB "System".  I have 4.4TB between Shared Folders and Duplication (2.2TB each), 134GB for PC Backups, 1.9TB Free Space and 860GB for System.

    7.28TB Total Size, but only 1.9TB free due to this very large System partition.  I would be fine with this if I understood where this 860GB was coming from.  I've read many forum posts about the System value having changed from how it was first calculated, but nothing that explains the really huge numbers people are reporting for System.

    D:\folders\{1618D36B-F4E7-4360-B070-A32070519DC9} is also empty on my system.  I'm eager to hear someone explain why this is occurring.

    D:\shares is 2.2TB so that's the actual files as reported by the console.  I do not know where the other 2.2TB in duplicated folders is stored.

    D:\DE\shares is only 22.7GB

    No other folders on my D drive are even larger than a GB.

     

    11 Januari 2011 0:45
  • There was another post here about running in AHCI mode if you have any drives over 1TB. I'm in the middle of chkdsk and formatting my hard drives, and plan on moving to a new case, so will probably be down for another few days, but will let you know how it goes. I am switching to AHCI mode which more or less requires a system reinstallation.

    11 Januari 2011 14:57
  • Hi Vacogroup did you find a solution to the problem? thanks wil

    Is Microsoft looking at the issue?

    22 Januari 2011 1:33
  • There was another post here about running in AHCI mode if you have any drives over 1TB. I'm in the middle of chkdsk and formatting my hard drives, and plan on moving to a new case, so will probably be down for another few days, but will let you know how it goes. I am switching to AHCI mode which more or less requires a system reinstallation.

    Hi MArkBk did you find a solution to the problem? thanks wil

    I'm wondering if Microsoft looking at the issue?
    22 Januari 2011 1:34
  • Same problem on a home built machine. My system shows 57GB for no reason. I have no duplication turn on, all updates and no clue why it says it. I have plenty of free space for now but would like that 37gb back without re-installing. I have built probably 20 whs boxes in the last couple years and never seen this issue. 
    26 Januari 2011 2:38
  • Well, this is weird. Right after I posted, I decided to create a bunch of dummy files using fsutil file createnew dummy.txt 100000000000 and filled up all avail space on my server. When it was full it still showed 57 GB but after I deleted all the dummy files the system allocation went back to 20gb. Then I sat there and waited a while and system went back up to 57gb. 
    26 Januari 2011 2:47
  • Hi Ken

    I myself have just encountered this problem of the system drive expanding to 25% of my total drive space. Total 3.91TB (Shared 2.4TB, PC Backups 62GB, System 1021GB and free space is now 547GB after being down to a paltry 37GB)  

    My problem started when i used the Drive Balancer (as i have used 50+ times before without a problem) For some reason the drives all went up to either 98%or 99% full. Drive Balancer carried on moving files for 3 days which is 2 days longer than any of my previous balances. I ended up cancelling it. Lo and behold.... a system usage of just over 1TB!

    I turned off duplication and removed my pc backups. Doing this has given me 547GB back instead of 37GB which is great! WHS's System still holding 1021GB, is not so great.

    I have read the forums and this is not an uncommon problem. I am disappointed that MS havent investigated both the causes and possible solutions.

    I really want to see a resolution to obtaining this wasted space. Can someone on the WHS team answer with any advice to all the people with this problem?

    Thanks in advance.

    30 Januari 2011 4:40
  • I am now at 65gb for the system, up from 57gb. I guess it is not as bad as some people are reporting but still a mystery.
    30 Januari 2011 5:22
  • I contacted msft and opened a ticket, I also pointed the ticket to this thread so do not hesitate to add details if you have this issue.

    https://connect.microsoft.com/WindowsHomeServer/feedback/details/637504/where-did-my-524gb-go?wa=wsignin1.0#tabs

    There is a button on the top of the ticket that says "User(s) can reproduce this bug: 1", do not hesitate to press the button, I guess that increase the importance for msft.

    thanks

    w


    w.
    31 Januari 2011 2:16
  • The issue on the connect support site has been "resolved - wont fix".  No reason given in the comments.
    06 Agustus 2011 23:29
  • Yeah I expected this from msft so 3 months I sold my server and whs license on amazon!

    w

     


    w.
    07 Agustus 2011 2:55
  • I had "system" space varying between 200 and 500GB on a 10TB system over the last year, and I just ran chkdsk on all my drives and it's back to 20GB.
    07 Maret 2012 17:15
  • I have a similar problem also, but to a HUGE degree.  I have 6 x 2TB hard drives for which equals just under 11 GB of useable storage.  My system partition is reporting as 3.9TB in size!  This is roughly twice as large as any other "piece of the pie". 

    When I turned off duplication, it jumped to 5.4TB, so I turned duplication back on.

    I have ran chkdsk via a command prompt script twice which took a couple of days each and tried everything above to no avail.

    Most recently, I have reinstalled the server.  At first, it was great, back to 20GB system and over 6 GB (over 50%) of free space.

    But over-night, the system volume returned to it's pre-server-reinstallation size of 3.9TB.

    I have been banging my head trying to figure out what caused it to jump in size.  I have been running it for over two years with roughly the same amount data and never had this issue until recently.  The only two things that I can think of that have happened since the system size was "normal" and now that could account for the problem are)

    1.  I had a corrupt computer backup database, which I had to delete the contents of.  All seemed to go fine with this procedure.  Backups were again created without issue.

    2.  I created a new share and moved about 500GB of data from another share to it.

    I am not sure if either of these two things had anything to do with my problem or not.

    But since a total reinstall has not resolved the problem, I am at a complete loss as to how to fix it and so I am posting in hopes that someone can help me and the others having this issue out.  Thank you for your time and attention.

    Regards,

    JM  

    10 Maret 2012 19:24
  • it's like a contageous disease, and now my WHS is having the same problem, after running WHS peacefully for the last three years.  Does this have anything to do with some recent WHS update? Can't believe that Microsoft left this serious problem as Wont'fix!!  Coming on Microsoft, the name needs some work to keep clean.  -Jin
    06 April 2012 3:48
  • Add me to the list of people who have seen a sudden surge in System partition size. Here are the events at my end:

    I had 4 drives with a total capacity of 5.23Tb of which System accounted for 23Gb.

    I added a 5th drive (3tb) and total capacity increased to 7.96Tb, less than what I expected, because System now accounts for 773Gb - an increase of 750Gb!!

    I've not installed any new software and not added any data. I just added the one drive. Weird, huh?

    28 Juni 2012 3:23
  • How to find the space..... :)

    My piece of the pie chart showing SYSTEM space used
    ( seen on the WHS 'Server Storage' pie chart) was 913GB
    after a reinstall (which had a couple crashes happen in the
    middle due to driver issues). I couldn't find a definitive answer
    to why this happened, or how to fix it.  So I researched as much
    as I could, and came up with the following things to check.

    From my research, it can consist of one or more of these 4 things:
    1) Duplicated Backups
    2) "Previous Versions" of files in the shared folders
    3) Orphan files
    4) Files manipulated by copying directly to the volumes
       using 'my computer' or 'explorer' INSTEAD of using the \\server\share\
       folders like we're supposed to.

    ==============================================================

    1) Space from Duplicated BACKUPS (these are only enabled if the
    registry hack described here
    ( http://www.hanselman.com/blog/WindowsHomeServerUnsupportedFeatureBackupDuplication.aspx )
    is enabled:

    As stated in the above link:

    Totally unsupported and may well go away in the future no warranty
    express or implied you're on your own don't ask for support from
    anyone it's your butt not mine Registry key from Regedit:
     
    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Home Server\Storage Manager\Folders
     
    From a list of GUIDs, find the one that has the FriendlyName
    of "Windows Home Server Computer Backup." Under that key there
    is an "Attributes" sub-key. Create a DWORD value "Reliable" = 1.
     
    (If these instructions are too technical or don't make sense,
    you might want to reconsider heading down this path, or you can
    find a nerd to do this. I don't want to make this too easy because
    if you do this unsupported thing, I'd hate to have your house burst
    into flames because if it.)
     
    Setting this value does one simple thing,
    it turns on duplication of the backups database.

    ==============================================================

    2) Space from 'Previous Versions' of files in the \\server\
       shares directories.  This was enabled in the original install
       of WHS from the very first OEM CD.  The Microsoft Whitepaper
       on Drive Extender ("Windows Home Server Technical Brief for
       Drive Extender" currently found at
       http://www.microsoft.com/en-us/download/details.aspx?id=23876)
       explains the issue:

       "Disabling Volume ShadowCopy Services
    The initial release of Windows Home Server used Volume ShadowCopy Services to support Previous Versions functionality for files stored in Shared Folders on the home server. Windows Home Server triggered a snapshot every 12 hours, enabling users to access previous versions of files stored in Shared Folders on a home server, through the Previous Versions functionality available in some Windows client operating system editions.
    The new media editions of Windows Home Server Power Pack 1 ship with Volume ShadowCopy Services for home server Shared Folders turned off.  If you purchased the initial version of Windows Home Server and upgraded to Power Pack 1, then this feature is still turned on and may take up considerable storage space on your home server.  The amount of space taken up by ‘System’ in the Server Storage pie chart includes the space used by volume shadow copies, which in some cases can be significant.

      Disabling Volume ShadowCopy Services and reclaiming the disk space

    1. Run mstsc.exe to start a Remote Desktop Connection session to your home server.
     Caution
    Be careful when using a Remote Desktop Connection to your home server. You can damage Windows Home Server functionality if you use it incorrectly.

    2. Open a Command Prompt, Click Start, Run and type CMD
    3. To delete all of the existing volume shadow copy snapshots and reclaim the disk space used by the snapshots, type vssadmin delete shadows /all
    4. To disable the Windows Home Server storage manager service from taking future snapshots, type regedit
    5. Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Home Server\Storage Manager\Volumes.
    6. Find a sub key which has a MountPoint value equal to D:
    7. Under that key set the value of SnapPeriod to 0

    ==============================================================

    3) Orphan Files occur when there has been a problem.  From the Whitepaper:
    Viewing Orphan Shadow Files
    Windows Home Server with Power Pack 1 proactively searches for orphan shadows on home server hard drives, and places them in an application folder on the primary data partition.  Orphan shadows could get created when hard drives appeared as ‘missing’ on home server systems running the initial release of Windows Home Server.  Orphan shadows are secondary copies of files stored in shared folders and in certain circumstances these shadow files may not have been properly cleaned up when hard drives went ‘missing’. 

    During the installation of the Power Pack 1 update to a home server running the initial release of Windows Home Server, the hard drives are analyzed to look for any orphan shadow files.   If any orphan files are found they are stored in an application folder in the following location - D:\folders\{1618D36B-F4E7-4360-B070-A32070519DC9}\

    The special location above will maintain the folder structure of the original file which allows the user to understand in which Shared Folder the original file was stored.

      To view any orphan shadow files

    1. Run mstsc.exe to start a Remote Desktop Connection session to your home server.
     Caution
    Be careful when using a Remote Desktop Connection to your home server. You can damage Windows Home Server functionality if you use it incorrectly.

    2. Open Windows Explorer and navigate to D:\folders\{1618D36B-F4E7-4360-B070-A32070519DC9}\

    These files can be deleted.

    ==============================================================

    4) Copying files directly to disk.

    See the Whitepaper for more info.  In a nutshell, if you copy anything to the disks using explorer or 'my computer' instead
    of the shares, it becomes part of the 'system' space reported by the WHS Server Storage pie chart.

    ==============================================================

    That's it.  I'm posting this here in an attempt to help anyone out who in the future might have this annoying and mysterious 'System' piece of the pie chart to be much huger than the normal 20GB that we usually expect.

    Regards,

    Nem

    • Diedit oleh Nem1234 25 Agustus 2012 16:28
    25 Agustus 2012 16:22
  • Hello!

    Old post, same problem, simple solution (but a few days of work - depending on the HDDs size):

    1. One of the HDDs had a bad block

    2. After switching broked HDD with a new one, Event viewer shows file inconsistency. I tried:for  /d  %1  in  (c:\fs\*)  do  chkdsk  /x  /r  %1, but System in pie chart went from 20GB to 867GB (not at once, but in a few steps!)

    3. At the end, I erased almost all backups but the last one, disabled data duplication, removed files from server to external disks, then removed all HDDs in theServer Storage, except primary (just remove, not physically disconnect), reboot WHS and Add HDDs again. Then copy, copy, copy!

    System partition is again 20GB in size and everything is normal.

    Looking forward to Windows Server 2012 Essentials. Drive Extender is dead, viva la Storage Spaces & new candys, which it will bring.

    Regards, DC

    17 Oktober 2012 15:43