locked
1.4 TB free, but running out of disk space when ripping a 37 GB ISO to WHS. RRS feed

  • Question

  •  

    First of all, let me say that I have been searching for the past 3 days for a solution to my problem.  I can't even find anyone that has the same issue.  Maybe my case is unique.

     

    I have been ripping my entire Bluray and HD-DVD library to my WHS without any problems.  Now I am trying to rip Batman Begins which is a 37 GB disc.  I have 4 TB of disk space on my WHS with 1.4 TB free.

     

    Here is how my disk config looks:

     

    Disk0  698 GB  94% Used

    Disk1  698 GB  97% Used

    Disk2  465 GB  93% Used

    Disk3  372 GB  70% Used (System Drive...why is the data not being copied off the D: drive???)

    Disk4  465 GB  95% Used

    Disk5  698 GB    0% Used

    Disk6  698 GB  24% Used

     

    When ripping Batman Begins it always tries to dump it to Disk1 which only has about 20 GB free...which of course fills up a little over halfway into ripping Batman Begins.  Why is WHS deciding to use this drive when it is almost full?

     

    How can I force the system to better balance my storage?

     

    Also, my D: drive normally has about 350 GB of free space, but currently only has around 100 GB.  It has always copied the data off of the D: drive correctly, but now it's not.

     

    Sorry for the long post!  Hopefully someone can help me find a solution to my problem!

    Tuesday, July 15, 2008 3:15 PM

All replies

  •  Tim Lastoria wrote:

     

    First of all, let me say that I have been searching for the past 3 days for a solution to my problem.  I can't even find anyone that has the same issue.  Maybe my case is unique.

     

    I have been ripping my entire Bluray and HD-DVD library to my WHS without any problems.  Now I am trying to rip Batman Begins which is a 37 GB disc.  I have 4 TB of disk space on my WHS with 1.4 TB free.

     

    Here is how my disk config looks:

     

    Disk0  698 GB  94% Used

    Disk1  698 GB  97% Used

    Disk2  465 GB  93% Used

    Disk3  372 GB  70% Used (System Drive...why is the data not being copied off the D: drive???)

    Disk4  465 GB  95% Used

    Disk5  698 GB    0% Used

    Disk6  698 GB  24% Used

     

    When ripping Batman Begins it always tries to dump it to Disk1 which only has about 20 GB free...which of course fills up a little over halfway into ripping Batman Begins.  Why is WHS deciding to use this drive when it is almost full?

     

    How can I force the system to better balance my storage?

     

    Also, my D: drive normally has about 350 GB of free space, but currently only has around 100 GB.  It has always copied the data off of the D: drive correctly, but now it's not.

     

    Sorry for the long post!  Hopefully someone can help me find a solution to my problem!

     

    Have you installed PP1 beta?  Can you logon to the server desktop and see if there is a process called DEMigrator.exe in the Task Manager?  (NOTE:  In order to see if DEMigrator.exe is running, make sure Show processes from all users is checked.)

    Tuesday, July 15, 2008 3:35 PM
    Moderator
  •  kariya21 wrote:
    Have you installed PP1 beta?  Can you logon to the server desktop and see if there is a process called DEMigrator.exe in the Task Manager?  (NOTE:  In order to see if DEMigrator.exe is running, make sure Show processes from all users is checked.)

     

    I am running PP1 RC4.  I do show the demigrator.exe process running.  CPU usage is 0% and memory usage is 14,252K.

    Tuesday, July 15, 2008 3:43 PM
  •  Tim Lastoria wrote:
    I am running PP1 RC4.  I do show the demigrator.exe process running.  CPU usage is 0% and memory usage is 14,252K.

     

    Something is not right.  You should still have ample space for 37 GB of data on your primary drive.  You said it's trying to copy to Disk 1.  How do you know this?

    Tuesday, July 15, 2008 3:50 PM
    Moderator
  •  kariya21 wrote:
    Something is not right.  You should still have ample space for 37 GB of data on your primary drive.  You said it's trying to copy to Disk 1.  How do you know this?

     

    I am running the Disk Management add-in which allows me to monitor disk activity.  When I begin to rip the Bluray disc, the only disk that shows activity (about 10,000 KB/sec) is Disk1.

    Tuesday, July 15, 2008 4:24 PM
  •  Tim Lastoria wrote:
    I am running the Disk Management add-in which allows me to monitor disk activity.  When I begin to rip the Bluray disc, the only disk that shows activity (about 10,000 KB/sec) is Disk1.

     

    Do you ever get an error message?  (Like not enough space available?)

     

    One other thought: Did you start with only a couple hard drives, install PP1 beta, then add more drives after installing PP1 beta?

    Tuesday, July 15, 2008 4:28 PM
    Moderator
  •  kariya21 wrote:
    Do you ever get an error message?  (Like not enough space available?)

     

    One other thought: Did you start with only a couple hard drives, install PP1 beta, then add more drives after installing PP1 beta?

     

    I do get the message from the ripping program (IMGBURN) about 60% through the rip saying that I ran out of disc space...and in fact Drive1 shows 100% full.

     

    The only drives I added after the PP1 beta were Disk5 and Disk6.

    Tuesday, July 15, 2008 4:34 PM
  •  Tim Lastoria wrote:
    I do get the message from the ripping program (IMGBURN) about 60% through the rip saying that I ran out of disc space...and in fact Drive1 shows 100% full.

     

    You might try ripping it to your local drive first, then copy it to the server afterwards.  (Perhaps IMGBURN is going "through" the share to first drive it sees.)

     

     Tim Lastoria wrote:
    The only drives I added after the PP1 beta were Disk5 and Disk6.

     

    Yep, that's what I guessed when I saw your numbers. Smile

     

    You might want to uninstall PP1 beta.  Once you do, it will free up the space on your primary drive.  (There is a known issue with data migration in PP1.  Please read my post, Drive Extender Not Migrating Data, in the PP1 beta forum.)

     

    However, I would also suggest that while the data is being migrated from your primary drive, you do not do anything else on your server, primarily because the data corruption patch (which is part of PP1) will no longer be installed.  Once it has completed migrating data from your primary drive, then you can re-install PP1 beta.

    Tuesday, July 15, 2008 4:45 PM
    Moderator
  •  kariya21 wrote:

     Tim Lastoria wrote:
    I do get the message from the ripping program (IMGBURN) about 60% through the rip saying that I ran out of disc space...and in fact Drive1 shows 100% full.

     

    You might try ripping it to your local drive first, then copy it to the server afterwards.  (Perhaps IMGBURN is going "through" the share to first drive it sees.)

     

     Tim Lastoria wrote:
    The only drives I added after the PP1 beta were Disk5 and Disk6.

     

    Yep, that's what I guessed when I saw your numbers.

     

    You might want to uninstall PP1 beta.  Once you do, it will free up the space on your primary drive.  (There is a known issue with data migration in PP1.  Please read my post, Drive Extender Not Migrating Data, in the PP1 beta forum.)

     

    However, I would also suggest that while the data is being migrated from your primary drive, you do not do anything else on your server, primarily because the data corruption patch (which is part of PP1) will no longer be installed.  Once it has completed migrating data from your primary drive, then you can re-install PP1 beta.

     

    Okay, I just removed PP1 and rebooted.  Data is flowing off of my D: drive nicely now.  Once this finishes, will it also move some data off of the drives that are 90% and fuller?  I did add the reg key dwords to up the default 10GB/20Gb threshold to 40GB and 60GB if that makes a difference.

    Tuesday, July 15, 2008 6:24 PM
  •  

    Also, as I'm watching this, I am only getting between 8 - 10 MB/sec.  Should I be getting a solid 30 MB/sec+???  Is this normal?
    Tuesday, July 15, 2008 7:18 PM
  • Windows Home Server won't move much (if any) of the data off the drives that are full to the drives that are more empty. Without Power Pack 1, it will move most of the data that's on the system drive to other drives, however.

    Regarding disk performance, your numbers are not abnormal for Windows Home Server. It's doing a lot behind the scenes.
    Tuesday, July 15, 2008 7:41 PM
    Moderator
  •  Tim Lastoria wrote:
    Okay, I just removed PP1 and rebooted.  Data is flowing off of my D: drive nicely now.  Once this finishes, will it also move some data off of the drives that are 90% and fuller?  I did add the reg key dwords to up the default 10GB/20Gb threshold to 40GB and 60GB if that makes a difference.

     

    I tried adding those keywords as well, but I couldn't get it to move the files.  Maybe you'll have better luck. Smile

    Tuesday, July 15, 2008 8:16 PM
    Moderator
  • Okay...so I have left PP1 uninstalled for several days now.  My D: drive started moving data to other free drives immediately, but stopped when it got to 46% (200 GB free).  In addition to that when I copy files to the server, it still tries to go to Disk1 which is still at 97% full.  Currently there is no balancing going on.

     

    Should I put PP1 back on now and see what happens?  Is there a way to get demigrator to finish moving data off of the D: drive???

    Thursday, July 17, 2008 3:33 PM
  •  Tim Lastoria wrote:
    Okay...so I have left PP1 uninstalled for several days now.  My D: drive started moving data to other free drives immediately, but stopped when it got to 46% (200 GB free).

     

    It's possible the rest of the space on your primary drive is being used by your backup database (which, as I recall, usually won't move).

     

     Tim Lastoria wrote:
    In addition to that when I copy files to the server, it still tries to go to Disk1 which is still at 97% full.

     

    Did you try using standard file copying through Windows Explorer (instead of IMGBURN)?

     

     Tim Lastoria wrote:
    Currently there is no balancing going on.

     

    Should I put PP1 back on now and see what happens?  Is there a way to get demigrator to finish moving data off of the D: drive???

    Thursday, July 17, 2008 3:40 PM
    Moderator
  •  kariya21 wrote:

     Tim Lastoria wrote:
    Okay...so I have left PP1 uninstalled for several days now.  My D: drive started moving data to other free drives immediately, but stopped when it got to 46% (200 GB free).

     

    It's possible the rest of the space on your primary drive is being used by your backup database (which, as I recall, usually won't move).

     

     Tim Lastoria wrote:
    In addition to that when I copy files to the server, it still tries to go to Disk1 which is still at 97% full.

     

    Did you try using standard file copying through Windows Explorer (instead of IMGBURN)?

     

     Tim Lastoria wrote:
    Currently there is no balancing going on.

     

    Should I put PP1 back on now and see what happens?  Is there a way to get demigrator to finish moving data off of the D: drive???

     

    Okay, I deleted all of my backups and they were stored on Disk5.  I'm pretty sure the data that is on D:, is data from my shares that should be moved.

     

    Yes, I did test just copying a file through Windows Explorer and it directs it to Disk1.

    Thursday, July 17, 2008 3:47 PM
  •  Tim Lastoria wrote:
    Okay, I deleted all of my backups and they were stored on Disk5.  I'm pretty sure the data that is on D:, is data from my shares that should be moved.

     

    Yes, I did test just copying a file through Windows Explorer and it directs it to Disk1.

     

    Did you add Disks 5 and 6 through the Console?  Did you have any errors when you added them?  Can you please verify in the Console that all 6 disks are added to the storage pool?

    Thursday, July 17, 2008 4:23 PM
    Moderator
  •  kariya21 wrote:
    Did you add Disks 5 and 6 through the Console?  Did you have any errors when you added them?  Can you please verify in the Console that all 6 disks are added to the storage pool?

     

    Disk5 and Disk6 were added through the console without any errors.  I do have 7 disks by the way (counting starts at Disk0).  All 7 disks are in the storage pool.

     

    See the following screenshot for more detailed info:  http://www.appstar.com/lastoria/whs.jpg

     

    Thursday, July 17, 2008 4:45 PM
  •  Tim Lastoria wrote:
    Disk5 and Disk6 were added through the console without any errors.  I do have 7 disks by the way (counting starts at Disk0).  All 7 disks are in the storage pool.

     

    See the following screenshot for more detailed info:  http://www.appstar.com/lastoria/whs.jpg

     

    Well, based on everything you've described, I think you might be looking at a Server Reinstallation.  However, 1 final question:  when you added the drives to the pool, which tab did you use to add them (Server Storage or Disk Management)?

    Thursday, July 17, 2008 5:03 PM
    Moderator
  •  kariya21 wrote:
    Well, based on everything you've described, I think you might be looking at a Server Reinstallation.  However, 1 final question:  when you added the drives to the pool, which tab did you use to add them (Server Storage or Disk Management)?

     

    Server Storage

     

    Okay, in the meantime I kept adding data to the server until those full drives were getting really low (under 10 GB free).  I am not doing any data copies and I see that my D: drive is now moving data off to Disk5.  Sweet!  I just had to put some pressure on it...LOL.  Hopefully it will move all of the data off this time and then I will re-install PP1...or do you think I should wait for the final release of PP1?

    Thursday, July 17, 2008 5:10 PM
  •  Tim Lastoria wrote:
    Server Storage

     

    OK.  I know that there were some issues with the new Disk Management (which you're using) and PP1, but I guess that doesn't apply here.

     

     Tim Lastoria wrote:
    Okay, in the meantime I kept adding data to the server until those full drives were getting really low (under 10 GB free).  I am not doing any data copies and I see that my D: drive is now moving data off to Disk5.  Sweet!  I just had to put some pressure on it...LOL.

     

    In RTM, that shouldn't be necessary, but as long as it works.... Smile

     

     Tim Lastoria wrote:
    Hopefully it will move all of the data off this time and then I will re-install PP1...or do you think I should wait for the final release of PP1?

     

    That's a personal preference.  For me, I've been waiting for the final bits before I put it on my live server (I don't put any beta software on my main storage device).  Also, just so you know, PP1 "will be shipping in all languages in fairly short order."  (Quote from Todd Headrick, seen here.)

    Thursday, July 17, 2008 5:29 PM
    Moderator
  • Tim, before you proceed any further could you submit a bug report, with server logs, on Connect? Windows Home Server should really not be leaving that much data on your system drive, especially if Power Pack 1 isn't installed and you have no other add-ins or software .

    You'll find instructions here, or in the Power Pack 1 documentation.
    Thursday, July 17, 2008 6:28 PM
    Moderator
  •  Ken Warren wrote:
    Tim, before you proceed any further could you submit a bug report, with server logs, on Connect? Windows Home Server should really not be leaving that much data on your system drive, especially if Power Pack 1 isn't installed and you have no other add-ins or software .

    You'll find instructions here, or in the Power Pack 1 documentation.

     

    Once I removed PP1 and added data in small increments to the almost full drives, that seemed to kick start the data migration off of my system drive.  I now have 347 GB free out of 352...the other 5 GB is non-WHS data.

     

    If I submit a bug report now, will that help even though everything seems to be okay now?

    Friday, July 18, 2008 3:04 PM
  • On the Australian forum there is a similar post. They found having Winzip take the larger files and then move them in 1 gig chunks worked.

    Ben
    Friday, July 18, 2008 3:45 PM
  • I have the same problem as this with my HP MediaSmart EX490:  The windows folder has *over 16GB* of data in it even though there's > 1.6TB of free disk space on other disks. 

    I'm trying to run the HP WHS Recover disk but it fails every time. 

    How do I diagnose this problem?

    TIA - Michael
    Sunday, November 29, 2009 7:15 PM
  • (replying to self -- sorry)

    The problem turns out to be huge log files (over 16 Gb worth, each ~300 Mb in size) under c:\windows\system32\LogFiles\W3SVC1.   WinDirStat helped me find the culprit.

    I didn't look into the log files yet but got back 12 Gb of space.  Would still like to know why I can't run a Recovery though ;).

    Michael

    Sunday, November 29, 2009 8:35 PM
  • You say it's failing, but don't give any details as to how. Could you post a separate thread about this issue?

    Regarding the log files, if you haven't deleted all of them yet you might want to keep and examine one of them, to see why you're getting such large logs. I have logs dating back to day 1 of my EX470, and there's only around 45 MB of logs in that folder in toto . I would be you've either installed something on your server which is causing the large files, or you're constantly being probed from somewhere.
    I'm not on the WHS team, I just post a lot. :)
    Sunday, November 29, 2009 9:09 PM
    Moderator
  • Hi Ken - Will post a separate thread on Recovery failure.  Basically it gets to "Step 2" (doesn't tell me what that is) and occasionally "Step 3" (doesn't tell me what that is, either) before displaying

    "Step 3 of 4 failed.  Click Help for information on how to address the problem."    (or Step 2 as the case may be).

    Clicking on the Help Button shows another dialog titled "HP Server Recovery"

    "There was an unidentified error."

    Helpful!

    I'll start another thread.

    Thanks - Michael
    Sunday, November 29, 2009 10:12 PM
  • I saw this thread and remembered reading about this a few days ago, so you may find this of relevance. Basically the size of the system disk limits the maximum file size you can copy despite the total amount of storage available across all drives.

    My WHS system for example, 74GB (69GB formatted) system drive: 20GB for Windows (C:), 49GB for the Primary Data Partition (D:), 2x 500GB and 1x 1TB drives pooled to the logical data partition. Thus, even though I have around 2TB of system storage, the maximum filesize supported for transfer across the network is always going to be 49GB.

    http://www.microsoft.com/downloads/details.aspx?FamilyID=40c6c9cc-b85f-45fe-8c5c-f103c894a5e2&DisplayLang=en

    (Page 9/22)

    In a single hard-drive system, Windows Home Server is configured with a 20-gigabyte system partition for the Windows Home Server operating system, and the remainder of the primary hard drive is allocated to the primary data partition. Each subsequent (or secondary) hard drive is logically added to the data partition. It appears to the user as one large hard drive; however, these drives are considered to be part of the secondary data partition. In reality, your home server consists of multiple hard drives of varying sizes.


    The primary data partition is used to store a unique entry for each file. If a home server has more than one hard drive, these files become “tombstones.” Tombstones are NTFS file system reparse points that Windows Home Server Drive Extender understands. They are tiny files that redirect to one or two “shadow” files on the hard drives that make up the secondary data partition. The shadow files are where your data is really stored. If folder duplication is enabled for a Shared Folder, there will be two shadow files. If duplication is Off, there will be one shadow file.


    Note

    The primary data partition in a home server should be as large as possible for two reasons:

    • You want to provide sufficient space to grow the file table for all of the files that you will store on your home server.

    • Windows Vista® and other home computer operating systems check to see if there is adequate space on the primary data partition prior to starting a copy operation.

    Monday, November 30, 2009 1:52 AM
  • I saw this thread and remembered reading about this a few days ago, so you may find this of relevance. Basically the size of the system disk limits the maximum file size you can copy despite the total amount of storage available across all drives.

    My WHS system for example, 74GB (69GB formatted) system drive: 20GB for Windows (C:), 49GB for the Primary Data Partition (D:), 2x 500GB and 1x 1TB drives pooled to the logical data partition. Thus, even though I have around 2TB of system storage, the maximum filesize supported for transfer across the network is always going to be 49GB.

    http://www.microsoft.com/downloads/details.aspx?FamilyID=40c6c9cc-b85f-45fe-8c5c-f103c894a5e2&DisplayLang=en

    (Page 9/22)

    In a single hard-drive system, Windows Home Server is configured with a 20-gigabyte system partition for the Windows Home Server operating system, and the remainder of the primary hard drive is allocated to the primary data partition. Each subsequent (or secondary) hard drive is logically added to the data partition. It appears to the user as one large hard drive; however, these drives are considered to be part of the secondary data partition. In reality, your home server consists of multiple hard drives of varying sizes.


    The primary data partition is used to store a unique entry for each file. If a home server has more than one hard drive, these files become “tombstones.” Tombstones are NTFS file system reparse points that Windows Home Server Drive Extender understands. They are tiny files that redirect to one or two “shadow” files on the hard drives that make up the secondary data partition. The shadow files are where your data is really stored. If folder duplication is enabled for a Shared Folder, there will be two shadow files. If duplication is Off, there will be one shadow file.


    Note

    The primary data partition in a home server should be as large as possible for two reasons:

    • You want to provide sufficient space to grow the file table for all of the files that you will store on your home server.

    • Windows Vista® and other home computer operating systems check to see if there is adequate space on the primary data partition prior to starting a copy operation.

    This is no longer the case if your server is up-to-date.  MS released a patch late last year that addressed this issue.
    • Proposed as answer by Michael.Murray Monday, November 30, 2009 2:33 AM
    Monday, November 30, 2009 2:13 AM
    Moderator
  • Thx for the link and the update guys.

    This may not be related but my WHS box won't let me apply Power Pack 3 for some reason although Windows Update has applied other fixes.

    I see that Power Pack 2 is a prerequisite for Power Pack 3.  How can I tell if Power Pack 2 is installed?

    Thanks - Michael
    Monday, November 30, 2009 2:35 AM
  • Thx for the link and the update guys.

    This may not be related but my WHS box won't let me apply Power Pack 3 for some reason although Windows Update has applied other fixes.

    I see that Power Pack 2 is a prerequisite for Power Pack 3.  How can I tell if Power Pack 2 is installed?

    Thanks - Michael
    What are the component numbers in your Console (Settings button, Resources tab)?
    Monday, November 30, 2009 2:40 AM
    Moderator
  • Unfortunately I cannot tell what the component numbers are in my Console as the WHS Console won't start -- keeps telling me "This computer has lost connection...".  I can RAS to my WHS box though.

    So WHS not in a good state :(.
    Monday, November 30, 2009 3:03 AM
  • Unfortunately I cannot tell what the component numbers are in my Console as the WHS Console won't start -- keeps telling me "This computer has lost connection...".  I can RAS to my WHS box though.

    So WHS not in a good state :(.
    In that case, try opening the Console from the server desktop and see what the component numbers are.
    Tuesday, December 1, 2009 2:07 AM
    Moderator
  • The console won't open from the server desktop via RAS connection either.  Ideas?
    Tuesday, December 1, 2009 4:53 AM
  • The console won't open from the server desktop via RAS connection either.  Ideas?
    In that case, I would say try a Server Recovery, and since you are having problems with that, I would recommend you call HP for support (perhaps you have a hardware issue).
    Tuesday, December 1, 2009 5:30 AM
    Moderator
  • Since you have this wonderful bit of knowledge about what is best practices for setting up a new WHS; It would be nice to put this information into a new installation guidelines.

    That way, people like me don't have to do a reinstallation because we setup WHS with a Primary drive of 100GB and a secondary drive of 1 TB. The theory being that the Windows operating system would work on the SYS Primary drive and Most if not all Data would go to the secondary.

     

    Is there a way to do this? Force WHS to not use the 100GB drive for Data? Without this well thought out program of data propagation put into the Installation Guide, people without previous knowledge about Servers might not make stupid mistakes.  

     

    If that sounded like a rant is was. I had to learn this after 3 days of installation and waiting for Microsoft update to finally upgrade my installation with Power pack 3 so my Windows 7 Pro computers could finally connect. Then as I am copying a very large Music library in 2.3GB chucks it stops dead. Then I start googling and find this fine mess that tells me " I chose poorly".

    Wednesday, April 28, 2010 6:08 PM
  • No. However, since PP1 WHS will always write to a free data disk first.

    If you ever have data on the system drive, your storage pool must be full (with duplication too)
    There is a Drive Balancer utility you can use to force the data off the system drive

     

     

    Wednesday, April 28, 2010 8:58 PM