Sign in
Microsoft.com
 
Microsoft
 
 
 

locked
(some) Large File Copies to WHS Shares fails (Drive Extender probs?) RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote
    I'm using Sync Toy 2 to copy media files to my Shares in WHS.  I've plenty of room in my Storage Pool over 4 drives with the two smaller 500GB drives now at 92-97% full (eg 10-30GB Free) and the two large 1TB drive mostly free.

    When I started coping large 40GB ISO's I started getting some "failed" to copy messages in SyncToy.  After trying to keep an eye on what was happening it "appears" that "some" of the ISO would start being copied to one of the almost full drives (eg 10-30GB Free), and would then fail as the drive fills up.  It would then repeat etc. 

    As this sounded similar to problems I had when doing large backups at
    http://social.microsoft.com/Forums/en-US/whssoftware/thread/6725a538-dd20-4cb5-8899-39807a9a016d I I tried rebooting then re-running but was still getting copy "failed" on some files.  (note: I was originally using the backup function over my media files but I now only back up the OS drive of my PC's and want to use SyncToy to "sync" the media files to the WHS Shares).

    I've now restarted the copies and the disk activity suggested it is now going to one of the larger free drives so here is hoping!

    Here are the only unusual logs that have been reported over the day:

    Straight after the last reboot: NLB Cluster 0.0.0.0 : Cluster mode cannot be enabled due to parameter errors. All traffic will be passed through to TCP/IP. Restart cluster operations after fixing the problem by running 'wlbs reload' followed by 'wlbs start'. and

    The configuration information of the performance library "C:\WINDOWS\system32\infoctrs.dll" for the "InetInfo" service does not match the trusted performance library information stored in the registry. The functions in this library will not be treated as trusted.

    I also got some of these in an earlier session but NOT after the last reboot so it has to be something else: The device, \Device\Scsi\Si3124r51, did not respond within the timeout period.



    Thanks
    Nathan
    Friday, March 6, 2009 8:47 PM

Answers

  • Question
    Sign in to vote
    1
    Sign in to vote
    I've noticed that Synctoy (and robocopy, IIRC) have an "interesting" way of copying files, one which is shared by some other copy utilities. They apear to look at the free space on the server (which could be reported as greater than the free space on any single drive in the server) and determine whether to attempt a copy based on that. If there's enough free space, they start the copy with a small amount of data and extend the file as needed during the copy process. This breaks the WHS algorithm for choosing a drive. WHS chooses the drive to place a file on by selecting the drive with the least free space, as long as it has enough space for the specified file. Because Synctoy is asking for e.g. 64 kb at first, then extends it repeatedly, WHS can (in the case of a huge BluRay .ISO) choose a drive with less free space than the final size of the drive.

    What happens when you try using Windows Explorer for the copy? Xcopy? Robocopy?
    I'm not on the WHS team, I just post a lot. :)
    • Proposed as answer by kariya21Moderator Sunday, March 8, 2009 3:36 PM
    • Marked as answer by jmone Sunday, March 8, 2009 7:40 PM
    Saturday, March 7, 2009 2:36 PM
    Moderator
  • Question
    Sign in to vote
    1
    Sign in to vote
    jmone said:

    jmone said:

    Thanks - I've filed the bug report on Connect as suggested.



    Ahhh "Lara Jones" has closed the ticket saying that this has been fixed in a "later version" - I updated the ticket saying that given WHS is up to date and I'm using the download for SyncToy2 from only last week.  Ken/kariya I may not have explained myself well, can you please advise if the following that I posted is a correct explanation (https://connect.microsoft.com/WindowsHomeServer/feedback/ViewFeedback.aspx?FeedbackID=421794):

    It appears that the way SyncToy copies files, and the way WHS advertises available storage space can result in a situation where large files fail to complete a copy.

    This failure is a combination of:
    1) SyncToy sees all the available space on the combined drive pool (not just the physical drive targeted)
    2) When SyncToy starts the copy process, WHS selects the physical drive based on the size of the file being saved - which apparently is just 64K that is then "gown" as data is transferred (according to Ken W in the post linked below)

    The result is that while a large file (like a Blu-ray ISO) may be 50GB, WHS will commence saving it to a physical drive with less available space than is actually required as it "thinks" the file being transferred is only 64K.

    The original thread on this is at the WHS forum http://social.microsoft.com/Forums/en-US/whssoftware/thread/a40bfdd9-2cc6-4289-b00e-0e98de16ff8f but I thought it best to also post the info here in the hope the WHS / SycToy devs may be interested.

    Thanks
    Nathan



    Later version means a version of Windows Home Server, a Power Pack or a QFE that has not yet been released to the public. Connect works out of the developer database and sometimes, items will show resolved as fixed even though the code is not yet available to the public. In this case, the code is still being tested and is under NDA.

    Thanks!
    Lara Jones [MSFT] | Program Manager
    Community Support and Beta | Windows Home Server Team
    Windows Home Server Team Blog
    Connect Windows Home Server
    Windows Home Server
    • Edited by Lara JonesModerator Wednesday, March 11, 2009 8:52 PM added additional wording
    • Marked as answer by jmone Wednesday, March 11, 2009 9:38 PM
    Wednesday, March 11, 2009 8:01 PM
    Moderator

All replies

  • Question
    Sign in to vote
    0
    Sign in to vote
    jmone said:

    I'm using Sync Toy 2 to copy media files to my Shares in WHS.  I've plenty of room in my Storage Pool over 4 drives with the two smaller 500GB drives now at 92-97% full (eg 10-30GB Free) and the two large 1TB drive mostly free.

    When I started coping large 40GB ISO's I started getting some "failed" to copy messages in SyncToy.  After trying to keep an eye on what was happening it "appears" that "some" of the ISO would start being copied to one of the almost full drives (eg 10-30GB Free), and would then fail as the drive fills up.  It would then repeat etc. 

    As this sounded similar to problems I had when doing large backups at
    http://social.microsoft.com/Forums/en-US/whssoftware/thread/6725a538-dd20-4cb5-8899-39807a9a016d I I tried rebooting then re-running but was still getting copy "failed" on some files.  (note: I was originally using the backup function over my media files but I now only back up the OS drive of my PC's and want to use SyncToy to "sync" the media files to the WHS Shares).

    I've now restarted the copies and the disk activity suggested it is now going to one of the larger free drives so here is hoping!

    Here are the only unusual logs that have been reported over the day:

    Straight after the last reboot: NLB Cluster 0.0.0.0 : Cluster mode cannot be enabled due to parameter errors. All traffic will be passed through to TCP/IP. Restart cluster operations after fixing the problem by running 'wlbs reload' followed by 'wlbs start'. and

    The configuration information of the performance library "C:\WINDOWS\system32\infoctrs.dll" for the "InetInfo" service does not match the trusted performance library information stored in the registry. The functions in this library will not be treated as trusted.

    I also got some of these in an earlier session but NOT after the last reboot so it has to be something else: The device, \Device\Scsi\Si3124r51, did not respond within the timeout period.



    Thanks
    Nathan



    How are you copying the files to the server?  Is your server up-to-date?  What are the component numbers (Settings button, Resources tab)?
    Saturday, March 7, 2009 12:06 AM
    Moderator
  • Question
    Sign in to vote
    0
    Sign in to vote
    Hi kariya

    How are you copying the files to the server?  - MS SyncToy

    Is your server up-to-date?  Yes

    What are the component numbers (Settings button, Resources tab)?
      Windows Home Server Console: 6.0.1800.0
      Windows Home Server Backup & Restore: 6.0.1800.36
      Windows Home Server Drive Extender: 6.0.1800.24
      Windows Home Server Remote Access: 6.0.1800.0
      Windows Home Server Storage Manager: 6.0.1800.24

    PS the latest SyncToy run just worked fine (it was all going to one of the empty 1TB Drive).  The problem seems to be when WHS starts using one of the full drives first. Just guessing.

    Thanks
    Nathan
    Saturday, March 7, 2009 1:30 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    OK - I just watched it fail.

    I had SyncToy coping a bunch of large ISO files over to the share folder in WHS and watched the Disk Usage (using the Disk Mgt Plug In) slowly decrease to 0, unsurprisingly the copy then failed with a SyncToy error message of "Cannot write to the destination file.  There is not enough space on the disk". 

    Once it failed WHS then freed up about 33GB (now 96% full) on this drive (probably the portion of the failed file copy that had been transferred already) as the next file stared being copied.  As this file is also over 33GB the same would happen.  At no stage did WHS try to use another disk with 700GB free, it just kept going.  Nothing relating to this was added to the Event Log, however at the bottom of the WHS Console it said that the "Storage Balanced" about 45-Mins before the fail.

    I'll now try a reboot and start the SyncToy copy again and see if it now picks up the drive with spare capacity.

    I'll hazard a guess that the WHS Drive Extender? only kicks in periodically (or at certain percentage point) and that a large single file can run over limit.  As this is 100% repeatable I'm thinking this one is a bug!


    Thanks
    Nathan
    Saturday, March 7, 2009 6:26 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    On this run it also did the same thing with the large ISO but it then moved onto a file that did fit onto that drive taking it to 98% (about 10GB free space) THEN it moved onto the other drive for the following file transfers.  It seems to confirm that WHS does not like to move to using another drive unless there is under say 20gb or so left.
    Thanks
    Nathan
    Saturday, March 7, 2009 10:36 AM
  • Question
    Sign in to vote
    1
    Sign in to vote
    I've noticed that Synctoy (and robocopy, IIRC) have an "interesting" way of copying files, one which is shared by some other copy utilities. They apear to look at the free space on the server (which could be reported as greater than the free space on any single drive in the server) and determine whether to attempt a copy based on that. If there's enough free space, they start the copy with a small amount of data and extend the file as needed during the copy process. This breaks the WHS algorithm for choosing a drive. WHS chooses the drive to place a file on by selecting the drive with the least free space, as long as it has enough space for the specified file. Because Synctoy is asking for e.g. 64 kb at first, then extends it repeatedly, WHS can (in the case of a huge BluRay .ISO) choose a drive with less free space than the final size of the drive.

    What happens when you try using Windows Explorer for the copy? Xcopy? Robocopy?
    I'm not on the WHS team, I just post a lot. :)
    • Proposed as answer by kariya21Moderator Sunday, March 8, 2009 3:36 PM
    • Marked as answer by jmone Sunday, March 8, 2009 7:40 PM
    Saturday, March 7, 2009 2:36 PM
    Moderator
  • Question
    Sign in to vote
    0
    Sign in to vote
    Hi Ken,  That makes sense but I don't think there is more to the story, as there also seems to be a point that WHS decides that a drive is "full".  I still have free 14, 11, and 16GB on the three drives (all now 96-98% full) which I've had the copy problem with so far.  WHS was happy to use these drives when they had 20GB?? or more but now the copying process has moved onto the drive that still has 350GB Free. 

    I'm still copying data onto my final drive (a TB takes a looooong time) so I'll see how I go with testing other progs to confirm you suspicions.  Regardles of what causes it it is still a WHS bug, and while easy to work around once you know the issue it should be down to the Storage Pool "Balancer" process to monitor then to start moving data off onto the other drive as disk space runs out or split the file over the two physical disks.

    Thanks
    Nathan 
    Saturday, March 7, 2009 8:46 PM
  • Question
    Sign in to vote
    1
    Sign in to vote
    Files can't be split over multiple disks; they are stored as single files in a standard NTFS file system.

    As for this being a bug in Windows Home Server, I don't think I agree. I think it would be more accurate to point to synctoy as the source of the problem, if you're encountering the behavior I described.
    I'm not on the WHS team, I just post a lot. :)
    Sunday, March 8, 2009 6:29 AM
    Moderator
  • Question
    Sign in to vote
    0
    Sign in to vote
    Ken - You are correct about the behavior of MS SyncToy Vs Windows Explorer & WHS. I've finished all my copying and WHS have now "Balanced" the drives so all of my "full" drives have 20GB each free with the final drive having 300GB free. 

    So Time for some testing using both MS SyncToy & Windows Explorer to try copying a 35GB ISO to WHS, and the results are:
    1) SyncToy --> WHS will pick one of the drives with only 20GB free (and hence it will fail)
    2) Windows Explorer --> WHS will pick the drive with 300GB free (and hence it will work)

    Sunday, March 8, 2009 6:36 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    Ken Warren said:

    Files can't be split over multiple disks; they are stored as single files in a standard NTFS file system.

    As for this being a bug in Windows Home Server, I don't think I agree. I think it would be more accurate to point to synctoy as the source of the problem, if you're encountering the behavior I described.

    I'm not on the WHS team, I just post a lot. :)



    Thanks Ken, As a user I just want it to work, I really don't care if it is a Bug in or "misunderstanding" between:
    1) MS WHS - which tells SyncToy that it has plenty of space then spits the dummy just because 1 drive in it's storage pool fills up, or
    2) MS SyncToy - which does not tell WHS how much it is really going to copy

    I recon a a quick conversation between the MS WHS and SyncToy devs would sort it out in 10mins flat!  For know at least we know what the problem is......

    Thanks
    Nathan
    Sunday, March 8, 2009 6:43 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    FYI - I've posted a summary over at the SyncToy forum http://social.microsoft.com/Forums/en-US/synctoy/thread/c2e3ae34-59b2-4add-838a-48d2634db7dd

    While we now know Why it can fail, and could use strategies to minimuse the issue I'm pretty keen for the issue between WHS and SyncToy to be resolved as both products are great at what they do - just not together!
    Sunday, March 8, 2009 7:43 PM
  • Question
    Sign in to vote
    1
    Sign in to vote
    jmone said:

    FYI - I've posted a summary over at the SyncToy forum http://social.microsoft.com/Forums/en-US/synctoy/thread/c2e3ae34-59b2-4add-838a-48d2634db7dd

    While we now know Why it can fail, and could use strategies to minimuse the issue I'm pretty keen for the issue between WHS and SyncToy to be resolved as both products are great at what they do - just not together!


    You could also file a bug report on Connect.
    Sunday, March 8, 2009 7:51 PM
    Moderator
  • Question
    Sign in to vote
    0
    Sign in to vote
    Thanks - I've filed the bug report on Connect as suggested.
    Sunday, March 8, 2009 10:38 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    jmone said:

    Thanks - I've filed the bug report on Connect as suggested.



    Ahhh "Lara Jones" has closed the ticket saying that this has been fixed in a "later version" - I updated the ticket saying that given WHS is up to date and I'm using the download for SyncToy2 from only last week.  Ken/kariya I may not have explained myself well, can you please advise if the following that I posted is a correct explanation (https://connect.microsoft.com/WindowsHomeServer/feedback/ViewFeedback.aspx?FeedbackID=421794):

    It appears that the way SyncToy copies files, and the way WHS advertises available storage space can result in a situation where large files fail to complete a copy.

    This failure is a combination of:
    1) SyncToy sees all the available space on the combined drive pool (not just the physical drive targeted)
    2) When SyncToy starts the copy process, WHS selects the physical drive based on the size of the file being saved - which apparently is just 64K that is then "gown" as data is transferred (according to Ken W in the post linked below)

    The result is that while a large file (like a Blu-ray ISO) may be 50GB, WHS will commence saving it to a physical drive with less available space than is actually required as it "thinks" the file being transferred is only 64K.

    The original thread on this is at the WHS forum http://social.microsoft.com/Forums/en-US/whssoftware/thread/a40bfdd9-2cc6-4289-b00e-0e98de16ff8f but I thought it best to also post the info here in the hope the WHS / SycToy devs may be interested.

    Thanks
    Nathan
    Wednesday, March 11, 2009 7:56 PM
  • Question
    Sign in to vote
    1
    Sign in to vote
    jmone said:

    jmone said:

    Thanks - I've filed the bug report on Connect as suggested.



    Ahhh "Lara Jones" has closed the ticket saying that this has been fixed in a "later version" - I updated the ticket saying that given WHS is up to date and I'm using the download for SyncToy2 from only last week.  Ken/kariya I may not have explained myself well, can you please advise if the following that I posted is a correct explanation (https://connect.microsoft.com/WindowsHomeServer/feedback/ViewFeedback.aspx?FeedbackID=421794):

    It appears that the way SyncToy copies files, and the way WHS advertises available storage space can result in a situation where large files fail to complete a copy.

    This failure is a combination of:
    1) SyncToy sees all the available space on the combined drive pool (not just the physical drive targeted)
    2) When SyncToy starts the copy process, WHS selects the physical drive based on the size of the file being saved - which apparently is just 64K that is then "gown" as data is transferred (according to Ken W in the post linked below)

    The result is that while a large file (like a Blu-ray ISO) may be 50GB, WHS will commence saving it to a physical drive with less available space than is actually required as it "thinks" the file being transferred is only 64K.

    The original thread on this is at the WHS forum http://social.microsoft.com/Forums/en-US/whssoftware/thread/a40bfdd9-2cc6-4289-b00e-0e98de16ff8f but I thought it best to also post the info here in the hope the WHS / SycToy devs may be interested.

    Thanks
    Nathan



    Later version means a version of Windows Home Server, a Power Pack or a QFE that has not yet been released to the public. Connect works out of the developer database and sometimes, items will show resolved as fixed even though the code is not yet available to the public. In this case, the code is still being tested and is under NDA.

    Thanks!
    Lara Jones [MSFT] | Program Manager
    Community Support and Beta | Windows Home Server Team
    Windows Home Server Team Blog
    Connect Windows Home Server
    Windows Home Server
    • Edited by Lara JonesModerator Wednesday, March 11, 2009 8:52 PM added additional wording
    • Marked as answer by jmone Wednesday, March 11, 2009 9:38 PM
    Wednesday, March 11, 2009 8:01 PM
    Moderator
  • Question
    Sign in to vote
    0
    Sign in to vote
    Please note that what I've said above is really more of an educated guess at what's happening, based on observation and some experimentation. There are probably other scenarios that would reach the same end result. And the 64k is merely an example; I don't know how much data synctoy tries to write at one time, or exactly how it deals with the process.
    I'm not on the WHS team, I just post a lot. :)
    Wednesday, March 11, 2009 8:40 PM
    Moderator
  • Question
    Sign in to vote
    0
    Sign in to vote
    Thanks!  Happy to wait.
    Nathan
    Wednesday, March 11, 2009 9:39 PM