locked
Shares unavailable during drive removal RRS feed

  • General discussion

  • Part of the design of WHS is to unshare the folders while a drive is being removed.  This is a bad feature, IMO, since drive removal (even when working as designed) takes a long time.

    These days, most people are going to have large drives.  500GB isn't uncommon, and I pack a LOT of data onto my WHS, so I use 1.5TB or 2TB drives.  My MediaSmart tends to get around 15MB/sec on USB drives.  So removing a mostly-full 500GB drive will take 10 hours.  A 1.5TB drive will take 30 hours!  To not have any file sharing for that long of a period, not to mention backups, is unacceptable.  And as drives get larger (I'm about to move to 2TB drives), the delay is going to get longer.

    I've added and removed a lot of drives in my time.  It appears that the intent of removing drives is to try to prevent files from being used.  However it doesn't necessarily meet this goal, since local programs like WMC can be accessing D:\shares directly.  The vast majority of the time, very few files are being used on any given drive whether directly or via shares.  The drive removal algorithm handles busy files well enough (by skipping the file and coming back to it later).  By the time it is done moving the rest of the files that are not in use, the one or two that were skipped would probably no longer be in use.

    So why not only disable the backups while moving the backup folder (reenabling them afterwards), and not disable the shares at all?  I'd rather the drive removal take an extra few minutes retrying busy files and report a file conflict (which happens often enough anyway) than have my shares unavailable for longer than a day.
    Saturday, June 13, 2009 6:24 PM

All replies

  • Part of the design of WHS is to unshare the folders while a drive is being removed.  This is a bad feature, IMO, since drive removal (even when working as designed) takes a long time.

    These days, most people are going to have large drives.  500GB isn't uncommon, and I pack a LOT of data onto my WHS, so I use 1.5TB or 2TB drives.  My MediaSmart tends to get around 15MB/sec on USB drives.  So removing a mostly-full 500GB drive will take 10 hours.  A 1.5TB drive will take 30 hours!  To not have any file sharing for that long of a period, not to mention backups, is unacceptable.  And as drives get larger (I'm about to move to 2TB drives), the delay is going to get longer.

    I've added and removed a lot of drives in my time.  It appears that the intent of removing drives is to try to prevent files from being used.  However it doesn't necessarily meet this goal, since local programs like WMC can be accessing D:\shares directly.  The vast majority of the time, very few files are being used on any given drive whether directly or via shares.  The drive removal algorithm handles busy files well enough (by skipping the file and coming back to it later).  By the time it is done moving the rest of the files that are not in use, the one or two that were skipped would probably no longer be in use.

    So why not only disable the backups while moving the backup folder (reenabling them afterwards), and not disable the shares at all?  I'd rather the drive removal take an extra few minutes retrying busy files and report a file conflict (which happens often enough anyway) than have my shares unavailable for longer than a day.

    You might want to post a suggestion for that on Connect and post the link here for other users to vote for it.
    Saturday, June 13, 2009 9:46 PM
    Moderator
  • You can submit a product suggestion on Connect, per kariya21's recommendation. However, disabling sharing during drive removal is required for Drive Extender to be able to remove a drive successfully, and I don't see this changing in this version of Windows Home Server (though Microsoft has proven me wrong in this sort of thing before, much to my delight :) ). Without doing so, there is no way to prevent a user from writing a new file to the shares, which could in turn have a shadow written to the drive being removed. There's also no way to prevent a user from opening a file whose primary shadow is on the drive being removed, and holding it open for hours or days, thus blocking drive removal. Finding out the exact cause of a file being held open for an extended period isn't particularly easy, either, for an end user. Given that Windows Home Server is designed for a non-technical "admin", figuring out how to close a file that's blocking a drive removal could be beyond their capabilities. 

    So overall, I would recommend just removing the drive overnight. In your case, the slowness of your drive connection makes that difficult, at best. Regarding drive connections, BTW, I recommend against the use of USB drives in the storage pool. USB is not, at heart, designed to be a mass storage bus, and if you read the forums here you will see that there have been a variety of problems that were eventually traced back to a USB drive. You should use internal drives, and eSATA (with the understanding that SATA can complicate the recovery process if your system drive fails) for external drives.

    I'm not on the WHS team, I just post a lot. :)
    Monday, June 15, 2009 2:46 PM
    Moderator
  • Thanks, I've added the suggestion to Connect.  Here is the link.

    Granted, it may not be practical to prevent new files from being written to the drive as it is removed, but when drive removal gets to the end, it rescans the drive again until there are no files left to move.  So any new files would get moved off the drive during the rescan.

    Also, instead of disabling shares and/or reporting file conflicts at the end of the scan, it would make sense to (offer to) close all handles on that drive in a manner similar to what 'chkdsk /f' does.  That would break any remaining open handles without disconnecting all users from everything.

    I would love to use all SATA drives, but even so the removal would take 15+ hours, which is still a long time to go without any server access.  And I've maxed out my HP's SATA ports, so USB is all that's left for me.

    I guess I've really outgrown my hardware, so it's probably time to plunk down $3,000 on a new, more powerful system.  But that's not going to prevent shares from being disabled many, many hours at a time.  I do still think drive removal can be improved by not disabling shares.

    I also think that maybe this has bothered me more than it has others; WHS has saved my data in a way that RAID could never have done.  In the last month, I've suffered five drive failures (I think I had an electrical problem ;-), three of them simultaneously!  One of them was my system drive.  WHS's folder duplication made it possible to save ALL of my data despite three drives failing at the same time.  But it then took four days to remove those three drives, and I couldn't just do it overnight.  That was a long time to be offline.  What made it even worse, was that one of the external drives was totally dead and had to be removed post-mortem; that took even longer since all tombstones had to be checked (I have over a million files), and it made no sense to have backups or shares disabled during that time because the drive being removed wasn't even there to be accidentally written to.
    • Edited by Keith Mason Wednesday, June 17, 2009 3:25 PM bad link
    Wednesday, June 17, 2009 3:24 PM
  • Thanks for the link to the Connect feedback item.

    For what it's worth, I tend to agree that there would probably be ways to deal with most of the issues leaving the shares online could cause. And I think that it would be good if this could be added to the product.

    The one that probably can't be dealt with is the file held open for an extended period of time. That's very hard to track down unless you have a lot of familiarity with Windows file sharing and networking concepts, and it's not (realistically) something that Windows Home Server can deal with at all in it's present incarnation.

    I'm not on the WHS team, I just post a lot. :)
    Wednesday, June 17, 2009 3:52 PM
    Moderator
  • I definitely support this, even with the technical challenges behind it. Hope we get some smart engineers working on it
    Wednesday, June 17, 2009 6:50 PM
  • I think the file held open is actually pretty easy to solve in a way that is no less invisible to the user than it is now.  When you run 'chkdsk /f', the file handles for the entire drive are explicitly closed.  I haven't tried to reverse engineer chkdsk, but it would seem to me that WHS engineers could easily get ahold of that source code and force all handles on a drive closed during drive removal.  This has the same desired effect as deleting the share, but on a per-drive basis which is preferrable.

    Besides, files staying open for a long time are already reported via file conflict errors; you'd probably see this before you started to do the removal.  Perhaps deleting the share closes the file; perhaps not (invalidating handles definitely would).  WHS doesn't provide any user-friendly mechanism for tracking down the reason for file conflict errors other than to indicate that they are open, nor does it provide a way to force them closed.  So why should a file conflict during drive removal be any different?
    Wednesday, June 17, 2009 9:06 PM