Sign in
Microsoft.com
 
Microsoft
 
 
 

locked
Deal breakers follow-up RRS feed

 > 

  • General discussion

  • Discussion
    Sign in to vote
    1
    Sign in to vote

    In a previous discussion there were many speculative posts regarding the 1GB chunking of files spread across multiple hard drives with duplication turned off.  Somebody mentioned speculation was irrelevant (and I agree) without actual testing.  So to that end I have relentlessly tested the following configuration:  a single 160GB system drive, with the remaining free space removed from the pool, along with two 80GB drives for the server folders.  I created a new server folder with duplication off and copied approximately fifty, 2GB video files to the folder.  After simulating a drive failure (by removing the power from the drive with the server on) I can report the following:

     

    1. All non-duplicated server folders report an error in the alert viewer and are unavailable.  Alert viewer recommends "if you believe that the hard drive has failed, then remove it from Server Storage.

    2. Running the remove hard drive wizards lets you know that ALL non-duplicated folders will be deleted.  I could not run a folder repair without removing the failed drive from the pool so I did.

    3. After running repair server folder on all folders after removing the failed drive from the pool, they report as available in the dashboard, and can be accessed via their drive letters.  They cannot be browsed via the network however.  The only way I could get them back was to delete and recreate.

    4. There is no data in any of the non-duplicated shares.  This included the video files and also some smaller pics in the pictures share.  Any duplicated shares are unaffected as expected.

     

    So basically I lost ALL of the non-duplicated data with a single drive failure.  I have screen shots if need be.  Questions, comments, concerns?

    Friday, June 4, 2010 12:17 AM

All replies

  • Discussion
    Sign in to vote
    0
    Sign in to vote

    Post a bug on Connect. The issue with the folders disappearing as shares and not coming back is, I believe already bugged, but you shouldn't have lost everything the way you did.

    Your results don't match with mine, but I'm not in a position to do a more rigorous test any more; I've moved my Vail server to "production" mode. Maybe I was lucky?

    I'm not on the WHS team, I just post a lot. :)
    Friday, June 4, 2010 12:34 AM
    Moderator
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    I filed connect bug# 565183.  There is another bug that may be related, #564443 (Perhaps the bug your referred to?).  That post mentions only a specific share being unavailable but they do say the share doesn't appear in the permissions tab.  I have the same symptom of the shares not being listed in the 'allow access to' tab.
    Friday, June 4, 2010 7:40 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    Just as I expected.  Thanks for doing the test.
    Tuesday, June 8, 2010 10:36 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    FlyGuy -- note that MS is waiting on your log files.
    Thursday, June 10, 2010 3:47 AM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    I hope that this is addressed and a new release is made incorporating a fix for this.  Even a single instance of this, given extraordinary amount of time that MS had to take to deal with the V1 data corruption issue, would be enough to temper movement to V2.
    Saturday, June 19, 2010 4:46 PM
  • Discussion
    Sign in to vote
    1
    Sign in to vote

    Now that VMWare Player is no longer crippled, I decided to download it & do some testing of my own.  Here's what I found:

    1. installed Vail to a 160GB VHD
    2. added three additional 60GB VHDs to the storage pool, for a total of 4 drives.  (most of the WHS hardware I've seen, including my own Acer H340, has four HDD bays)
    3. created a new share with duplication turned off
    4. copied 12 uncompressed video files to the new share: 4 that were around (but strictly less than) 4GB, 4 that were 7-8GB, and 4 that were 18-20GB.
    5. removed one of the 60GB VHDs from the virtual machine

    At first nothing happened.  While I had the Vail desktop open in VMware, I didn't install the Vail client so I didn't see so much as a warning.  Once I opened the Dashboard on the server, I noticed a "critical error" box in the upper right; as you'd expect, clicking it reported a missing or failed drive.  Browsing to the "server folders and hard drives" page showed the same thing.  However, the filesystem was still reporting that nothing was wrong.  Explorer on my physical host, Explorer inside Vail, and even Powershell inside Vail all continued to show 12 files.  WMP on the physical host could even play them back.

    Then I tried skipping around inside some of the larger files.  Sure enough, I could make WMP crash on certain sections of video with 100% reliability.  I was about to break out an MD5 tool to see exactly where the missing pieces lay, but it turns out Vail will do the work for you.  Simply walking thru the "remove drive" wizard on the will create a report of files that will be permanently lost if you complete the removal process.  Unfortunately, the dialog is constrained to showing the first 5, and the complete report it links you to on http://yourserver/blah.html is a not-very-human-readable log file.  So I clicked Finish on the wizard and went back to Powershell.  Results:

    • Small files - theory predicts 25% will be lost - 0/4 were lost.
    • Medium files - theory predicts 44% will be lost - 1/4 were lost.
    • Large files - theory predicts 68% will be lost - 4/4 were lost.

    All in all, I'd say the theory* held up darn well, especially considering the tiny sample size.

    For the record, I plan on upgrading to Vail very soon anyway.  The other advantages of the new storage architecture (modern OS core; performance; CRC correction against bit rot) are too important to ignore.  If drive failure keeps me up at night, well, that's just more incentive to edit & compress my video backlog more quickly.  Or I can just throw hardware at it.

    Also, I did not encounter any unexpected behavior / bugs.  The surviving videos have no errors / missing chunks.  Duplicated shares were unaffected, as were shares with smaller files.

    *example calculation: an 18GB file requires 4 chunks.  If we assume chunks are randomly and independently distributed, then each has a 25% chance of living on the drive I yanked.  So the chance an 18GB file survives is (1-0.25)^4 = 31.6%.  I have a few TB of uncompressed video files ranging up to 100GB each.  Such a file would survive less than 1 in 1000 drive failures, versus 1 in 4 on WHSv1.

    Monday, July 5, 2010 5:07 AM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    ...
    For the record, I plan on upgrading to Vail very soon anyway.
    ... 
    *example calculation: an 18GB file requires 4 chunks.  If we assume chunks are randomly and independently distributed, then each has a 25% chance of living on the drive I yanked.  So the chance an 18GB file survives is (1-0.25)^4 = 31.6%.  I have a few TB of uncompressed video files ranging up to 100GB each.  Such a file would survive less than 1 in 1000 drive failures, versus 1 in 4 on WHSv1.

    Two things:

    1. Bear in mind that Vail is in beta. Use as production software isn't supported or recommended. Nobody can tell you when Vail will reach the RTM milestone, but I predict it won't be much before the end of the year based on the V1 timeline.
    2. DE data blocks are documented elsewhere in the forum as being 1 GB, not 4.

    I'm not on the WHS team, I just post a lot. :)
    Monday, July 5, 2010 12:14 PM
    Moderator
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    Well in that case I was extremely lucky to not lose any 8GB files.  Or more likely, Vail is tuned to keep chunks from the same file / same share near each other.

    I've been using WHS since it was known only as "Project Q" without irrevocable data loss.  Suffice to say I'm already more comfortable with Vail than I was with any version of Q up to PP2.  I just need to buy more storage, that's all :)

    Monday, July 5, 2010 7:25 PM
  • Discussion
    Sign in to vote
    1
    Sign in to vote

    Or more likely, Vail is tuned to keep chunks from the same file / same share near each other.

    That's what I hope will come out of this whole "discussion" about the new idea of striping files >1GB... as long as files have a bigger chance to be distributes round-robin style than to be just stuffed as tightly as possible together (I wouldn't care to have the server defragmenting or "reorganizing" it's files once a week automatically *hint* *hint*) that's still a definite dealbreaker for me.

    Just look at you case:

    You hat a 60GB HDD loss and lost 4*18-20GB + 1*7-8GB = nearly 1,5 times the amount of files that could even physically fit on this HDD (not taking into account that you loose 12% space now anyways due to checksumming, which is more or less fine for me...). If the server would both on write and then afterward at maintenance really try to fit all chunks of 1 file together on one HDD, you would lose max. the files of 1 HDD + 1 file - this is somehow a risk that seems more reasonable to me than the current situation, observed by you.

    Monday, July 5, 2010 11:19 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    So in conclusion Vail, by design, is only viable if ALL shares are duplicated when large files are concerned.

    Why is duplication even a user option given this design?...it should be permanently switched on.

    This makes it a definite deal breaker for me - notwithstanding the need to build a 40TB server, the additional power costs, replacement UPS and overall TCO is prohibitive when my existing equipment is more than sufficient.  And to be honest Vail seems to be very much of a technology upgrade, akin to Vista, where most of the changes are either cosmetic or under the skin, not really compelling new "killer" technology for the end-user, certainly not for me.

     

    Wednesday, July 14, 2010 11:23 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    the joe and jane sixpack home user
     
    1. will have a prebuilt WHS1 that can't run 64 bit and therefore can't be
    upgraded
     
    2. probably won't even know that a new version is available
     
    3. won't be able to purchase an upgrade version of Vail.
     
    On Wed, 14 Jul 2010 23:23:34 +0000, PMFranklin wrote:
     
    >And to be honest Vail seems to be very much of a technology upgrade, akin to Vista, where most of the changes are either cosmetic or under the skin, not really compelling new "killer" technology for the end-user, certainly not for me.
     
    Barb Bowman

    http://www.digitalmediaphile.com

    Thursday, July 15, 2010 9:05 AM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    the joe and jane sixpack home user
     
    1. will have a prebuilt WHS1 that can't run 64 bit and therefore can't be
    upgraded
     
    2. probably won't even know that a new version is available
     
    3. won't be able to purchase an upgrade version of Vail.
     
    On Wed, 14 Jul 2010 23:23:34 +0000, PMFranklin wrote:
     
    >And to be honest Vail seems to be very much of a technology upgrade, akin to Vista, where most of the changes are either cosmetic or under the skin, not really compelling new "killer" technology for the end-user, certainly not for me.
     
    Barb Bowman

    http://www.digitalmediaphile.com


    1.  What features will compel joe and jane to go out and buy a new Vail box off the shelf - why do you assume I'm talking about upgrades?

    2.  They found out about WHS1, so why wouldn't you expect them to find out about Vail - you think there will be less marketing for Vail than WHS1?

    3.  I never mentioned upgrades, not for Joe and Jane, not for enthusiasts - so why mention this?

    I see a lot of reference to "Joe and Jane" on this forum.  I'd like to see some hard data, because where I live WHS has never been seriously marketed either by MS or OEMs, and I suspect most users are more knowledgable than we give them credit for in order to choose WHS in the first place.  The real "Joe and Jane" users I suspect have not looked beyond external USB drives yet.

    In any event that's a whole separate discussion, as it adds nothing to the main point I was raising which is that Vail seems to have been designed to run with duplication on, otherwise to reduce the risk to the same levels as WHS1  for large non-duplicated files, the TOC increases significantly.

    Thursday, July 15, 2010 9:53 AM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    "Joe and Jane six pack": You can buy an HP MediaSmart Server at local "big box" stores in the US. That should tell you all you need to know about the primary audience for Windows Home Server. Nobody has released hard sales numbers for Windows Home Server (not that I've seen, anyway, and I have looked), but I would guess that it's in the high tens (maybe very low hundreds) of thousands of units, with most of those being OEM servers. As for marketing, when Windows Home Server was introduced (in 2007) it was an end cap in every Best Buy at Christmastime. You don't need more than that, though Microsoft and HP did take out ads, and provide press releases, through the months leading up to initial availability. :)

    Upgrades: the average user won't. They bought a Windows Home Server "appliance", and they expect it to sit in a corner, neglected, and work for years. (Which it will; my production server is an EX47 series, and I've never had a problem with it.) Enthusiasts will upgrade, but they're a small piece of the pie.

    And yes, you're correct. Vail (like V1) is designed to be used with duplication turned on for all shares.

    I'm not on the WHS team, I just post a lot. :)
    Thursday, July 15, 2010 12:36 PM
    Moderator
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    "And to be honest Vail seems to be very much of a technology upgrade, akin to
    Vista" - open to interpretation of what you really meant.
     
    On Thu, 15 Jul 2010 09:53:54 +0000, PMFranklin wrote:
     
    >3.  I never mentioned upgrades, not for Joe and Jane, not for enthusiasts - so why mention this?
     
    Barb Bowman

    http://www.digitalmediaphile.com

    Thursday, July 15, 2010 12:48 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    "And to be honest Vail seems to be very much of a technology upgrade, akin to
    Vista" - open to interpretation of what you really meant.
     
    On Thu, 15 Jul 2010 09:53:54 +0000, PMFranklin wrote:
     
    >3.  I never mentioned upgrades, not for Joe and Jane, not for enthusiasts - so why mention this?
     
    Barb Bowman

    http://www.digitalmediaphile.com


    I'll take your word for it, though it's pretty clear I'm talking about features and not an inplace upgrade to existing WHS'.  For clarity, Technology Upgrade = new platform = move to Windows Server 2008 codebase.  In context I'm referring to the move MS are making with the WHS platform, and not the ability or inability to upgrade in place.  As with Vista, major changes were made but necessarily visible, or compelling enough to the user.

    Would you care to answer the other 2 questions put to you?

    Thursday, July 15, 2010 1:14 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    "Joe and Jane six pack": You can buy an HP MediaSmart Server at local "big box" stores in the US. That should tell you all you need to know about the primary audience for Windows Home Server. Nobody has released hard sales numbers for Windows Home Server (not that I've seen, anyway, and I have looked), but I would guess that it's in the high tens (maybe very low hundreds) of thousands of units, with most of those being OEM servers. As for marketing, when Windows Home Server was introduced (in 2007) it was an end cap in every Best Buy at Christmastime. You don't need more than that, though Microsoft and HP did take out ads, and provide press releases, through the months leading up to initial availability. :)

    Upgrades: the average user won't. They bought a Windows Home Server "appliance", and they expect it to sit in a corner, neglected, and work for years. (Which it will; my production server is an EX47 series, and I've never had a problem with it.) Enthusiasts will upgrade, but they're a small piece of the pie.

    And yes, you're correct. Vail (like V1) is designed to be used with duplication turned on for all shares.

    I'm not on the WHS team, I just post a lot. :)

    It's a different situation in my country, with little or no marketing, and little to no stock from major OEMs.  In any event I still would expect the majority of WHS users to have a certain level of knowledge to actually purchase one in the first place.  As already mentioned, I'm not talking about WHS in-place upgrades, but Ken, please answer my other question, what features in Vail will compel WHS1 owners to go buy a new WHS2 appliance?

    WHS1 may be designed to be used with duplication turned on for all shares, and it may even be recommended, but it certainly is not mandatory or results in an unsupported configuration.  But it is also designed to run from a single HD which does not allow duplication. If I am wrong, can you point me to the documentation confirming this, and also advise why, if this is the case, MS have chosen to allow the user to switch duplication on/off from within the dashboard?

    I am quite frustrated that this loosely worded phrase crops up regularly when we are talking about this whole striping situation for large files.   Users who choose not to duplicate are not misusing their servers as I feel is being implied, but are concerned that the level of risk posed by using Vail is significantly greater than in WHS1.

    It seems very few from MS or the MVPs cannot accept that people use their servers in this way or that Vail increases the risk of data loss when storing large files without duplication on when compared to a WHS1 server.  My opinion is that this is a backward step and it's a deal breaker for me.  It would just be nice for the increased risk to be acknowledged and not swept under the carpet as some "unsupported scenario".

    Thursday, July 15, 2010 1:47 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    As I thought I made pretty clear in the post you quoted ("... Upgrades: the average user won't. ..."), I don't see anything in Vail today that would compel an existing "Joe Average" user of V1 to buy a new server. (There's lots that might encourage him to do so, however. :) ) As for duplication, I'm also pretty clear on that, all over the forums: it's recommended, not required. The documentation does make clear, however, that you risk all the data in any share that isn't duplicated.

    Drive failure/duplication: The risk to "Joe Average's" data with Vail and V1 is effectively identical. If you turn off duplication for a share, you risk the loss of 100% of the data in that share in the event of a drive failure. If you don't take regular backups of your server, and take those backups off-site, you risk total loss if you experience a force majeure event. It's your choice; if you can afford to lose the data in a share, turn off duplication. If you're okay with starting from scratch, don't take backups.

    Personally, I consider Vail's Drive Extender functionality a significant improvement over V1. It does away with many of the anomalies and irritations of the V1 Drive Extender, at the cost of an addidtional 12% overhead for the CRC checking (which was undoubtedly added because Microsoft saw a significant number of issues with torn writes, bit flips, etc. even if they were never identified as such publicly). Data loss due to drive failure or force majeure event is a non-issue for me because I use duplication and take backups off-site regularly.

    I'm not on the WHS team, I just post a lot. :)
    Thursday, July 15, 2010 5:34 PM
    Moderator
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    So in other words Vail is really targeting an extremely small set of user... right? I would have thought that they would develop Vail to fill the void in sales that V1 failed to do. I bought two WHS disks at about $170-180 each, thinking I would install one and give one to someone I know that might want to give it a try. I couldn't give the second one away (I still have it.. unopened)... no one I know was even vaguely interested in it. WHS v1 was never a "joe and jane sixpack home user" product, and it does not look like Vail will be either.

    Art [artfudd] Folden
    --
    "Ken Warren [MVP]" wrote in message news:ccf02e6d-3e2d-4b25-9047-5310f1a7e668@communitybridge.codeplex.com...

    "Joe and Jane six pack": You can buy an HP MediaSmart Server at local "big box" stores in the US. That should tell you all you need to know about the primary audience for Windows Home Server. Nobody has released hard sales numbers for Windows Home Server (not that I've seen, anyway, and I have looked), but I would guess that it's in the high tens (maybe very low hundreds) of thousands of units, with most of those being OEM servers. As for marketing, when Windows Home Server was introduced (in 2007) it was an end cap in every Best Buy at Christmastime. You don't need more than that, though Microsoft and HP did take out ads, and provide press releases, through the months leading up to initial availability. :)

    Upgrades: the average user won't. They bought a Windows Home Server "appliance", and they expect it to sit in a corner, neglected, and work for years. (Which it will; my production server is an EX47 series, and I've never had a problem with it.) Enthusiasts will upgrade, but they're a small piece of the pie.

    And yes, you're correct. Vail (like V1) is designed to be used with duplication turned on for all shares.

    I'm not on the WHS team, I just post a lot. :)

    Thursday, July 15, 2010 5:59 PM
  • Discussion
    Sign in to vote
    1
    Sign in to vote

    Ken, you're totally missing the point, or choosing to gloss over it, as the two OSs are not effectively identical.  The 100% risk of loss on WHS1 is only applicable if the share is smaller than the size of a physical disk.  A number of users are storing large movie collections, where the share spans the size of a physical disk.  In my own situation I have the original disks, so by not duplicating I am prepared for having to re-rip the contents of the failed HD.  With Vail it would be my entire collection.  Of course media I cannot replace is duplicated with a robust backup strategy in place, so we don't need to discuss that element further here.

    I have seen your views elsewhere on that duplication should not be an option, fine, I respect that opinion.  But that's not the product I bought into with WHS1, or from the beta, what I have seen in Vail, as both allow non-duplication, and until such time as it is not an option the differences between WHS1 and Vail should be allowed to be discussed.  I'm not sure why you have to be so dismissive of the situation, just because it is not the way you use your WHS, or at a level of risk you are prepared to accept.

    This is how the two OSs behave;

    o - WHS1 share - non-duplicated - containing large files (1GB+) would put at risk those files on the drive that failed (not necessarily the entire share).
    o - Vail share - non-duplicated - containing large files (1GB+) would put at risk all those files that had part of the file on the drive that failed.

    As an example, user has a 8TB WHS (Average Joe with a 4 disk OEM unit), made up of 4x2TB HDs.  Duplicated data is 2TB, Non-duplicated data is 6TB.  A single disk fails.

    o - Obviously both WHS1 and Vail will protect the duplicated data.
    o - On WHS1, the non duplicated data on the failed drive is lost.  A maximum of approx 2TB data.
    o - On Vail, the non duplicated data on the failed drive is lost, but as they are 1GB chunks of data, potentially all the 6TB of non duplicated data is lost, as demonstrated above.

    So in summary, instead of putting at risk 2TB of data under WHS1 by turning off duplication, in this example the user has 6TB of data at risk. 

    So this behaviour is absolutely not identical in both versions of the OS.  Vail is a retrograde step and puts more data at risk period.

    Thursday, July 15, 2010 6:28 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    I know how the two versions of Windows Home Server behave, probably better than you do. And I understand your point: you like the file based DE V1 better, because of your mix of data.

    I'm sorry if you don't like hearing this, but users with a single 6+ TB share are a tiny percentage of all Windows Home Server users. This is based on dozens of users that I've spoken to, and the ratio is about 4-1 for "Joe Average". And most of the enthusiasts don't have a single huge share, just lots of data overall. For the average user, there's no apparent difference between DE V1 and DE V2; you can lose all of a share if that share isn't duplicated and a disk fails. In the end, that's all the user cares about. It's an unfortunate fact of software design in this space that you don't design for the 1%, you design for the 99%. If you can do something to help out the 1% you do, otherwise they need to figure out a strategy to mitigate their risks. In this case, the pain of DE V1 for the majority vastly outweighs the pain of DE V2 for the minority.

    Now, I would love it if Microsoft could tweak DE to decrease the risk people are worrying about (even if my own testing didn't reveal that risk as being actual, rather than theoretical). I just don't think it's a good investment of resources to do so. Particularly not at this stage of the game; a DE rewrite now could result in an additional delay of several months, and I already don't expect general availability for Vail much before the end of the year (if then).

    I'm not on the WHS team, I just post a lot. :)
    Friday, July 16, 2010 2:55 AM
    Moderator
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    I don't like DEv1 better.  It has a lower risk profile than Vail in the scenario identified.  To get to the crux of it, my point would be that I would expect Vail to have the same or lower risk profile than the product it is replacing.  The key is that Vail is a backward step.  I would like to migrate to Vail in the future, but my useage and risk profile would prevent me from doing so without incurring substantial costs.  I have not made any demands or acted petulantly.

    We are not talking about resource allocation, priorities or marketing.  It's irrelevant to the discussion between us as that is Microsoft's decision, not ours.  We are talking about an issue, and it is then up to Microsoft to decide whether they will do anything about it or not.   But let's not forget one of the examples cited for the reason for this design was that _one_ user had tried to store a single 400GB file.  This user wasn't average, this user wasn't the majority.

    It's for MS to decide whether the concern is important enough for them to make any changes, and assess the impact to delivery.  Again it's not something that is important to this discussion other than for our own selfish reasons.  My understanding was that these forums were provided to give ALL our feedback, observations and concerns on the Vail preview.  Not for us to decide between us on the decisions MS should make.  Similarly please let us know if this feedback is not welcomed because it only affects an alleged small number of users (which may incidentally grow significantly during Vail's lifetime) or meet your opinion of what Vail should and should not do, and we can stop wasting our time.

    Friday, July 16, 2010 8:33 AM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    "It's an unfortunate fact of software design in this space that you don't design for the 1%, you design for the 99%."

    Not only unfortunate IMO, but a near-sighted "fact of software design". If the 1% were catered to - to any kind of reasonable degree - then that 1% might increase to 2% or 5% or 10%... etc.

    Sure you will now likely site costs as an excuse... and perhaps a valid excuse to some degree, but when an OS is in development anyway, some things of this nature added to the mix should not significantly increase the cost of development. But... if the developers intent is to only reach a very small segment of the general public to begin with (which seems to be the case), then I suppose they would not be far-sighted enough to consider the 1% or so.. :)

    Art [artfudd] Folden
    --
    "Ken Warren [MVP]" wrote in message news:0510d442-f3e7-4484-b54c-5ef2c0ce5a58@communitybridge.codeplex.com...

    I know how the two versions of Windows Home Server behave, probably better than you do. And I understand your point: you like the file based DE V1 better, because of your mix of data.

    I'm sorry if you don't like hearing this, but users with a single 6+ TB share are a*tiny* percentage of all Windows Home Server users. This is based on dozens of users that I've spoken to, and the ratio is about 4-1 for "Joe Average". And most of the enthusiasts don't have a single huge share, just lots of data overall. For the average user, there's no apparent difference between DE V1 and DE V2; you can lose all of a share if that share isn't duplicated and a disk fails. In the end, that's all the user cares about. It's an unfortunate fact of software design in this space that you don't design for the 1%, you design for the 99%. If you can do something to help out the 1% you do, otherwise they need to figure out a strategy to mitigate their risks. In this case, the pain of DE V1 for the majority vastly outweighs the pain of DE V2 for the minority.

    Now, I would love it if Microsoft could tweak DE to decrease the risk people are worrying about (even if my own testing didn't reveal that risk as being actual, rather than theoretical). I just don't think it's a good investment of resources to do so. Particularly not at this stage of the game; a DE rewrite now could result in an additional delay of several months, and I already don't expect general availability for Vail much before the end of the year (if then).

    I'm not on the WHS team, I just post a lot. :)

    Friday, July 16, 2010 6:55 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    "...expect Vail to have the same or lower risk profile than the product it is replacing."

    I agree, but MS is notorious for NOT making a replacement product 'better' (in all ways and features) than that which it replaces. Of note is Windows Explorer from OS edition to edition, Outlook Express to Windows Mail, Windows Mail to Windows Live Mail, etc., right through most all of the 'Live" products, and now from WHS v1 to WHS v2. I don't expect that to ever change, and as a result MS will lose (and is losing) more and more of their market share as time passes. It is a sad fact with MS, especially since Bill Gates left.

    Art [artfudd] Folden
    --
    "PMFranklin" wrote in message news:1787b582-cb79-4fdc-9510-6813d968e8d9@communitybridge.codeplex.com...

    I don't like DEv1 better.  It has a lower risk profile than Vail in the scenario identified.  To get to the crux of it, my point would be that I would expect Vail to have the same or lower risk profile than the product it is replacing.  The key is that Vail is a backward step.  I would like to migrate to Vail in the future, but my useage and risk profile would prevent me from doing so without incurring substantial costs.  I have not made any demands or acted petulantly.

    We are not talking about resource allocation, priorities or marketing.  It's irrelevant to the discussion between us as that is Microsoft's decision, not ours.  We are talking about an issue, and it is then up to Microsoft to decide whether they will do anything about it or not.   But let's not forget one of the examples cited for the reason for this design was that one user had tried to store a single 400GB file.  This user wasn't average, this user wasn't the majority.

    It's for MS to decide whether the concern is important enough for them to make any changes, and assess the impact to delivery.  Again it's not something that is important to this discussion other than for our own selfish reasons.  My understanding was that these forums were provided to give ALL our feedback, observations and concerns on the Vail preview.  Not for us to decide between us on the decisions MS should make.  Similarly please let us know if this feedback is not welcomed because it only affects an alleged small number of users (which may incidentally grow significantly during Vail's lifetime) or meet your opinion of what Vail should and should not do, and we can stop wasting our time.

    Friday, July 16, 2010 7:05 PM
  • Discussion
    Sign in to vote
    1
    Sign in to vote

    This, however, is maybe Microsoft's biggest problem in the last ten years.  "Nobody does it that way, so why bother developing it"  because of that philosophy, Microsoft has left itself open to people who innovate with a forward thinking viewpoint instead of always looking into the rear view mirror.  MS wasn't first with digital audio tools, they copied.. Zune after iPod.  The Windows Phone solution is still a quagmire.  (Kin was an utter disaster) and most of those things have been a complete boondoggle for investors and other technology people because they were designs based completely on rear-view-window methodology  "this is how people work"   "Nobody has that much data".   That's kind of a ridiculous viewpoint.  When XP was introduced "Nobody" had 2GB of RAM.  Now it's the floor.  Nobody had a HDD bigger then 128G (LBA1) .. now 2TB.


    Microsoft has to start looking forward with it's products a little bit better.  They can't say "well, this is what we've discovered" they have to offer something in the way of unique innovation that says to the end user: we know you didn't think about it this way, but believe us, in 2 years, this is the way you will think of it.  That's called leading the technology forefront.  While I love the concept of Aurora, with a Small Business based WHS w/Activedirectory and cloud services, it is mostly a response to others getting their first.


    Here's the reality.  Microsoft has more products on the market that should COULD connect to WHS.  From XBOX's to Media Centers.  Cable card tuners drop this year.  A single HD recording in BEST Quality in Windows7 is 8.2GB an hour.  Record a movie?  That's up to 18.4GB (at least in mine)   How spread across drives is that?

     

    This idea that "nobody does it" is something that is a ridiculous viewpoint for MS to take.  Nobody does it today, but on the other front, Microsoft is busy trying to get people to adopt MediaRoom and MediaCenter and how much data are they storing? A ton.

    MS could come up with incredible features too.. the XBOX now lets you store your discs to it's internal HDD.  I'm sure it's possible to get an XBOX to connect to a WHS with a right upgrade to the XBOX.. so where is that?  Allowing them to store their media on a WHS?  And so on.

    I admit, I guess I'm in that 1%.  I have more then 18TB in my WHS.  I will probably upgrade again soon.  I backup every machine in my house to there.  Store my music, TV, pictures and all of my software a lot of which I purchase online (and now that Microsoft is having us download MAPS and TechNet, etc in ISOs, guess where all those 4G+ files are being stored??) 


    Joe Sixpack in general isn't buying WHS.  Joe Sixpack is buying the cheapest computer available and if something bad happens, it happens.  HP is busy marketing WHS to small and home businesses who are concerned with the same thing I am.  So is Acer and Asus.  So, if all of Microsoft's OEM partners have managed to catch on and are busy marketting this as a data storage pool for a home office and are providing massive storage, then why isn't MS catching onto that?

    I can walk into Microcenter right now and buy an Iomega 4TB NAS.  I can buy and 8TB NAS straight off the shelf.  And even in JBOD, if one disc fails, I keep everything that isn't on that disc. 

    So, is the point to basically say: if you have more data then 4TB, give up, go buy someone else's NAS or buy a different product?  Because Vail is slowly moving itself into a territory where I'd only put small data on it or have smaller drives.  There is no way I'd hang 8TB on a Vail server as it is currently positioned.  Not even remotely. 


    But I suppose that's ridiculous and Joe Sixpack will never have that much data or ever worry about it.  640k is still more then enough for them.


    If MS doesn't decide to be innovators and work to offer next generation ideas, thinking ahead instead of behind, Vail will be the equivelent of Kin.  And someone else will roll out a home storage server that users will trust more because it has lower risks.  And people will wonder what the ____ MS was doing when a single recording from Media Room / Media Center split across all sorts of drives, and then a drive fails and all their recordings are messed up.

    I think the train may be too far.  And MS may be too committed to this new drive technology.  I've tested as above, and I will tell you, not a single file over 1GB survived in a 4-drive configuration.  Not a single one.   I can't even record a 1/2 hour show in HD and be under 1G.  I can't even record SD at that rate without going to low quality.  So, by default, no files created in the default format from Microsoft's own media center are at all protected in any way as sits.

    That strikes me as brilliant strategy.

    Monday, July 19, 2010 5:11 AM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    I'm going to have to agree with you there.  The average user will not ever buy a Windows Home Server.  The power user and small business user are the majority of users and you are now kicking us in the junk.  Forcing redundancy on every share is an unnecessary burden and not one I'm willing to take so I will be forced to move elsewhere or stick with WHS v1.
    Monday, July 19, 2010 9:56 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    "It's an unfortunate fact of software design in this space that you don't design for the 1%, you design for the 99%." This thought process shows why Apple is so succesful in the consumer space, and why a large percentage of Microsoft employees are walking around campus with iPhones at the moment.

    Most consumers have *no idea* that they need a Windows Home Server, and certainly no idea about what they could do/accomplish with one. You need to show them, by designing an innovative, competitive product that *just works*. Do you really want the likes of Consumer Reports doing a comparison of Home Servers this winter, and pointing out that WHSv2 only provides half the storage space of competing models when using redundancy? That *not* using redundancy on all the family camcorder footage, or backed up kids DVDs means that losing one drive blows all your data away? Standard consumer thought process for the low end, 4-drive space..will I want to buy the 4TB HP WHS box? Or for the same, or cheaper price, would I want the 6TB competing server? 50% more storage for the same money

    And, as for the technical personna..do you really thing we're going to trust our many TB of data to a brand new, software RAID0 technology, on a product where a previous data corruption issue took over a YEAR to fix?

    I like home server, a lot. I loved the concept when it came out, I stuck with all the issues in V1, and have tested WHS on everything from a small 4 drive box, to a dual socket server hooked up to an FC SAN, to a Hyper-V VM. With Vail, I'll have the option of not worrying about DEv2 by using a proper hardware RAID6 controller (they run around ~$500 these days, far cheaper than the many extra drives and SATA ports I'd need to store my data with the massive DE2 overhead). Most people however, won't have that option

    Wednesday, July 21, 2010 1:21 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    "It's an unfortunate fact of software design in this space that you don't design for the 1%, you design for the 99%." This thought process shows why Apple is so succesful in the consumer space, and why a large percentage of Microsoft employees are walking around campus with iPhones at the moment.

    Most consumers have *no idea* that they need a Windows Home Server, and certainly no idea about what they could do/accomplish with one. You need to show them, by designing an innovative, competitive product that *just works*. Do you really want the likes of Consumer Reports doing a comparison of Home Servers this winter, and pointing out that WHSv2 only provides half the storage space of competing models when using redundancy? That *not* using redundancy on all the family camcorder footage, or backed up kids DVDs means that losing one drive blows all your data away? Standard consumer thought process for the low end, 4-drive space..will I want to buy the 4TB HP WHS box? Or for the same, or cheaper price, would I want the 6TB competing server? 50% more storage for the same money

    And, as for the technical personna..do you really thing we're going to trust our many TB of data to a brand new, software RAID0 technology, on a product where a previous data corruption issue took over a YEAR to fix?

    I like home server, a lot. I loved the concept when it came out, I stuck with all the issues in V1, and have tested WHS on everything from a small 4 drive box, to a dual socket server hooked up to an FC SAN, to a Hyper-V VM. With Vail, I'll have the option of not worrying about DEv2 by using a proper hardware RAID6 controller (they run around ~$500 these days, far cheaper than the many extra drives and SATA ports I'd need to store my data with the massive DE2 overhead). Most people however, won't have that option

    I agree with you on this, DEv2 cant complete in any way with a Raid5-based setup now due to the data loss that can happen.
    Wednesday, July 21, 2010 2:31 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    So in summary, instead of putting at risk 2TB of data under WHS1 by turning off duplication, in this example the user has 6TB of data at risk. 

    So this behaviour is absolutely not identical in both versions of the OS.  Vail is a retrograde step and puts more data at risk period.

    This is an interesting thread - I hadn't realised the risk implications of non-duplicated storage!

    Presumably one could store files that one didn't want duplicated on individual drives outside of the storage pool and therefore those drives would be just normal NTFS drives. I have done just this with WHS V1 as I was fed up with getting random tombstone errors. So now I just duplicate files that would be a lot of effort to recreate.

    If one did this (use individual drives outside of the storage pool) is it still possible to specify folders on these drives as WHS shares?

    Friday, July 23, 2010 12:26 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    So in summary, instead of putting at risk 2TB of data under WHS1 by turning off duplication, in this example the user has 6TB of data at risk. 

    So this behaviour is absolutely not identical in both versions of the OS.  Vail is a retrograde step and puts more data at risk period.

    This is an interesting thread - I hadn't realised the risk implications of non-duplicated storage!

    Presumably one could store files that one didn't want duplicated on individual drives outside of the storage pool and therefore those drives would be just normal NTFS drives. I have done just this with WHS V1 as I was fed up with getting random tombstone errors. So now I just duplicate files that would be a lot of effort to recreate.

    If one did this (use individual drives outside of the storage pool) is it still possible to specify folders on these drives as WHS shares?


    They can be shared with systems as you can make manual shares but they will not be shown/accessible from the WHS website.
    Friday, July 23, 2010 2:03 PM
  • Discussion
    Sign in to vote
    1
    Sign in to vote

    Vail has simply moved to using a chunk based file system. There's benefits to that kind of file system and there are negatives to that kind of file system. It's just a software design decision. It's better for some, worse for others.

    I've personally switched to Fedora running Grayhole which is a program that offers basically what DEv1 did. A file (rather than chunk) based JBOD concatenation pool with the option of duplication. It'd be great if there were such a program available for Windows operating systems, unfortunately the only Windows OS that offers it is WHSv1. But that's also why Greyhole was created, it was something that WHSv1 offered that wasn't available in Linux, so someone created it. Now the shoe is on the other foot.

    Hopefully the file based method of JBOD concatenation will return to Windows OSes at some point. It just doesn't appear that will happen in Vail.

    Friday, July 23, 2010 9:57 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote

    >> They can be shared with systems as you can make manual shares but they will not be shown/accessible from the WHS website.

    Some more caveats.  Probably obvious, but stating for the record, since I may need to adopt this strategy myself at some point...

    * if you wanted to give a manual share one of the "standard" names (eg \\Server\Videos), you'd have to delete the predefined share from the WHS console.

    * there wouldn't be an easy way to assign permissions to the user accounts managed by WHS.  I'm sure WHS accounts are associated with Windows accounts under the hood, but last time I checked (v1) the implementation was not as trivial as you'd think.  At minimum, truly mimicking WHS would require crawling some registry keys.  Any such mechanism would be subject to breakage at any time since MS does not guarantee back-compat at this level.

    * sharing manual folders with a HomeGroup would mostly alleviate problem #2, assuming all of your clients are running Win7 or later, but might pose problems of its own.  I'd have to try it.

    Saturday, July 24, 2010 2:24 PM
  • Discussion
    Sign in to vote
    0
    Sign in to vote
    All you need to realize that you can't stereotype an "average user"  There is no such thing.  So stop doing that and you might find some better solutions.
    Saturday, July 24, 2010 8:47 PM
  • Discussion
    Sign in to vote
    1
    Sign in to vote
    All you need to realize that you can't stereotype an "average user"  There is no such thing.  So stop doing that and you might find some better solutions.

    Perhaps not, but there is certainly a probability distribution.  Integrals under that curve predict revenue.  Without revenue you can't pay your developers to finish all the great features I'm sure they're planning for WHS v3.  

    Sorry to be pedantic, but business fundamentals really do matter.  Even at Microsoft (usually ;)).

    Sunday, July 25, 2010 12:34 AM