locked
Vail Deal Breakers\Complaints RRS feed

  • General discussion

  • 1.  Where is the PIE CHART?    This is a quick and easy way to see overall storage capacity, free space, duplication info, etc.    This is an extremely useful graphic!  I am sure somebody can write an add-in for this, but this should be a default part of the "Dashboard".   Additionally the new status icons are way too small.

    2.  No NTFS Formatting of the drives?    The fail-safe of knowing you can always just connect a drive to any windows box in a pinch cannot be overlooked.   Nobody wants proprietary formats that lock the user in MS!

    3.  10 Drive Limit?   I currently have 14 drives in WHS v1.

    These are really surprising omissions I can only hope these are addressed before GA.     WHS v1 PP3 is working so well for me I may just stay on it if this is the direction of this product.

     

    Tuesday, April 27, 2010 12:23 AM

All replies

  • 2 and 3 are a deal breaker for me. I sure hope they work something out in regards to this. 
    Tuesday, April 27, 2010 12:24 AM
  • 1) The 'pie chart' is being implemented, that's one of the things that is just not ready yet.

    2) Technically speaking, NTFS locks the user in MS as well :) We understand the importance of the described scenario and are thinking about ways to make it easier. That said, similar technologies (e.g. BitLocker and all kinds of RAID) often don't allow seamless mounting on other platforms but customers are actively using and are reasonably happy with them.

    3) 14 shouldn't be a problem in most cases. We don't prevent the user from adding more than 10 drives, it's just not stable enough in this beta release. We're testing on machines with more than 10 disks at the moment and are seeing occasional bugs, which we need to fix before raising our official limits. The number of disks is less important than the combined capacity of all disks though.


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 1:35 AM
  • Regarding #1: The pie chart took up a lot of space, and the information wasn't that useful, to be honest.

    Regarding #2: I've just submitted this product suggestion on Connect. Please vote it up. :)

    Regarding #3: Go buy fewer, larger drives. They'll use less power, run quiter, and last longer (probably). But there is no hard stop built into the product to prevent you from adding more, it's just not a number that Microsoft seems to plan to support.

    You can certainly stay on Power Pack 3 (or future updates; I'm sure they will happen) if you choose. I'm already running Vail at home, and will be moving it to production in a week or two in defiance of my own warnings. :)


    I'm not on the WHS team, I just post a lot. :)
    Tuesday, April 27, 2010 1:41 AM
    Moderator
  • I (and many other people) run FlexRaid on WHS v1. It is a snapshot software RAID4. I absolutely need to have real-time access to individual data drives. Just like in WHS v1 I can access C:\fs\<X> drives.
    I understand it is not possible in WHS v2. Is it correct? If yes, it is a total deal-breaker for me.
    Tuesday, April 27, 2010 2:49 AM
  • @Ken, I've GOT "bigger" drives, and ten is not enough    BTW I did not read this the way you did, I read it as they are only supporting 10 in the Beta, what they "plan" hasn't been stated

    @BulatS - I sure hope DE is fixed in this version, and doesn't decide to do its thing when the drive is being accessed, like kicking on when a video is being streamed and totally jacking it up - you know what I'm talking about ;)

    Tuesday, April 27, 2010 3:40 AM
  • You can connect more than 10 drives, and add them to the storage pool. It's "unsupported", probably for reasons stated elsewhere, but it works.
    I'm not on the WHS team, I just post a lot. :)
    Tuesday, April 27, 2010 3:45 AM
    Moderator
  • I am not familiar with FlexRaid, but from what I gather it operates on top of a file system. Drive Extender v2 is a volume driver that sits *under* the file system. In other words, on a running Vail machine Drive Extender presents its data as regular NTFS volumes (each share, e.g. Music, Videos, etc. is a separate volume with its own drive letter), and all applications that live on top of NTFS and interact with NTFS through the documented interfaces should work just fine. In fact, application compatibility has *dramatically* improved compared to WHS v1, and making sure that we look just like regular NTFS volumes to applications (both local and remote) was one of our primary design goals for v2.

    Internally, these NTFS volumes are sliced and diced into 1 GB chunks, which are distributed (in multiple copies if duplication is enabled) across multiple physical disks according to our own on-disk schema. DEVolume.sys (our driver, working only on Vail at the moment) is the only driver currently in existence that can parse this layout and present the aforementioned NTFS volumes to applications. Consequently, any system not running DEVolume.sys is currently unable to retrieve the data from directly connected DEv2 disks -- that includes all client Windows OS. This affects only situations when DEVolume.sys is not loaded / running -- primarily recovery scenarios, as mentioned in the original post.

    So, bottom line is that I think that it will work. I encourage you to try and please let me know if it works or not (bulats at microsoft dot com)


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 3:49 AM
  • Buying bigger drives isn't necessarily an option. It's not like we all have a lot, if any, disposable income. One of the big advertised points of WHS is having the ability to use many drives of any type and size together. If I have to go buy a bunch of larger drives I could just then switch to raid all together and that would defeat the purpose of WHS. 

    How about instead we get the limit raised on the number of drives supported? After all I have this nice shiney norco case with room for 22 drives. I have room to grow.

    Tuesday, April 27, 2010 3:53 AM
  • I'm just curious -- what's the total capacity of all your drives?


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 3:54 AM
  • I'm just curious -- what's the total capacity of all your drives?


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team

    The console reports 10.46tb

    1x 1.5tb

    6x 1tb

    4x 750gb

    2x 500gb

    Any chance we can get parity for data protection as opposed to duplication? That would be nice.

    Tuesday, April 27, 2010 4:02 AM
  • Bulat, in WHS v1 I can access every drive from pool separately, through use of paths like C:\fs\K, C:\fs\P, and so on.
    Can I do similar thing in Vail? This is what is required for FlexRaid to work. It reads data from pool drives individually, and then creates parity data on a drive that is not in pool. It has to know on which physical drive each file is located.
    Tuesday, April 27, 2010 4:05 AM
  • > Any chance we can get parity for data protection as opposed to duplication? That would be nice.

    Not in Vail, that's for sure :)

    What's new in Vail though is built-in ECC which detects and corrects single and double bit flips per disk sector, even when duplication is disabled. So reliability is improved even when duplication is disabled (not that we think that it should ever be disabled).


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 4:08 AM
  • > Any chance we can get parity for data protection as opposed to duplication? That would be nice.

    Not in Vail, that's for sure :)

    What's new in Vail though is built-in ECC which detects and corrects single and double bit flips per disk sector, even when duplication is disabled. So reliability is improved even when duplication is disabled (not that we think that it should ever be disabled).


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    That is a very welcome change!
    Tuesday, April 27, 2010 4:10 AM
  • > Any chance we can get parity for data protection as opposed to duplication? That would be nice.

    Not in Vail, that's for sure :)

    What's new in Vail though is built-in ECC which detects and corrects single and double bit flips per disk sector, even when duplication is disabled. So reliability is improved even when duplication is disabled (not that we think that it should ever be disabled).


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team

    BulatS,

    In the product suggestion Ken created earlier today, you said, "@DeKoquonut: yes, connecting a set of drives to a different Vail server and reading data off them is a supported scenario. Fact is, you can even promote foreign storage to become the default storage on the new machine, so no copying is required."

    Can you please clarify what you mean?  Are you saying I can take any hard drive from one Vail server (including the primary drive) and add it to the storage pool on a different Vail server without having to copy anything?  If so, how do you get around the fact that when you try to add the drive to the pool, it will want to automatically format the drive?

    Tuesday, April 27, 2010 4:23 AM
    Moderator
  • Drive Extender in Vail is block-based, not file-based. Which means a) that separate replicas of a file are not individually accessible by a user (which was the case in v1) and b) that even a single replica of a file can be physically located on multiple drives (like on a striped RAID array).

    This, by the way, fixes a notorious v1 issue. There was a guy who backed up his clients into a single 400 GB file, and copied the file to a v1 home server. Now, he had 1 TB of free space, but it was distributed across four drives (250 GB on each), and he couldn't copy the file because free space on each drive wasn't large enough to accomodate the file (which was necessary for file-based Drive Extender v1). Now, in Vail, it would work just fine since a single file can span multiple physical disks.

    Going back to your question, duplication is enabled per volume, so you will know that X: (Music) is duplicated while Y: (Videos) is not. That might help to configure FlexRaid accordingly.


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 4:25 AM
  • Unfortunately this new Drive Extender in Vail is going to be a deal breaker for me.  The entire reason I chose WHS over other RAID type boxes is BECAUSE I can yank drives and recover the data in my main PC.  It has already happened to me 3 times because my main drive died.  You just lost a WHS customer and avid user.
    Tuesday, April 27, 2010 4:25 AM
  • Unfortunately this new Drive Extender in Vail is going to be a deal breaker for me.  The entire reason I chose WHS over other RAID type boxes is BECAUSE I can yank drives and recover the data in my main PC.  It has already happened to me 3 times because my main drive died.  You just lost a WHS customer and avid user.
    Well on the plus side you can schedule backups of your main drive in Vail. So restoring the machine will be even easier than it was before.
    Tuesday, April 27, 2010 4:28 AM
  • Micksh,

    Bulat is saying, in effect: "I don't know that it will or won't work, and it's outside the scope of what Microsoft is going to do for you. If you want to know whether you can use FlexRAID, please try it and see what happens."

    That said, the physical disks appear in the Disk Management tool. Those physical disks show up as formatted with the FAT32 file system if they're part of server storage (which is, umm, interesting). Logical volumes are maintained within the Drive Extender filter driver, and do not appear within Disk Management. So if you require access to the physical disks for FlexRAID to do it's job, then no, you're out of luck. If you can work with logical volumes, then it may work okay.

    Why use FlexRAID rather than Duplication, by the way? I would recommend you try the new version of Drive Extender; it really does perform better.


    I'm not on the WHS team, I just post a lot. :)
    Tuesday, April 27, 2010 4:29 AM
    Moderator
  •  

    Why use FlexRAID rather than Duplication, by the way? I would recommend you try the new version of Drive Extender; it really does perform better.


    I'm not on the WHS team, I just post a lot. :)
    Flex raid uses parity. So one drive protects all of your data whereas in WHS you need twice the amount of storage to protect your data. That's a pretty huge advantage for parity systems.  I've thought about going that route myself just so I could have more usable space without having to buy more drives.
    Tuesday, April 27, 2010 4:31 AM
  • Unit of protection in FlexRaid is physical drive. So, it needs to read content of the physical drive. Accessing volumes is not good enough for FlexRaid. It needs to know what physical HDD contains each part of each volume.
    Just like C:\fs\D\DE\folders directories in WHS v1. I know it is not documented or supported by Microsoft but it worked.
    And FlexRaid works on file level so block-based DE will add too many complications if not make it impossible to work.

    So most likely it will not work. I may install it just to see if there is any workaround. But even if there is I doubt it will work reliably.
    I wish instead of redesigning DE you have told that guy with 400GB file to split it. WinRAR can easily split large files.

    There is a side effect. Consider one drive dies and if there is no duplication or parity WHS v1 loses only files located on that drive. All other data is intact.
    Since files can be split between drives in WHS v2 it may lose much more, potentially all data. Is it correct?

    Duplication is way too expensive comparing to FlexRaid. I can have 10 or more TB of data and single 2TB parity drive.
    Tuesday, April 27, 2010 5:20 AM
  • Well, we aren't selling it yet :) The whole purpose of running a public beta is gathering feedback from avid users, and act upon that information. Don't give up yet, there's a good chance that we'll think of something.
    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 5:22 AM
  • For the love of god add the option to have a real raid system.  Raid 5 or Raid 6 is much better then file duplication.  This thing is ment to be a home SERVER.. key word is server why limit the speed of the drive access to one drive. Raid arrays have many advantages over any sort of duplications.  If a user wants to backup 5 to 10 machines a night you will have bandwidth issues to the drives. Add to this media services video streaming 1080p video 2-3 steams of this and most harddrives won't be able to keep up.

     

    PLEASE PLEASE PLEASE this is built off of server 2008 raid support is already there let users use it and not waste space on file duplication and slow file access. Am I the only one who sees the problem here. 

     

    Tuesday, April 27, 2010 5:29 AM
  • You can take *all* hard drives from one Vail server (or all minus one if all volumes are duplicated) and connect it to a different Vail server *that has no storage of its own* (that is, only the system volume is present). The target Vail server will recognize that a foreign disk set (aka 'non-default server storage') is connected. At this time the user will be able to promote the foreign disk set to become the default disk set for the target Vail server. Promoting to default does not format disks; it retains all data, establishes the association with the server, fixes all SMB shares etc.


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 5:31 AM
  • Consider one drive dies and if there is no duplication or parity WHS v1 loses only files located on that drive. All other data is intact.
    Since files can be split between drives in WHS v2 it may lose much more, potentially all data. Is it correct?

    That's my understanding as well, and with v1 I'm aware I have data at risk, but it's data that will not be "that" difficult to replace or not going forward. Critical data is duplicated to ensure a drive failure won't wipe it out, and I save considerable disc space with an acceptable (to me) level of risk of data loss. Although far from a perfect solution, the drive balancer tool made available elsewhere would at least make sure only a minimum amount would be at risk per disk.


    It sounds like v2 duplication is essentially required or your duplicated files may very well be completely wiped out with a single drive failure. This would keep me off v2 for the foreseeable future since all but two of my current shares are critical and duplicated and the rest make up 90% of my used space. Going from 5 to 10 TB in drives isn't in my budget.

     


    Tuesday, April 27, 2010 5:41 AM

  • It sounds like v2 duplication is essentially required or your duplicated files may very well be completely wiped out with a single drive failure.

    I think it is even worse than that. Even duplication is not a panacea. Now consider two drives die. In WHS v1 only files on these drives are lost.
    I believe in Vail, depending on where particular files are stored, it is theoretically possible that you lose all your data. And that is with duplication enabled!

    Consider you have 6 data drives. A file is split between drives 1-3. Its duplicated copy is split between drives 4-6. Same for all other files.
    Now drives 1 and 4 die and they happen to have the same duplicated part of your file. Part of the file is lost, but the worst part is that the data on remaining good drives is useless because it doesn't allow to recreate the file, one part is missing. The same theoretically can happen for all files.

    I hope Microsoft has some mechanism that reduces risk of that to happen but the fact is - it is possible to lose more data than contained on dead drives.

    And this whole block-based system just adds complications. The more complications the more chances for a bug or other accident.
    I don't think it is acceptable.
    Tuesday, April 27, 2010 6:03 AM
  • I think it is even worse than that. Even duplication is not a panacea. Now consider two drives die. In WHS v1 only files on these drives are lost.

    In all fairness, not many systems, including WHS v1 with full duplication, can fully survive a simultaneous two drive failure.

    rtk
    Tuesday, April 27, 2010 6:13 AM
  • I think it is even worse than that. Even duplication is not a panacea. Now consider two drives die. In WHS v1 only files on these drives are lost.

    In all fairness, not many systems, including WHS v1 with full duplication, can fully survive a simultaneous two drive failure.

    rtk

    His point was at least you can recover the data left on the remaining drives.
    --
    Tuesday, April 27, 2010 10:55 AM
  • I'm just curious -- what's the total capacity of all your drives?


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team

    14.55 TB\5.39TB Free.   Duplication on 4.5TB.

    I have 1-500gb and 1-750gb which I could retire.   The rest are 1TB, 1.5TB and 2TB sizes.

     

     

     

    Tuesday, April 27, 2010 12:12 PM
  • No, shawk, we had this debate some 2 1/2 years ago. The key word is home , not server. The costs associated with building RAID into the product in a way that's reliable and delivers good performance are very high for a consumer market. HP has sold tens of thousands (at least; I don't have any sales figures but has to have been at least that many) of their HP MediaSmart servers. If they had to include RAID in them, A) they would be bigger, and B) they would cost hundreds more than they do today. How many $900 HP MediaSmart Server units would have been sold?

    You, however, are welcome to install on a RAID array if you want (and can get the install to work; remember that you will need different drivers than previously). Let us know the results. I don't plan to do so; my current test server doesn't even include a RAID HBA because I prefer Drive Extender to RAID.


    I'm not on the WHS team, I just post a lot. :)
    Tuesday, April 27, 2010 12:44 PM
    Moderator
  • In furtherance to this issue (which I knew would come up a long time ago :) ) I have created this feedback on Connect . Vote it up...

    Edit: try this link . Note: 41 people have voted it up and 1 has voted it down, so the original link is also good. I wonder if someone missed when they clicked, or just doesn't like me very much? :) Anyway, it's generally visible; 42 votes in a few hours is pretty telling. If someone's particular Connect programs are messed up, there's not a lot I personally can do. Jonas or someone at Microsoft will have to help sort it out.


    I'm not on the WHS team, I just post a lot. :)
    Tuesday, April 27, 2010 12:47 PM
    Moderator
  • 2) Technically speaking, NTFS locks the user in MS as well :) We understand the importance of the described scenario and are thinking about ways to make it easier. That said, similar technologies (e.g. BitLocker and all kinds of RAID) often don't allow seamless mounting on other platforms but customers are actively using and are reasonably happy with them.

    Microsoft OS's aren't the only operating systems capable of reading NTFS partitions :)  But those people using RAID are not using WHS are they?  One of the big reasons why DIY'ers really like WHS is because it's not like a traditional RAID.  If I wanted something that was more like a traditional raid, I'd use Server 2008 + hardware raid, or linux + software raid.  Please, please give us a way to easily mount a drive in another machine.  If you don't allow this, I suspect many WHS DIY'ers will just switch to unRAID, or more traditional hardware raids.

    Regarding FlexRAID, this is a product that many DIY'ers also like because while they like WHS as a whole, they don't want to use data duplication for things like movies.  If WHS offered a similar technology to FlexRAID, we wouldn't have to worry about ever using it.  MS should really look into a parity type solution as well as a duplication solution.

    Tuesday, April 27, 2010 1:10 PM
  • Just get a page not found.


    I admit, for me, the problem here is that by segmenting files in 1GB chunks, which allows for greater distribution across drives seems like a good one, except what that works out to be is like a huge RAID-0.

    My current WHS system is:

    1 400GB boot partition

    4 1TB

    2 2TB

    1 1.5TB

    The 10 drive limit wouldn't hit me.. yet.  But it likely will at some point.   To get away from tricky-to-discuss issues, I have a lot of recordedTV that gets saved from my Media Centers.  Last check on the pie chart tells me 528GB.  And it's mostly garbage for the kids (Blues Clues, etc.) anyway.. Let's take one of their Disney Channel HD movies (thank you QAM) which is roughly 11GB total file size.

     

    That gets spread out across the drives in 1GB "chunks"..  problem:  one drive fails.  It's -gone-.  So, the moment I lose 1 drive out of 7, I've lost data.  This isn't unexpected.  But, what is unexpected is that in this method, I didn't lose just data contained on that drive, I've lost data on every single hard drive (potentially) that was larger then the 1GB chunk limit and touched that drive.  Now, the single drive failure becomes a potential insane disaster.  Losing a 1TB drive could cause me loss of the vast majority of anything saved across the network.

    This quickly becomes "no good". 

     

    In regards to the other issues.. the 10 drive issue is I'm sure a beta thing, so not worried about that; the integration with WMC will come later, so not worried about that.. but changing the drive method to what seems to be an incredibly riskier proposition is a bad one.  A really bad one.

    Tuesday, April 27, 2010 1:27 PM
  • How does this work (that is, if only the system volume is present)  ... has something changed?  If you install WHS, on even an 80GB HDD, it devotes about 60GB over to it's pool, right during install.  So, how do you end up with a Vail with no data pool at all to recognize these? 
    Tuesday, April 27, 2010 1:28 PM
  • Just get a page not found.


    I admit, for me, the problem here is that by segmenting files in 1GB chunks, which allows for greater distribution across drives seems like a good one, except what that works out to be is like a huge RAID-0.

    My current WHS system is:

    1 400GB boot partition

    4 1TB

    2 2TB

    1 1.5TB

    The 10 drive limit wouldn't hit me.. yet.  But it likely will at some point.   To get away from tricky-to-discuss issues, I have a lot of recordedTV that gets saved from my Media Centers.  Last check on the pie chart tells me 528GB.  And it's mostly garbage for the kids (Blues Clues, etc.) anyway.. Let's take one of their Disney Channel HD movies (thank you QAM) which is roughly 11GB total file size.

     

    That gets spread out across the drives in 1GB "chunks"..  problem:  one drive fails.  It's -gone-.  So, the moment I lose 1 drive out of 7, I've lost data.  This isn't unexpected.  But, what is unexpected is that in this method, I didn't lose just data contained on that drive, I've lost data on every single hard drive (potentially) that was larger then the 1GB chunk limit and touched that drive.  Now, the single drive failure becomes a potential insane disaster.  Losing a 1TB drive could cause me loss of the vast majority of anything saved across the network.

    This quickly becomes "no good". 

     

    In regards to the other issues.. the 10 drive issue is I'm sure a beta thing, so not worried about that; the integration with WMC will come later, so not worried about that.. but changing the drive method to what seems to be an incredibly riskier proposition is a bad one.  A really bad one.

    Can we get some clarification on this post?   Maybe we are not understanding these changes fully?  

    If I enable duplication for all shares on the server I should be 100% certain I can tolerate the loss of ANY singular drive and be certain of ZERO data loss just as in V1.  

    Is this not the case with the new block level changes????

     

     

     

    Tuesday, April 27, 2010 2:26 PM
  • Can we get some clarification on this post?   Maybe we are not understanding these changes fully?  

    If I enable duplication for all shares on the server I should be 100% certain I can tolerate the loss of ANY singular drive and be certain of ZERO data loss just as in V1.  

    Is this not the case with the new block level changes????

    It should be the case, I'm 100% sure. Otherwise, what's the point of duplication.
    There are two problems:
    1. Without duplication loss of single drive can lead to loss of all data. Just as if I lose one drive in WHS v1 I lose all PC backups spread across multiple drives. This may be OK for PC backups but not for my main data.
    2. With duplication enabled loss of two drives can lead to loss of all data. This wasn't an issue with WHS v1. In v1 you lose only what was on died drives.

    Another problem is that if server dies (motherboard or system drive dies on Vail) in order to access your data you need another Vail server capable of holding all your data disks at once.

    Being able to read single data disk on another system is not good enough as files may be spread across all drives. Without second identical Vail system all your data is not readable until you repair your server.

    I want NTFS back!

     

    Tuesday, April 27, 2010 3:15 PM
  • In furtherance to this issue (which I knew would come up a long time ago :) ) I have created this feedback on Connect . Vote it up...
    I'm not on the WHS team, I just post a lot. :)


    Ken,

    I get a page not found.  Suspect it is because I'm in the NDA Partner Program.  Can you get the feedback posted to the NDA Partner Feedback page?


    BullDawg
    Tuesday, April 27, 2010 4:03 PM
  • This block system(I think of it as stripping) while it might help with the issue you discuss, now requires multiple drives to recover anything over 1GB in size .. hrm, lets me see ... my entire video collection.  I didn't like normal RAID solutions because if my expensive card dies, I have to buy an exact duplicate to ensure I can read my drives.  With WHSv1, I have several $100 8port SATA controllers, if one dies, I buy another of same kind or a different kind. There are numerous other RAIDish like systems that exist that I looked into, unRAID being a runner up for me, but I'm not much of a linux person.  --At this point my coworker would come in exulting ZFS--  But I wanted a simple solution that protected against drive failures and that if the system itself had issues, I could possibly recover my data by putting the drives in another system.

    Something my friends and I have had to do multiple times now, frankly, do to WHS issues.  A reinstall of WHS couldn't read the tombstones one time, another it completely died and corrupted everything.  For me, this was my own fault, I couldn't figure out how to remove a drive and killed my system.  But in each case, the loss was very minor and in reality, it was video I could recover from originals.  I did loose my backups but have since learned of the BDDB plugin .. I'm going to loose my backups in the upgrade too huh?

    Now, perhaps it's crazy to think I could upgrade .. sure, we are going from 32bit to 64bit ... but I don't care about the WHS OS as much, in theory I should be able to wipe it now, reinstall and loose close to nothing.  But is it safe to assume there will be no conversation option to this new disc layout?  That will be a hard nut to swallow.

    Among the 5 people(inc myself) that have WHS box, only 1(father) has less then 10TB.  I have 4x1.5TB + 6x2TB with only 3.2TB free but really only 1.6TB free as I duplicate everything.  I have a 4U 20+1 disk case and bought a MB that I can fill up with 8 port SATA cards.  All I wanted in v2 was something that handled drives dieing in a nicer way(wow does WHS suck at handling dieing drives) .. and the steaming capabilities that WebGuide had(but hopefully much better).  Sadly, in exchange for just one of those items, I get a disk system that sounds way more complex and will likely cause more problems for me than it solves.  MS - I'm extremely puzzled!

    Most your audience might have 1 or 2 TB of data, but your enthusiasts, whom encourage the lay person to buy your product, want/need something better.  Sure it sucks to loose half your capacity, but that was a trade off I was willing to make for simplicity.  But now I would have to trade that simplicity in for complexity and get a media streaming system that doesn't appear much better than WebGuide did several years ago.  I'm sorry, but WTF is going on here??

    Tuesday, April 27, 2010 4:10 PM
  • I forgot a big one .. one MS has to know about but I'll throw my voice in ... STOP KILLING THE DRIVES DURING DUPLICATION.  This is freakin server, it should be able to do more than one thing at once.  For the love of god release a SP to fix this.  I copy 100GB over and then the system is useless until it finishes the duplication.
    Tuesday, April 27, 2010 4:18 PM
  • An option would be if you can assign a hot spare drive in WHS, if a drive fails all data is copied to the hot spare. The chance that two drives fail in one day is extremely low. Also, as said you can back-up your WHS. That alone should be enough protection.
    Tuesday, April 27, 2010 6:34 PM
  • An option would be if you can assign a hot spare drive in WHS, if a drive fails all data is copied to the hot spare. The chance that two drives fail in one day is extremely low. Also, as said you can back-up your WHS. That alone should be enough protection.

    Hot spare would help only if duplication is enabled on all data. Without duplication whole storage can be ruined if one drive fails. And duplication on all data is too expensive.

    WHS system drive backup doesn't protect from motherboard failure, for example.
    Tuesday, April 27, 2010 6:59 PM
  • It sounds like someone decided to come up with a solution to a problem that doesn't exist. Other than the case mentioned I can see no reason to make a change like this, and even with that case there are several ways to solve the problem without resorting to breaking everything else.

    Tuesday, April 27, 2010 7:11 PM
  • There's a new feature in Vail that allows keeping the system volume on a separate drive (e.g. if you want to keep the system volume on an SSD, or use RAID mirroring to protect the system drive etc.) By default after the installation, the system drive will contain the system volume and a Drive Extender data partition. In Vail, you can remove the Drive Extender partition from that drive, leaving the system volume and a bunch of unallocated space. If you then remove all data drives, your system will have a system volume and no default storage pool.
    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 7:40 PM
  • The real problem with file-based duplication was that nothing could be done with files that are open for use, and some files (esp. if some addins are present) are permanently open, generating unresolvable file conflicts.

    Additionally, reparse points has caused considerable appcompat issues with a number of applications. We tested a lot of 3rd party applications, found quite a few bugs (not necessarily in our code), and who knows how many more are there, undiscovered -- we obviously can't possibly test all 3rd party apps.

    With the block-level approach, we were able to hide all Drive Extender machinery under the file system. Now, to the outside world we look pretty much like a regular NTFS volume, while all the magic can be done transparently: removing a disk, adding a new disk -- all while the shares are mounted and visible, files stay open etc. etc. Also, by looking just like plain old NTFS to applications, we removed a whole class of application compatibility issues. In our testing right now we see A LOT fewer problems and bugs than at the comparable stage of development of our v1 product.


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 7:47 PM
  • The real problem with file-based duplication was that nothing could be done with files that are open for use, and some files (esp. if some addins are present) are permanently open, generating unresolvable file conflicts.

    Additionally, reparse points has caused considerable appcompat issues with a number of applications. We tested a lot of 3rd party applications, found quite a few bugs (not necessarily in our code), and who knows how many more are there, undiscovered -- we obviously can't possibly test all 3rd party apps.

    With the block-level approach, we were able to hide all Drive Extender machinery under the file system. Now, to the outside world we look pretty much like a regular NTFS volume, while all the magic can be done transparently: removing a disk, adding a new disk -- all while the shares are mounted and visible, files stay open etc. etc. Also, by looking just like plain old NTFS to applications, we removed a whole class of application compatibility issues. In our testing right now we see A LOT fewer problems and bugs than at the comparable stage of development of our v1 product.


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team


    Bulat

    There seems to be a lot of confusion\concern relating to these changes in regards to the 1GB file splitting across drives.   

    I think clarification on how this can impact recoverability from hardware failure is needed here.     

    Brian

    Tuesday, April 27, 2010 8:03 PM
  • There's a new feature in Vail that allows keeping the system volume on a separate drive (e.g. if you want to keep the system volume on an SSD, or use RAID mirroring to protect the system drive etc.) By default after the installation, the system drive will contain the system volume and a Drive Extender data partition. In Vail, you can remove the Drive Extender partition from that drive, leaving the system volume and a bunch of unallocated space. If you then remove all data drives, your system will have a system volume and no default storage pool.
    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team

    So, why the 160GB requirement? Why not allow the decision - to not have the system volume be part of the drive pool - to be made during installation, rather than winding up with 100GB of unallocated space after the fact?
    Tuesday, April 27, 2010 8:04 PM
  • Actually this decision can be made during installation, but it's controlled through a registry setting and is not exposed in the UI.

    Instructions are here: http://social.microsoft.com/Forums/en-US/whsvailbeta/thread/32844aae-9f41-41cb-8a4a-f6c26ddfdd6f


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 8:10 PM
  • Yeah, I think so. We'll need to clarify this in a more structured manner though, there are way too many frayed loose ends in this topic already... I guess I should write a separate post on this subject.


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Tuesday, April 27, 2010 8:14 PM
  • The real problem with file-based duplication was that nothing could be done with files that are open for use, and some files (esp. if some addins are present) are permanently open, generating unresolvable file conflicts.

    Additionally, reparse points has caused considerable appcompat issues with a number of applications. We tested a lot of 3rd party applications, found quite a few bugs (not necessarily in our code), and who knows how many more are there, undiscovered -- we obviously can't possibly test all 3rd party apps.

    With the block-level approach, we were able to hide all Drive Extender machinery under the file system. Now, to the outside world we look pretty much like a regular NTFS volume, while all the magic can be done transparently: removing a disk, adding a new disk -- all while the shares are mounted and visible, files stay open etc. etc. Also, by looking just like plain old NTFS to applications, we removed a whole class of application compatibility issues. In our testing right now we see A LOT fewer problems and bugs than at the comparable stage of development of our v1 product.

    Maybe I'm unique, but I install maybe 2-3 addons and none of those would have any reason to keep files open permanently. Even then, a solution based on volume shadow copy would seem to be a quicker and easier method of dealing with that than a new file system layer/method/etc.

    I've used and abused my WHS for doing all kinds of things it wasn't meant to, and never saw an issue with reparse points. The only applications I ever saw complain were things like Teracopy that didn't understand how you could have 15Tb free on a 60Gb drive. Fortunately, they provide an option to ignore things like that.

    Yes block-level hides the guts under the file system, at the cost of breaking file system compatibility with every OS made in the last decade, at the cost of removing one of the greatest (unsupported) features of WHS over a RAID, and at the cost or dramatically increasing the potential for catastrophic data loss.

    Tuesday, April 27, 2010 8:20 PM
  • Maybe I'm unique, but I install maybe 2-3 addons and none of those would have any reason to keep files open permanently. Even then, a solution based on volume shadow copy would seem to be a quicker and easier method of dealing with that than a new file system layer/method/etc.

    I've used and abused my WHS for doing all kinds of things it wasn't meant to, and never saw an issue with reparse points. The only applications I ever saw complain were things like Teracopy that didn't understand how you could have 15Tb free on a 60Gb drive. Fortunately, they provide an option to ignore things like that.

    Yes block-level hides the guts under the file system, at the cost of breaking file system compatibility with every OS made in the last decade, at the cost of removing one of the greatest (unsupported) features of WHS over a RAID, and at the cost or dramatically increasing the potential for catastrophic data loss.

    hear hear! ... 3 cheers! ... Give the man a drink ... someone needs to digg or twitter or facebook this ... I haven't installed anything that didn't say it was for WHS.  Not saying something couldn't break, but I would not expect everything to work, especially anything trying to do disk maintenance... now maybe this was easier/better than trying to get others to fix their code, or dealing with frustrated users who blame you MS.  I can't argue that.  But this is seriously scary, implimenting a brand new RAID like solution that nobody else can read/write, not even other windows boxes.  I liked WHS because recovering wouldn't be (to) hard.  Aside from the easy desktop backup situation, this was the sole reason for choosing WHS ... and now it's gone.
    Tuesday, April 27, 2010 10:12 PM
  • Well, it didn't take long for me to break it. 

     I set up a 500 gig drive on the MB sata controller, and another 3-2TB drives on a supermicro SAS controller.  The 2TB drives added without problem to the data pool..First impressions....it's going to be a nice improvement.

    BUT...an hour later I came back to play a little,and low and behold, all my drives are offline....what's this?  Maybe I need to reboot...that fixes 95% of problems.  Reboot....all my drives are offline....oops....not good.  So I figured out how to do a repair....this is good.....NOT.  After repair, I now have shared folders but they have no names, and can no longer be accessed.

    It was fun while it lasted

    Wednesday, April 28, 2010 4:32 AM
  • bellaireroad,

    Can you please send the following logs from C:\Programdata\Microsoft\Windows Server\Logs to me:

    DEDriverProxy*
    DEService*

    bulats at microsoft dot com

    Thanks!

     


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Wednesday, April 28, 2010 5:00 AM
  • The word "striping" for file storage terrifies me. Having seen way too many RAID 5 failures in businesses, I haven't built a RAID 5 array since 2005. Even in businesses, I see people ignoring or missing a failed disk warning or having multiple simultaneous disk failures (I've seen a couple of them).

    I pretty much only use WHS1 for PC backups for myself and for my clients. My clients (and I) already have dedicated file servers with backups enabled, so its file serving capability isn't that useful to me.

    In small offices with no server, I'm more likely to enable file shares directly on a client PC and have WHS make nightly backups of the client PC. That solves the "open file" issue and guards against multiple disk failures. Of course, changes in the shared data on the client PC will be lost if the PC fails, but, for most non-Enterprise uses, losing a day of changes isn't the end of the world. Losing EVERYTHING on a striped storage array is a lot bigger deal. And if WHS had an option to back up certain client PCs more than once a day, that'd minimize losses of changes on rapidly-changing file shares.

    When I do use the WHS shared folders, I make backups of important files rather than enable folder duplication. I consider folder duplication as "a zillion times better than what most people have", but I don't trust it for my own use. I'm even less inclined to trust striping. 

    I realize that there is no perfect AND inexpensive technology for storing and securig large amounts of data. Big hard disks fail way too often, striping is dangerous, and mirroring is expensive. I hope MS can come up with something that solves the holes that WHS1 has without delivering a whole new set of problems. It's a tough one. 

    Myron Johnson 
    RTA Information Technology
    Tempe, AZ
    http://rtait.com

    Wednesday, April 28, 2010 6:58 AM
  • Let's not lose our perspective and get too caught up in the potential failings of the new DE.  I assume we are here because we liked v1, so let's not forget that compared to alternatives, WHS does have clear advantages.

    Let's NOT focus on things like how losing a drive will lose unduplicated data, since that is obvious and not how WHS is intended to be used.  Let's also be clear that dual disk failures will lose data on ANY storage system short of RAID 6/15 which are not in the same class economically.  And remember that although 50% space efficiency is as low as it gets, higher efficiencies either increase processing overhead or reduce protection, as users of RAID 1 vs RAID 5 vs RAID 0 will be very familiar with.  I think these are things we already know and I at least would not want to focus on them.

    Having said all that, I do agree that the data needs to be accessible off-system.  Rebuilding a failed system is a non-trivial task and sometimes you just don't have the time for that, but you still need your data.  There has to be some way to get data off without requiring a rebuild.  I won't say it's a deal beaker, I can use offline folders which I tend to do anyway, but I certainly would like to see a more robust implementation.

    Speaking of robustness, block based DE is certainly a step up from v1 in that department.  I think this is the right direction as it removes a lot of dependencies and edge cases and just works.  It has problems certainly, but that's what we're here for, to find them and address them.

    Keep up the good work!

     

    Wednesday, April 28, 2010 8:41 AM
  • ...
    I wish instead of redesigning DE you have told that guy with 400GB file to split it. WinRAR can easily split large files.
    ...
    I suspect that particular user was storing something like an encrypted volume (TrueCrypt or similar) on his server. If it's an active file, splitting it isn't an option.

    I'm not on the WHS team, I just post a lot. :)
    Wednesday, April 28, 2010 12:23 PM
    Moderator
  • And remember that although 50% space efficiency is as low as it gets, higher efficiencies either increase processing overhead or reduce protection, as users of RAID 1 vs RAID 5 vs RAID 0 will be very familiar with. 

    Actually, 38% is as best as it gets with duplication.  88% without duplication.  This is due to the 12% overhead of the CRC functionality.  A MSFT guy gave the 12% overhead somewhere in this forum.  So if you have 100TB of disk space and duplicate everything, only 38TB is usable.  I give the 100TB number 1) because it's nice and round to do such math on and if we say v2 has the same 32disk limit and 3TB drives will likely exist in the next 12mo then we are looking at possible 100TB systems :)  Isn't that exciting!!!

     

    And if anyone says -- 'what could you possible use with 100TB (or even 38TB) of storage' -- you should leave :P  I have no doubt I could easily find ways to fill this up.  Especially if my 38TB figure is correct.  I could use 38TB without being overly creative even.

    Wednesday, April 28, 2010 2:46 PM
  • I think it is even worse than that. Even duplication is not a panacea. Now consider two drives die. In WHS v1 only files on these drives are lost.

    In all fairness, not many systems, including WHS v1 with full duplication, can fully survive a simultaneous two drive failure.

    rtk
    There is a HUGE difference in how WHS v1 and WHS v2 would handle a 2 drive failure. At least on WHS v1 you would lose a small fraction of your data in a 2 drive failure. In WHS v2 you will have lost everything - right? If they want to go with blocks and forgo the simplicity of the WHS v1 style simple duplication then they should have gone whole hog and made the software RAID more efficient than ridiculous duplication with one drive failure capacity. The only reason we put up with the inefficiency of duplication is for the simplicity of the scheme and availability of data on the individual drives. Now with those lost, why duplication? Why not something more like RAID-6? Why not something more efficient? This seems like a half-way implemented kludge offering neither the benefits of WHS v1 style simplicity or RAID-6 efficiency, speed and 2 drive failure capability. As it stands this new DE offers the worst of both worlds.
    Wednesday, April 28, 2010 4:55 PM
  • It sounds like someone decided to come up with a solution to a problem that doesn't exist. Other than the case mentioned I can see no reason to make a change like this, and even with that case there are several ways to solve the problem without resorting to breaking everything else.


    It's not a "solution" it's a straight up kludge they had to use since they could not implement WHSv1 style simple duplication on the Win 2008 OS. So this was the best they could come up with. As usual with MS coming up short. There needs to be more competition in this market.
    Wednesday, April 28, 2010 5:05 PM


  •  

    The real problem with file-based duplication was that nothing could be done with files that are open for use, and some files (esp. if some addins are present) are permanently open, generating unresolvable file conflicts.

    Additionally, reparse points has caused considerable appcompat issues with a number of applications. We tested a lot of 3rd party applications, found quite a few bugs (not necessarily in our code), and who knows how many more are there, undiscovered -- we obviously can't possibly test all 3rd party apps.

    With the block-level approach, we were able to hide all Drive Extender machinery under the file system. Now, to the outside world we look pretty much like a regular NTFS volume, while all the magic can be done transparently: removing a disk, adding a new disk -- all while the shares are mounted and visible, files stay open etc. etc. Also, by looking just like plain old NTFS to applications, we removed a whole class of application compatibility issues. In our testing right now we see A LOT fewer problems and bugs than at the comparable stage of development of our v1 product.


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team

    Okay that is fair enough and very reasonable. There are advantages to that approach I can see. But I think if files below 1GB are not broken up into multiple blocks then a driver should be issued for Win 7 and Win XP to read those files. In my case and most cases I'd say the critical files are under 1GB. For me files larger than that are videos for which I have the original disc or source anyway.  But yes I suppose for those critical files instead of worrying about how many drives of failure the system can withstand, a way to more easily group those files for offsite backup would be better. So you could have a folder or subfolder that is specifically earmarked for offsite duplication and the interface could allow for a choice of offsite backup provider for those critical folders.
    Wednesday, April 28, 2010 5:12 PM
  • Actually this decision can be made during installation, but it's controlled through a registry setting and is not exposed in the UI.

    Instructions are here: http://social.microsoft.com/Forums/en-US/whsvailbeta/thread/32844aae-9f41-41cb-8a4a-f6c26ddfdd6f


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team


    Why? Or rather why not?

    Wednesday, April 28, 2010 5:13 PM
  • My deal breakers:

    1. Painful transition from v1 to v2.

    2. Non-duplicated folders will now be entirely lost because of a single drive failure.  My videos folder is 8-10TB and cannot be feasibly duplicated.  In WHS v1 I can lose a portion of my videos folder if one of my drives die, but certainly not the entire thing.

    3. If I have a multiple drive failure including system drive, or anything complicated really, I can simply throw my drives in one of my other PCs and recover SOME of the data rather than losing everything.

    4. 10 drive limit.  I know this is a soft limit but it really needs to be increased.  20TB arrays will be necessary if we are going to have a reasonable amount of storage space with full duplication on.

    It feels like you have changed WHS from a very cheap, very flexible, very recoverable system to an incredibly crappy, proprietary RAID-0+1 implementation.

    Wednesday, April 28, 2010 5:29 PM
  • Remember that Home Server is not a boxed product and reaches the end user already pre-installed. OEMs and system builders (including people who act like one by buying WHS software on newegg and installing on white boxes) are fine with a registry key, and UI additions require a lot of work (need manual testing, translation into multiple languages, etc.)


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Wednesday, April 28, 2010 10:14 PM
  • bellaireroad,

    Can you please send the following logs from C:\Programdata\Microsoft\Windows Server\Logs to me:

    DEDriverProxy*
    DEService*

    bulats at microsoft dot com

    Thanks!

     


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    You have mail
    Wednesday, April 28, 2010 10:42 PM
  • Actually, 38% is as best as it gets with duplication.  88% without duplication.  This is due to the 12% overhead of the CRC functionality.  A MSFT guy gave the 12% overhead somewhere in this forum.  So if you have 100TB of disk space and duplicate everything, only 38TB is usable.  I give the 100TB number 1) because it's nice and round to do such math on and if we say v2 has the same 32disk limit and 3TB drives will likely exist in the next 12mo then we are looking at possible 100TB systems :)  Isn't that exciting!!!

    I suspect the math is slightly wrong there, but since I am not in the know, I will not speculate on the exact number, which is anyway not that important.

    What is important is that you realize that data protection is not free.  It never has been and I suspect never will be, no matter how efficient it gets.  The alternative we have to duplication is parity, until someone invents something better.  Parity is more efficient space-wise, but more intensive on the CPU and disk load.  Due to that added load, parity based systems frequently have added hardware to compensate, which increases cost.  This cost can be more than the cost of additional drives, which are relatively inexpensive these days.

    So the bottom line is: data protection will cost more than no protection.  How much more depends on the system.  I think WHS is the best alternative for those who do not want to waste a lot of time managing or upgrading their systems, but still want some protection for their data.  Nothing else comes close, and believe me, I've tried them all.

    If you have very expensive high performance disks, that picture may be different, so be sure to do your research and your math.

     

    And if anyone says -- 'what could you possible use with 100TB (or even 38TB) of storage' -- you should leave :P  I have no doubt I could easily find ways to fill this up.  Especially if my 38TB figure is correct.  I could use 38TB without being overly creative even.

    Truly, we live in an age of excess when our whims and not our needs dictate our behaviour.  Personally, I only have about 50GB of absolutely irreplaceable data, which has many layers of protection applied to it.  The rest is just a pain to regenerate, so I duplicate it anyway.  I could probably fill 38TB or even more, but if I were to lose it all through mishap, I would probably just shrug and move on.  No big deal.

     

    Thursday, April 29, 2010 12:48 AM
  • While this seems reasonable, it really isn't.  WHS V1 is a solid product.  But, for a lot of us, the debate isn't between something like WHS and Server 2003 or Server 2008.  The debate was always between WHS V1 and something like, say, FreeNas, or NasLite.  Those products are either Free, or at a very low cost.  So, the $99 Partner expense for an OEM copy was weighed into the equation.  Many of us (partners) liked some of the aspects of WHS enough, specifically remote connect and the ability to utilize plugins as it developed, that we found it easier for the end user to get along with.  And, in a one or two drive system, this change in WHS disk management really doesn't mean much.

    But on the other end, for anyone who has discussed this with MS at any of the TS2 events that were held about it, there was all sorts of discussion about how to integrate it for small business.  HP marketted a product line around that.  That's great.  And, if partners look through their kits, we all received a lot of nice posters, media, and even some PDFs etc. extolling the virtues of a WHS for a small office, etc.

    Here's the thing.. with something like a FreeNas or a NasLite or an integrated product (free) I could have a 20 drive setup, and if one drive fails, I either: just lose the data on that drive OR if it's RAID, I lose nothing.   Now, for WHS, because it's a simple product, I have no problem with the: if a drive fails, I =just= lose the data on that drive.  If I have 5 drives and 3 drives fail, I should lose all the data on those three drives, and that's it, the 2 drives that remain function.  I can pull those three drives and try to data recover them or whatever.

    I think as this is currently evaluated, it really neuters WHS and makes it a very "small" solution; it really makes having large WHS a real wasted effort; it's much safer/easier/cheaper to just role out any of the other solutions (see above with something like a ZFS Freenas) vs. WHS. 

    Which is too bad, because WHS hit the perfect market:  It managed to do damn near everything that a linux NAS did (with the exception of DLNA, and functions that could be added via plugin) but it did it in an interface end users trusted, which made it easy to sell.  If WHS loses some of that safety blanket that even free alternatives provide, then it becomes a much more difficult sell for partners.  It makes WHS a product that is designed for a system of 1, maybe 2 hard drives with an external as a constant server backup.  Suddenly, what seemed like a good deal for consumers becomes not so good of a deal.

    Thursday, April 29, 2010 2:25 AM
  • How about letting the server owner select the file size at which "striping" begins?

    The example of a 400 GB file that can't be handled with DE1 is an example of a file that the owner is probably going to have to let be striped. But what if you could say, "Don't stripe anything smaller than 4 Gigabytes"?. That'd give DE massive flexibility in managing file locations while allowing the owner to decide the level of "striping risk" that's allowable.

    Under this scenario, all files smaller than, say, 4 GB would be recoverable as long as the disk they are on is readable. Individual-disk recovery could be done with a converter to read the 1 GB block-level files and their indexes. You could even have a setting that says, "If a file is between 4 GB and 100 GB in size, try to move all the parts to a single disk when you aren't too busy".

    In action, the file system would stream bits into 1 GB files until the "limit" is reached. Then it'd be free to stream additional bits onto other disks if necessary for management or speed reasons.

    Since I'm not an OS designer, there may be some big holes in this scenario. But I think letting the owner select the "right" balance between performance and recoverability, while still keeping files small enough to let DE manage them efficiently, would be assuring to those of us who are concerned about "striping" and don't want WHS to perform 100% duplication of our data.

    Myron Johnson
    RTA Information Technoogy
    Tempe, AZ
    http://rtait.com

    Thursday, April 29, 2010 6:41 AM
  • How about letting the server owner select the file size at which "striping" begins?

    Myron, letting the server owner choose a block size for DE is really not likely to be considered by Microsoft. Vail builds the storage pool during setup, and the block size is fixed at that time. Changing it later is probably impossible.

    What Microsoft is more likely to consider is a suggestion that the block size be increased from the current 1GB to 2 or 4. Although that presents some challenges for smaller disks (<40 blocks on a 160 GB drive), there aren't many smaller disks in use any more.

    So could you please submit a product suggestion on Connect ?


    I'm not on the WHS team, I just post a lot. :)
    Thursday, April 29, 2010 2:40 PM
    Moderator
  • I am really concerned about the "striping" like many, I have many DVD and Blu-Ray rips which are not duplicated on WHS v1.  I have the original disks so they can be restored if I suffer a hard disk failure.

    On Vail most of these files will be striped across many drives.  One drive failing could result in losing the majority of my collection, and instead of having to re-rip a few DVDs and Blu-Rays that were lost I could potentially have to re-rip everything that was not duplicated.  So it seems that to replicate WHS v1 levels of protection is to duplicate everything - particularly where large files are concerned, increasing the cost to the consumer, or be faced with a much more lengthy recovery process as more data will be lost.  My opinion is that this is very much a backward step.

    In keeping with Myron's suggestion, I would propose that similar to the way in which WHSv1 allowed the user to set duplication on/off on a per share basis, a similar option to set "striping" to on/off on a per share basis should be introduced - which would force large files to reside on one physical disk, space permitting of course, and minimising the impact of a single disk failure against multiple large files.

    [edit: This Connect suggestion is quite appropriate https://connect.microsoft.com/WindowsHomeServer/feedback/details/554996/allow-disabling-of-file-chunking-striping-for-non-duplicated-folders

    Thursday, April 29, 2010 3:05 PM
  • <non-constructive criticism>

    I have to agree with everybody in the "WTH" camp.  I have been using WHS1 since its first beta.  I anxiously awaited RTM and built my first box only to be blindsided by the nasty corruption bug.  Only some corrupt MP3s, re-ripped so not a huge deal to me.  Now FFW to PP3 and I love WHS, recommend it to all my friends.  Survived server crashes, failed power supplies, bad mobos, all recovered in one way or another with no zero data loss.  So then I downloaded and installed Vail the day the beta is available.  Looks great, looking forward to actually being able to restore my x64 machines without having to install a second nic.

    Then I head to this forum.  Read a few threads and slowly but surely lose all faith in this product.  Overhead for data protection is expected.  50% is expected in a "RAID1" environment.  That's an easy pill to take.  Now it's going to be closer to 70%!  Really?  Seriously? That's the best you can do?

    Then the next bomb hits and all I can think is "Are you freakin' high?"  Who the heck makes the decisions around there?  As others, I have multiple TBs of non-duplicated WHS1 data.  Lose one drive, lose all the data on that drive and it's an acceptable risk to me.  Now with Vail if I lose a single drive it's almost a certainty (higher than 10% is a certainty!) that I will lose more than just the data on that drive!?  Pass me whatever you're hittin'!

    </non-constructive criticism>

    I am married to Microsoft.  I have worked in IT for 10 years.  I have a 2000 and 2003 MCSE, 2008 MCITS, Exchange 2007 MCITP.  I owe my livelihood to MS.  Apple is the devil.  Linux is fun for someone like me but I could never recommend freenas or openfiler to somebody who isn't a geek.  This may be an early beta for Vail but given these two "features" I will never "upgrade" to a Vail WHS.  I feel like my spouse just cheated on me.

     

    Thursday, April 29, 2010 4:04 PM
  • <non-constructive criticism>

    I have to agree with everybody in the "WTH" camp.  I have been using WHS1 since its first beta.  I anxiously awaited RTM and built my first box only to be blindsided by the nasty corruption bug.  Only some corrupt MP3s, re-ripped so not a huge deal to me.  Now FFW to PP3 and I love WHS, recommend it to all my friends.  Survived server crashes, failed power supplies, bad mobos, all recovered in one way or another with no zero data loss.  So then I downloaded and installed Vail the day the beta is available.  Looks great, looking forward to actually being able to restore my x64 machines without having to install a second nic.

    Then I head to this forum.  Read a few threads and slowly but surely lose all faith in this product.  Overhead for data protection is expected.  50% is expected in a "RAID1" environment.  That's an easy pill to take.  Now it's going to be closer to 70%!  Really?  Seriously? That's the best you can do?

    Then the next bomb hits and all I can think is "Are you freakin' high?"  Who the heck makes the decisions around there?  As others, I have multiple TBs of non-duplicated WHS1 data.  Lose one drive, lose all the data on that drive and it's an acceptable risk to me.  Now with Vail if I lose a single drive it's almost a certainty (higher than 10% is a certainty!) that I will lose more than just the data on that drive!?  Pass me whatever you're hittin'!

    </non-constructive criticism>

    I am married to Microsoft.  I have worked in IT for 10 years.  I have a 2000 and 2003 MCSE, 2008 MCITS, Exchange 2007 MCITP.  I owe my livelihood to MS.  Apple is the devil.  Linux is fun for someone like me but I could never recommend freenas or openfiler to somebody who isn't a geek.  This may be an early beta for Vail but given these two "features" I will never "upgrade" to a Vail WHS.  I feel like my spouse just cheated on me.

     


    Very Nice! I agree completely!
    Thursday, April 29, 2010 5:15 PM
  • Alas, I don't. How big were the logs?

    Anyway, let's try it the proper way. Can you please file a bug through Connect and attach the logs?


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team
    Thursday, April 29, 2010 5:28 PM
  • Alas, I don't. How big were the logs?

    Anyway, let's try it the proper way. Can you please file a bug through Connect and attach the logs?


    Bulat Shelepov, Test Lead (Drive Extender), Windows Home and Small Business Server Team

    well I hope you got them now,  I used the vail log connector software to send it in
    Friday, April 30, 2010 1:57 PM
  • If you don't file a bug, and note the CAB number the log collector reported in that bug, Bulat will never find them. If you don't have the CAB number, you can run the log collector again (it should only take a couple of minutes), note the cab number, let it upload, and file the bug on Connect .
    I'm not on the WHS team, I just post a lot. :)
    Friday, April 30, 2010 2:12 PM
    Moderator
  • There are two types of users, those whole have lost data and those who will loose data.

    A very large 3 letter acronym TELCO. has 450K spinning disks, 10k failures per year. This means 27+ per day, effectively >1 per hour, sure you can loose data.

    RAID 6 will survive it but now we are into enterprise class and some SMB rigs. The whole RAID thing leaves me cold. It only protects against bad drives. There are cases of failed server components other than drives. Rebuild from that and talk about Royal PITA. For me a separate old XP with attached USB box full of old EIDE and Synctoy do the job.

    Sunday, May 2, 2010 12:05 PM
  • Not neccesarily - Lime Technology has very cheap raid system called "Unraid" and satifies many of the complaints put forth hear. http://www.lime-technology.com/

    1. Has a parity drive
    2. Tolerates a single failed hard drive. Array parity permits reconstruction of a single failed data disk.
    3. Drives are not striped and can be read by any computer (although the system uses rieser as the file system)
    4. Tolerates single drive read errors . Read errors from a single disk are corrected "on-the-fly".
    5. Increased flexibility: not all hard drives need to be the same size or speed. The only requirement is that the parity drive must be as large as or larger than any of the data drives.
    6. Better fault tolerance: in the unlikely event of catastrophic hardware or software failure, data corruption would be much more isolated than that which could occur with other RAID organizations. It is almost impossible to lose all your data, since each individual data drive has its own file system.
    7. Better power management: not all hard drives are required to be spinning in order to access data normally; hard drives not in use may be spun down.
    8. Up to 20 hard drives support!
    9. Software cost - $120

    The down side - since the drives are not striped your not going to get the write performance of a RAID 5 or RAID 6 system, but on a home system it's not neccesary. It has plenty of speed to serve multiple HD bitstreams simultaneously.

    The user interface is a little low tech and updates are slow but sure becuase this is a one man show. amazing what one guy can do though.

    Seems that microsoft could adopt this technology (Just buy the guy out and make him a milionare) and they could bring it to the masses. and update the user interface and viola the perfect home raid solution!

     

    Sunday, May 2, 2010 4:41 PM
  • I think what we're running into though is a real misconception of what users want.

    I run WHS V1 at home.  At work, we run a FreeNAS.

    Now, both would just be called as far as I'm concerned about (JBOD). What WHS allowed for was that it would keep a set of files on two drives instead of one, "just in case".  This allowed a non-RAID solution to duplication without loss of space or risk of real failure.  Done without user intervention, this provided a real world way to store "important" information without risk, and  non-essential data, if a drive failed, was lost.


    Yes, this meant users had to know what from what.  But, for the most part, this happens often.  Backups/etc. duplicated.  Stuff like "Videos" not really, because if a drive fails, just redo what was on that drive.  In the case of a large drive, it could be annoying, but everything not lost.

     

    Now we're moving into a system that doesn't do hardware ready raid, and does a striping method which seemingly ensures high data risk with an insanely high overhead.  Having viewed the Drive Extender thread, it appears as though we've taken a major, major step back on an assessment of the purpose that end users want.  End users want general safety, and of course, storage readily available that makes sense.  Explaining to an end user that "ooh, your HP failed, sorry, let's get the data off.. wait, can't, we have to build another Vail server to do that, and you didn't just lose data on one drive that went bad, data on all three drives is hosed."


    I may regret it if one of my drives in the Freenas fails at the office.  Bad deal.  But in general, I back up what's I really need to come back.  And one drive fails, it's still online with the other 10, who all have their data. 


    This sounds as though MS has constructed Vail to take on the worst features imaginable for user reliability.. I'm hoping with this being Beta1 there is time to reconsider this, otherwise, they are rolling out a turkey.

    Sunday, May 2, 2010 10:34 PM
  • I think what we're running into though is a real misconception of what users want.

    I run WHS V1 at home.  At work, we run a FreeNAS.

    Now, both would just be called as far as I'm concerned about (JBOD). What WHS allowed for was that it would keep a set of files on two drives instead of one, "just in case".  This allowed a non-RAID solution to duplication without loss of space or risk of real failure.  Done without user intervention, this provided a real world way to store "important" information without risk, and  non-essential data, if a drive failed, was lost.


    Yes, this meant users had to know what from what.  But, for the most part, this happens often.  Backups/etc. duplicated.  Stuff like "Videos" not really, because if a drive fails, just redo what was on that drive.  In the case of a large drive, it could be annoying, but everything not lost.

     

    Now we're moving into a system that doesn't do hardware ready raid, and does a striping method which seemingly ensures high data risk with an insanely high overhead.  Having viewed the Drive Extender thread, it appears as though we've taken a major, major step back on an assessment of the purpose that end users want.  End users want general safety, and of course, storage readily available that makes sense.  Explaining to an end user that "ooh, your HP failed, sorry, let's get the data off.. wait, can't, we have to build another Vail server to do that, and you didn't just lose data on one drive that went bad, data on all three drives is hosed."


    I may regret it if one of my drives in the Freenas fails at the office.  Bad deal.  But in general, I back up what's I really need to come back.  And one drive fails, it's still online with the other 10, who all have their data. 


    This sounds as though MS has constructed Vail to take on the worst features imaginable for user reliability.. I'm hoping with this being Beta1 there is time to reconsider this, otherwise, they are rolling out a turkey.


    I've been looking forward to the WHS V2 release for some time and I tend to agree.   I don't have any of the problems associated with MS's reasons for the re architecture of Drive Extender, and I really don't see any must have features in this vail preview.  The CRC check is nice but even that has a 12% drive space penalty..... and I already have a 50% penalty on duplication.  62% overhead??   And the explanation behind why the chunking has been implemented is ridiculous.     How many home uses have single 400GB files?!!!!

    The more I look at this the more I am think I will just stay on V1.  Sometimes if something works it is best to just leave it as is....

     

     

    Sunday, May 2, 2010 11:09 PM
  • That's why the system I quoted above would be a better way to go.

    1. Easy for the general consumer to use
    2. The space benifit due to use of a parity drive rather than mirrored hard drives
    3. ability to rebuild the sytem after a failed disc
    4. Easily expandable as needed, add more drives as data expands
    5. Flexable, can use any disc that's laying around as long as parity drive is the largest (including mixing of SATA & IDE)
    6. No striping so no casatropic total loss, Free tools available to read the disc from explorer for the surviving disks
    Sunday, May 2, 2010 11:10 PM
  • "...And the explanation behind why the chunking has been implemented is ridiculous.     How many home uses have single 400GB files?!!!!..."
     
    I bet it is a lot less than the 20% that use USER shared folders. "Less than 20%" was the justification to do away with them.

    --
    ______________
    BullDawg
    Associate Expert
    In God We Trust
    ______________
     
    "fladtheimpaler" <=?utf-8?B?ZmxhZHRoZWltcGFsZXI=?=> wrote in message news:90c38091-6db0-4450-abe1-213ce3f625c3...

    I think what we're running into though is a real misconception of what users want.

    I run WHS V1 at home.  At work, we run a FreeNAS.

    Now, both would just be called as far as I'm concerned about (JBOD). What WHS allowed for was that it would keep a set of files on two drives instead of one, "just in case".  This allowed a non-RAID solution to duplication without loss of space or risk of real failure.  Done without user intervention, this provided a real world way to store "important" information without risk, and  non-essential data, if a drive failed, was lost.


    Yes, this meant users had to know what from what.  But, for the most part, this happens often.  Backups/etc. duplicated.  Stuff like "Videos" not really, because if a drive fails, just redo what was on that drive.  In the case of a large drive, it could be annoying, but everything not lost.

     

    Now we're moving into a system that doesn't do hardware ready raid, and does a striping method which seemingly ensures high data risk with an insanely high overhead.  Having viewed the Drive Extender thread, it appears as though we've taken a major, major step back on an assessment of the purpose that end users want.  End users want general safety, and of course, storage readily available that makes sense.  Explaining to an end user that "ooh, your HP failed, sorry, let's get the data off.. wait, can't, we have to build another Vail server to do that, and you didn't just lose data on one drive that went bad, data on all three drives is hosed."


    I may regret it if one of my drives in the Freenas fails at the office.  Bad deal.  But in general, I back up what's I really need to come back.  And one drive fails, it's still online with the other 10, who all have their data. 


    This sounds as though MS has constructed Vail to take on the worst features imaginable for user reliability.. I'm hoping with this being Beta1 there is time to reconsider this, otherwise, they are rolling out a turkey.


    I've been looking forward to the WHS V2 release for some time and I tend to agree.   I don't have any of the problems associated with MS's reasons for the re architecture of Drive Extender, and I really don't see any must have features in this vail preview.  The CRC check is nice but even that has a 12% drive space penalty..... and I already have a 50% penalty on duplication.  62% overhead??   And the explanation behind why the chunking has been implemented is ridiculous.     How many home uses have single 400GB files?!!!!

    The more I look at this the more I am think I will just stay on V1.  Sometimes if something works it is best to just leave it as is....

     

     


    BullDawg
    Monday, May 3, 2010 3:21 AM
  • I think the 400GB file was just one example of the benefits of v2 extender and probably not the strongest.

    One thing I have noticed in the last several days of running Vail is how it behaves like a true file server.  Absolutely no weirdness that was caused by the v1 drive extender implementation.  No lag when streaming video caused by demigrator moving files around.  no lag when accessing shares due to the old tombstone implementation.  This thing just works.  My kids were streaming a DVD into the living room while I ran a full server backup and the thing did not even hiccup.  With v1 they would have had to forget their movie if I tried to do anything else with the box.  It reminds me of my old XP Pro file server with Raid 1 except I can choose to turn duplication off for my media and then still capture it all on offsite backups just in case a disk does die. 

    With complete image based backup of the entire server via Windows Server Backup being implemented I think this thing is perfect.  I have all my files offsite on two drives (just in case some nasty bug rears its head) and am working with everything live on VAIL now and am really excited to see it get polished up and released.

    If you guys haven't tried it, I would strongly recommend seeing the performance improvements that have been accomplished by moving drive extender below NTFS.

    Chad

    Monday, May 3, 2010 3:46 AM
  • I've been playing with Vail in a HyperV box.  I'm wondering what you mean by "True File Server"  .... there are numerous Linux based NAS that I'd consider "true file servers" that don't exhibit these issues; then again, most of those support real RAID controllers if need be, and if not, if using JBOD, they support pulling a disc and mounting it elsewhere.

     

    "With Complete Image Based backup of the entire server.."

     

    Yeah.. you're going to complete backup a 10TB server.. where?  Have another 10TB that you can hookup in a single volume somewhere?  If you do, then why did you need WHS to begin with?

    Monday, May 3, 2010 8:12 AM
  • I work in tech support for a large software vendor.  Vail is running on top of a state of the art operating system and the new architecture is more advanced under the hood.  I get these concepts.     

    I am not concerned about how well this product works when the hardware and software are both working as expected.  That's the easy part.

    I am concerned more what happens when the unexpected occurs.

    This is supposed to be a HOME solution.   Most users who post on here are somewhat technical, and I would say the majority are most likely more technical then the "set it and forget it" solution MS is targeting for WHS.

    The changes which are being implemented in this version namely 1GB file chunking\striping and lack of file recovery from non vail machines could very well to cause the average user data loss.  

    Product development is not cheap and my guess is this architecture is on the rails and there are no stopping these changes this far into the dev cycle.

    So everybody drink the kool aid about how great this new release will be.....  until you have a hardware issue and limited technical knowledge of how this product works under the hood.   

    And if for some reason things go very wrong, and you are not able to re assimilate your drives to a new storage pool you are done.    MS has left you without a life raft.


    Monday, May 3, 2010 1:11 PM
  • No, shawk, we had this debate some 2 1/2 years ago. The key word is home , not server. The costs associated with building RAID into the product in a way that's reliable and delivers good performance are very high for a consumer market. HP has sold tens of thousands (at least; I don't have any sales figures but has to have been at least that many) of their HP MediaSmart servers. If they had to include RAID in them, A) they would be bigger, and B) they would cost hundreds more than they do today. How many $900 HP MediaSmart Server units would have been sold?

    You, however, are welcome to install on a RAID array if you want (and can get the install to work; remember that you will need different drivers than previously). Let us know the results. I don't plan to do so; my current test server doesn't even include a RAID HBA because I prefer Drive Extender to RAID.


    I'm not on the WHS team, I just post a lot. :)

    I completely disagree as far as a software raid solution is concerned.  For one, there is already a solution in windows 2008 (albeit mirroring or RAID 5 -- not the best options) which this is based off of, and for another, the cost of implementing RAID would be cheap b/c the XOR calculations would all be done via CPU.  By the time VAIL is ready to ship, all mediasmart servers most likely will have i5 dual cores w/hyperthreading and at lowest spec, an i3 or regular core2duo (like they're shipping high end now) so the cost to implement would be nil, if any at all.  I know the emphasis is on HOME but it needs to be decided whether MS is going to stick to that or not.  Is it going to address the person that has storage needs and backup needs only?  In that case the guy w/the 400GB file should be a non-issue as well as non compatible 3rd party apps, b/c all third party apps for non-power uses will be installed via the console & designed specifically for the HOMEserver.  Or for a power user with large chunks of data (400GB+) and 3rd party apps not installed via console;  a.k.a emphasis on the server part.  If emphasis is on the HOME then an expanded version of the file system, not a complete rewrite should be fine.  But, if there is the emphasis on the server part they should allow RAID & parity. Seriously, losing half of your space & more for ECC correction hardly seems like a good deal, esp. if Raid parity correction can reduce the loss to 25%.  Also, if VAIL is only meant to last more than 2 years, 16TB is great.  Otherwise it's a small amount of data really.  Most systems shipping today have 1-2GB pre-installed, and anyone having a homeserver will likely have at least 3 comps. (2 desktops, and a laptop is not uncommon at all).  So low ball that at 3GB backup + 2 TB media (even my grandparents have camcorders now and do simple video editing) + 12% overhead + 1/2 data loss for duplication = 16TB+ pretty quickly.   Why do you think HP re-added the ability for port multiplication even when all the servers come w/4 HD bays?  Because users demanded it. 

    • Edited by EPOL Monday, May 3, 2010 5:10 PM included previous post quote
    Monday, May 3, 2010 5:08 PM
  • I completely disagree as far as a software raid solution is concerned.  For one, there is already a solution in windows 2008 (albeit mirroring or RAID 5 -- not the best options) which this is based off of, and for another, the cost of implementing RAID would be cheap b/c the XOR calculations would all be done via CPU.  By the time VAIL is ready to ship, all mediasmart servers most likely will have i5 dual cores w/hyperthreading and at lowest spec, an i3 or regular core2duo (like they're shipping high end now) so the cost to implement would be nil, if any at all.  I know the emphasis is on HOME but it needs to be decided whether MS is going to stick to that or not.  Is it going to address the person that has storage needs and backup needs only?  In that case the guy w/the 400GB file should be a non-issue as well as non compatible 3rd party apps, b/c all third party apps for non-power uses will be installed via the console & designed specifically for the HOMEserver.  Or for a power user with large chunks of data (400GB+) and 3rd party apps not installed via console;  a.k.a emphasis on the server part.  If emphasis is on the HOME then an expanded version of the file system, not a complete rewrite should be fine.  But, if there is the emphasis on the server part they should allow RAID & parity. Seriously, losing half of your space & more for ECC correction hardly seems like a good deal, esp. if Raid parity correction can reduce the loss to 25%.  Also, if VAIL is only meant to last more than 2 years, 16TB is great.  Otherwise it's a small amount of data really.  Most systems shipping today have 1-2GB pre-installed, and anyone having a homeserver will likely have at least 3 comps. (2 desktops, and a laptop is not uncommon at all).  So low ball that at 3GB backup + 2 TB media (even my grandparents have camcorders now and do simple video editing) + 12% overhead + 1/2 data loss for duplication = 16TB+ pretty quickly.   Why do you think HP re-added the ability for port multiplication even when all the servers come w/4 HD bays?  Because users demanded it. 

    I know I should just leave this alone... but let's just give it one last try.

    RAID5 has its good points and its bad points.  There are tons of RAID5 implementations out there which can be had relatively inexpensively.  If anyone wants a RAID5 solution, they will be literally swimming in choices.  Certainly there is no need to buy a WHS if all you want is RAID5.

    DE is not RAID5.  It was not meant to be and was not designed with the same objectives.  The primary requirement was that heterogenous storage could be easily set up, managed and upgraded, and RAID5/0/1/10/6 are all not able to do that.  Any comparison of performance or efficiency is meaningless if a major requirement is not met.  That is not to say these technologies do not have their place, but that place is clearly not in WHS with its stated requirements.

    DE's current direction is an evolutionary response to known issues and limitations.  About all we know for sure at this point is that they will solve those known issues and limitations.  Any other critical flaws, inconveniences, or unexpected benefits of this new system will only be revealed in time.  If anyone has reason to believe there will be a problem, they should design a scientific experiment to explore that.  If there is adequate evidence, it will surely be of interest to all of us, including Microsoft.  Until there is such evidence, it will be hard to credit any such claims, no offense intended.

     

    Thursday, May 6, 2010 1:40 PM
  • Microsoft OS's aren't the only operating systems capable of reading NTFS partitions :)

    There was a time when they were, and everyone was up in arms about it.  The truth is, once the technology matures and the driver makes its way into other products that can be used for recovery, I think people will be fine.

    Overall, I'm happy with where this thing is going.  Nothing is keeping me from using hardware RAID at home with some server OS, but the fact is that stuff is a pain in the butt.

    Is it simply possible to limit the stripe width on the new technology?  Instead of potentially spanning a share (e.g. Movies Y:) across all drives in the pool, can the space be pre-allocated and the number of physical drives used to store the data pre-defined?  By default, most shares would contain an automatic setting that allows WHS to manage accordingly.

    One drawback to this approach is I'm boxing specific folders.  Any share I choose to explicitly allocate physical storage to may end up hoarding free space on my server.

    Friday, May 28, 2010 8:19 PM
  • I get the feeling that the vast majority of the people mad about these changes have multi-terabyte movie collections :)

    Is it possible to move this off to some sort of iSCSI device (there are plenty of open-source softwares for this) and then mount it in Vail as a non-pooled drive?  Could I then still use the interface and streaming features of Vail for my non-pooled content?

    Friday, May 28, 2010 8:24 PM
  • I get the feeling that the vast majority of the people mad about these changes have multi-terabyte movie collections :)

    Is it possible to move this off to some sort of iSCSI device (there are plenty of open-source softwares for this) and then mount it in Vail as a non-pooled drive?  Could I then still use the interface and streaming features of Vail for my non-pooled content?

    That is exactly what I do. I have currently 14TB in my Vail server and attached to it via iSCSI is a 16TB DroboPro. My movie collection is actually duplicated on both, but not duplicated again within Vail. I share the DroboPro and use it to serve the video as well as holding my iTunes media as well. All my iTunes machines have this mapped drive as the library. Of course as we know iTunes was spawned by the devil, so it only takes one of my family adding a new iTunes PC and leaving the "add to library as you add files" checked when adding all 9TB and it tries and duplicate everything in the iTunes library.

    If you are reading this and are saying to yourself "is this guy really saying that he is using his Vail server as an iTunes server for streaming and AppleTV, then yes. I am. It was horrid on the WHS v1 but it is working well on Vail.

    Ian

    Sunday, May 30, 2010 8:15 PM
  • For me, the dealbreakers are:

     

    Inability to see non-DE shares on the web interface/use the silverlight player etc

    Inability of the Web interface to see mkv files

    Inability to link a Windows Live ID to my user account, and therefore inability to use the streaming/DNLA capabilities of WMP12

    And a general lack of "killer application".

     

    I see nothing here (apart from the lovely web interface, which is useless to me due to the above) that would benefit me in any way over and above just using Windows 7 and a cheap RAID controller to do RAID5, in fact, this solution would let me use WMP12 streaming over the internet as I could link a LiveID

    Sunday, May 30, 2010 8:50 PM
  • I just bought a Asus Motherboard M4A785-M from Bestbuy for $ 90.00 with Raid.

    http://www.bestbuy.com/site/olspage.jsp;jsessionid=74D10B4571D60B9AD831560D4EC67C91.bbolsp-app01-41?_dyncharset=ISO-8859-1&_dynSessConf=-7918441247045864502&id=pcat17071&type=page&st=M4A785-M&sc=Global&cp=1&nrp=15&sp=&qp=&list=n&iht=y&usc=All+Categories&ks=960

    You should have the same debate today... And not keep refering to a debate from years ago...

     

     


    IMPORTANT NOTICE: This communication, including any attachment, contains information that may be confidential or privileged, and is intended solely for the entity or individual to whom it is addressed. If you are not the intended recipient, you should contact the sender and delete the message. Any unauthorized disclosure, copying, or distribution of this message is strictly prohibited. Nothing in this email, including any attachment, is intended to be a legally binding signature.
    Wednesday, June 9, 2010 12:49 AM
  • Wow I just read though this whole thing and I must say Vail is a complete deal breaker if I understand how the new DE system works. So to recap here: You've basically reimplemented a type of RAID, such that I can no longer pull a drive and slap it in my desktop to recover what I can? This is insane, the whole reason WHS was so attractive to me is the fact that you have a quasi RAID without all the draw backs of losing everything if you just happen to have 2 disks die on you.

    Its being argued on this thread (in the later portion) that 'In all fairness, not many systems, including WHS v1 with full duplication, can fully survive a simultaneous two drive failure'. I won't argue that point but the thing is in RAID (and apparently this 'Improved' DE) you're totally FUBAR unless you have some backup system, where as in WHS v1 you can at least try to pickup the pieces. I'd rather get SOMETHING back out of my storage pool as opposed to NOTHING.

    As to those arguing that they can do a 'full backup on another box' I'd like to see you fork up the money to build another 25tb storage pool, or heck even some of the 100tb storage pools like the kids at MediaSmartServer.net have going.

    As to the story of the guy with the 400gb file, that's just bad practice in my opinion, as others have stated he should have split the files up in something more reasonable or looked to a solution other than WHS.

    I have to wonder if the mark was missed here, in my mind the largest reasonable file size is 50gb, the size of a dual layer bluray disk. Heck even at this point lets say the average is closer to 4.7-10gb (DVD) I don't think its unreasonable to expect the storage pool to consist of drives at least 80gb in size. The cost of drives has dropped dramatically in the past few years and the densities have soared. To try to optimize the file system for unrealistic scenarios makes zero sense to me. And depending on whose kool-aide you drink 'RAID is Dead'.

    On the bright side of all this at least there was a beta to get this out there in the open instead of just throwing it as a nasty surprise when we get Vail RTM. I've added my vote into the Microsoft connect site and I've informed everyone I know to do the same.

    Thanks.

    Tuesday, July 13, 2010 4:10 AM
  • I would just like to thank Microsoft for developing Vail. Although a long time user and beta tester of WHSv1, I have sought out new solutions to my storage/server needs. If it weren't for Vail I would have never discovered ZFS which is available on Opensolarais, FreeBSD, and Nexentastore just to name a few. Maybe its not fair as I'm not comparing apples to apples. Microsoft is developing Vail with a specific set of uses and users in mind and I no longer fit into their projected demographic.  My storage needs outweigh my backup needs and many other backup solutions are available. In the storage area, I can't afford a 66% overhead to protect my data from just ONE drive failure. With ZFS an 8 drive raidz2 device has only 20% overhead and will protect against 2 simultaneous drive failures. When hard drives are free it won't matter.

    Microsoft should seriously buy the rights to ZFS or develop a system as good, instead of trying to reinvent the wheel. Microsoft.. your wheel isn't round.  

    If anyone hasn't seen ZFS in action, check out this quick video http://www.youtube.com/watch?v=QGIwg6ye1gE

    Tuesday, July 13, 2010 2:05 PM
  • I would say being able to get the drive and shuve it in another machine to 'pick up the pieces' is no deal breaker.

     

    Serious IT folk will know:

     

    1. Any good filesystem like WHS v2, ZFS and most RAID solutions will not allow you to remove a drive and read it from another machine.

    2. It's entirely possible MS will add this feature in anyway.

    3. If you have to rely on being able to remove a drive and read it on another machine, your server drive system is not designed correctly.

    4. ZFS is fine but it's set and forget, you have to forget it because there's no adding drives to a pool.

    For me and most other serious IT people, DE v2 will be no 'deal breaker' and doesn't take away any features that a serious filesystem wouldn't have anyway. I think people like the 'idea' that they can put their drive in another system but don't realise that in the end, they cannot do this with most RAID systems anyway, because you shouldn't need to..

     

    In any case, you cannot have CRC without getting rid of NTFS, so it's a welcome change here and, once the new BETA comes out, which will hopefully work for people outside the US, I will be giving it the run-through again.

     

     

     

    Wednesday, July 14, 2010 10:57 PM
  • I thought VAIL was targeted to home users.
     
    On Wed, 14 Jul 2010 22:57:36 +0000, David775 wrote:
     
    >For me and most other serious IT people, DE v2 will be no 'deal breaker' and doesn't take away any features that a serious filesystem wouldn't have anyway. I think people like the 'idea' that they can put their drive in another system but don't realise that in the end, they cannot do this with most RAID systems anyway, because you shouldn't need to..
     

    Barb Bowman

    http://www.digitalmediaphile.com

    Thursday, July 15, 2010 9:02 AM
  • "I thought VAIL was targeted to home users. "

    So did I.. but then WHS v1 was 'supposed' to be targeted to the average home user. Yet, it has failed to become common among multi-computer home users, and in fact is pretty well unheard of among same.

    IMO... they need to trim Vail to be the simple solution for multi-computer home users, and let “Aurora” be the solution for the techy-type enthusiast and the small business users. IMO, it is very simple (although not simple to do perhaps) what is needed out of a home server:

    1. First it should be a reliable one-only backup system - in other words not necessitating having 'any' other backup system or program - it needs to be fail-safe, short of the house burning down.
    2. It should serve the home user the programs and media etc., that makes it the one-machine-only source for all stored media content (including saved TV).
    3. It shouldn't require the user to either be a techy or have to hire one in order to run it.

    Art [artfudd] Folden
    --
    "Barb Bowman [MVP]" wrote in message news:894113a1-41f8-4f57-95ef-f25c4895501c@communitybridge.codeplex.com...

    I thought VAIL was targeted to home users.

    On Wed, 14 Jul 2010 22:57:36 +0000, David775 wrote:

    For me and most other serious IT people, DE v2 will be no 'deal breaker' and doesn't take away any features that a serious filesystem wouldn't have anyway. I think people like the 'idea' that they can put their drive in another system but don't realise that in the end, they cannot do this with most RAID systems anyway, because you shouldn't need to..

    -- Barb Bowman
     http://www.digitalmediaphile.com

    Thursday, July 15, 2010 5:59 PM
  • Hi BulatS,

    Just to throw in my 2 pence worth :

    28 Drives - > WHS V.1 PP3

    1x 750GB -> System

    5x 1TB

    16x 1.5 TB

    6x 2TB

    Current Cap : 38889GB

    So anything that says , corruption over 10 or 16 drives is scary.

    • Edited by Da1ek Sunday, August 8, 2010 12:11 AM cant count..
    Sunday, August 8, 2010 12:10 AM
  • That's a lot of TB's.
    I'm real ínterested in the external backup strategy & method for this configuration.


    .
    .
    .
    .

    "Da1ek" wrote in message news:becc3059-f28c-47c1-907c-b3e5edf26d2b@communitybridge.codeplex.com...

    Hi BulatS,

    Just to throw in my 2 pence worth :

    28 Drives - > WHS V.1 PP3

    1x 750GB -> System

    5x 1TB

    16x 1.5 TB

    6x 2TB

    Current Cap : 38889GB

    So anything that says , corruption over 10 or 16 drives is scary.


    Have a nice day!
    Sunday, August 8, 2010 7:15 AM
  • Hi Leen,

    The strategy is based on the type of content,

    Most of my stored movies come from Blu-Ray / HD-DVD / DVD and are therefore not backed up given the source discs are the offline backup. A percentage of my TV shows also source from DVD so again , similar.

    Everything else I want to back up, some tv recordings , mp3s , photos , files , etc etc etc , goes out to a an LTO-4 Tape set once every 6 months, that gets held at my dads place 400 miles away :) and cycled during the most convient visit.

    5 tapes x 800gb = 4TB.

    Tho once the LTO 5 drives drop in price a bit (1.5TB per tape is lovely but £1500 is a bit rich for my blood at the mo ) and a cheap autoloader maybe ( ebay here I come ) I can see me moving in that direction and backing up more, and more often... ( 12 tapes x 1.5 = 18TB ). 

    As we all know , Home Server v.1 , effectiively means you half the amount of storage you have , so my near 38TB , really becomes 19TB, of which some is free ( duplicated ) space etc

    Break down in terms of real data ( before duplication ) :-

    • Must back up 3.1TB 
    • Comfortable with WHS Duplication alone 14TB ( 340Blu Rays , 130 HD-DVDs , 1000+ DVDs )

    So to my mind , WHS gives me a sensible way to prevent having to worry about drive failures , or mother board failures etc, but i wouldnt trust it with anything i think i really really must keep, after all homeserver has no ' make the server fire and flood proof ' feature :)

    Cheers

     

     

     

    • Edited by Da1ek Sunday, August 8, 2010 9:42 AM missing word Duh!
    Sunday, August 8, 2010 9:40 AM
  • ...
    So anything that says , corruption over 10 or 16 drives is scary.

    This is a limitation of the current build, and may be raised in future builds. It's also my understanding that it represents a comfort level, not a hard stop. So you could conceivably test with more drives if you wanted to.

    But please understand that 16x2TB drives is a whole lot of storage for the average home user. And when 3 TB drives are available it will be even more.


    I'm not on the WHS team, I just post a lot. :)
    Sunday, August 8, 2010 4:57 PM
    Moderator
  • ...
    So anything that says , corruption over 10 or 16 drives is scary.

    This is a limitation of the current build, and may be raised in future builds. It's also my understanding that it represents a comfort level, not a hard stop. So you could conceivably test with more drives if you wanted to.

    But please understand that 16x2TB drives is a whole lot of storage for the average home user. And when 3 TB drives are available it will be even more.


    I'm not on the WHS team, I just post a lot. :)

    I suppose that there will be this time exact details of the boundaries of the RTM version?
    Sunday, August 8, 2010 6:13 PM
  • I suppose that there will be this time exact details of the boundaries of the RTM version?
    I'm sure if there are any hard drive limitations on RTM, MS will say so when the time comes.
    Sunday, August 8, 2010 6:17 PM
    Moderator
  • So in summary:: The WHS users with enormous multi TB movie collections are less happy than the actual target audience for the product.

    Looks like MS got DE just right.

     

     

    Saturday, August 14, 2010 9:35 PM
  • That's really about the size of it, scoob. And I literally laughed out loud. :)

    Personally, I think it would be great if Microsoft could find a way to make the people with enormous movie collections happy without compromising performance and reliability for the target audience. But if Microsoft has to choose one or the other, it's a decision that should really make itself.


    I'm not on the WHS team, I just post a lot. :)
    Sunday, August 15, 2010 2:36 AM
    Moderator
  • How about just giving Vail the ability to read / write normally formatted NTFS volumes if the user wants that?  I don't see ANY reason why the whole drive extender architecture is being forced on everyone, even those who don't need it.  People seem to forget that WHS is more than just storage.
    Monday, August 16, 2010 8:32 PM
  • So in summary:: The WHS users with enormous multi TB movie collections are less happy than the actual target audience for the product.

    Looks like MS got DE just right.

     

     


    Please explain what the target audience is then?  Last time I looked the WHS site was advertising it as an expandable media storage hub for the home.  Having a large movie collection hardly seems outside of that scope does it?
    Monday, August 16, 2010 8:48 PM
  • That's really about the size of it, scoob. And I literally laughed out loud. :)

    Personally, I think it would be great if Microsoft could find a way to make the people with enormous movie collections happy without compromising performance and reliability for the target audience. But if Microsoft has to choose one or the other, it's a decision that should really make itself.


    I'm not on the WHS team, I just post a lot. :)


    Surely it's that not difficult, as Vail can operate on a single drive some logic must already be programmed to stop files over 1GB from being split across drives when duplication is turned off, otherwise it couldn't operate on a single drive.  Removing splitting on non-duplicated shares would equate to the same level of risk as WHS1, not a backward step.

    Or...force duplication on all shares, with no option to turn it off.  Of course that will make Vail systems uncompetitive when compared against RAID5 NAS boxes as it will be harder for OEMs to advertise them with their maximum storage capacity v. true available capacity (duplication on) ;)

    Monday, August 16, 2010 8:55 PM
  • Surely it's that not difficult, as Vail can operate on a single drive some logic must already be programmed to stop files over 1GB from being split across drives when duplication is turned off, otherwise it couldn't operate on a single drive.  Removing splitting on non-duplicated shares would equate to the same level of risk as WHS1, not a backward step.

    Or...force duplication on all shares, with no option to turn it off.  Of course that will make Vail systems uncompetitive when compared against RAID5 NAS boxes as it will be harder for OEMs to advertise them with their maximum storage capacity v. true available capacity (duplication on) ;)

    Forcing duplication would be my own preference. I use it for all shares now.

    But you've got the first bit backward. There's undoubtedly a single algorithm in Drive Extender V2 today which allocates blocks on all available drives all the time. If there's one drive, all blocks go on that drive. If there are 10 drives, 10% will wind up on each drive. What you want is an additional allocation algorithm, and I don't see it happening, for performance, reliability, and cost reasons.


    I'm not on the WHS team, I just post a lot. :)
    Monday, August 16, 2010 11:32 PM
    Moderator
  • Surely it's that not difficult, as Vail can operate on a single drive some logic must already be programmed to stop files over 1GB from being split across drives when duplication is turned off, otherwise it couldn't operate on a single drive.  Removing splitting on non-duplicated shares would equate to the same level of risk as WHS1, not a backward step.

    Or...force duplication on all shares, with no option to turn it off.  Of course that will make Vail systems uncompetitive when compared against RAID5 NAS boxes as it will be harder for OEMs to advertise them with their maximum storage capacity v. true available capacity (duplication on) ;)

    Forcing duplication would be my own preference. I use it for all shares now.

    But you've got the first bit backward. There's undoubtedly a single algorithm in Drive Extender V2 today which allocates blocks on all available drives all the time. If there's one drive, all blocks go on that drive. If there are 10 drives, 10% will wind up on each drive. What you want is an additional allocation algorithm, and I don't see it happening, for performance, reliability, and cost reasons.


    I'm not on the WHS team, I just post a lot. :)


    Your preferences all due, but you're ready to abandon your origin preference (duplication) for a forced duplication speaks volumes.

    Nevertheless hard to imagine that Microsoft engineers have lost in a storage technology that is only to save by it enforces duplication. Making duplication a mandatory matter, instead of a flexible option, is the admission that a first order balance between performance and easy to use (storage) features on the one hand, data security on the other hand, failed.

    Besides consider that forced duplication has the effect of an additional tax on the purchase of hard drives for WHS: buy 100% get 50%
    The prices for hard drives fall MS increases again. The slogan "easy to expand" seems like a farce.

    This is a serious problem. If before the purchase of WHS not completely clarifies each drive on storage capacity is value only half an illegal invasion of the operating system manufacturer in the right of the purchaser the full capacity of the hard drive to have that promised the manufacturer may be.

    Ken, You want to give additionally a marketing chellenge to Microsoft?
    I am afraid that your preferences are occasionally something unfortunate. Even Microsoft cannot exploit and increase a small homeserver market with the help of a storage guzzling spoon-feeding machine.

    You can mistaken even, isn't it, Ken.

    And I prefer to make my own decisions, with WHS or without it, if necessary. But enforced duplication is the absolute deal breaker.

    Tuesday, August 17, 2010 3:57 PM
  • Surely it's that not difficult, as Vail can operate on a single drive some logic must already be programmed to stop files over 1GB from being split across drives when duplication is turned off, otherwise it couldn't operate on a single drive.  Removing splitting on non-duplicated shares would equate to the same level of risk as WHS1, not a backward step.

    Or...force duplication on all shares, with no option to turn it off.  Of course that will make Vail systems uncompetitive when compared against RAID5 NAS boxes as it will be harder for OEMs to advertise them with their maximum storage capacity v. true available capacity (duplication on) ;)

    Forcing duplication would be my own preference. I use it for all shares now.

    But you've got the first bit backward. There's undoubtedly a single algorithm in Drive Extender V2 today which allocates blocks on all available drives all the time. If there's one drive, all blocks go on that drive. If there are 10 drives, 10% will wind up on each drive. What you want is an additional allocation algorithm, and I don't see it happening, for performance, reliability, and cost reasons.


    I'm not on the WHS team, I just post a lot. :)


    Ken, I know your preference from other posts.  But simply MS have chosen not to enforce duplication.  Given the backward, almost catastrophic, results if duplication is not switched on - and this applies to more than just users with large movie collections - then if the design of DE is simply set in stone, then duplication must be removed as an option.

    And I don't think I've got the bit backward, in fact you're agreeing with my sentiment.  Yes a change is needed, but surely it is not as significant as your original post suggesting it's one thing or another states.

    Please elaborate as to why performance and reliability are an issue - don't simply throw a one-liner statement like that in without justification to kill the conversation - just because this is not an issue that will affect you, or doesn't tie in with your view of WHS.  In terms of performance, are you saying Vail needs striping to perform effectively?  This further reinforces that duplication must not be an option, and that minimum specs require more than one HD.  Or that extra processing will be required within DE, again - though I doubt it should be significant - then it shouldn't bump up minimum specs significantly.

    In terms of reliability, this is a moot subject, as without duplication, any file over 1GB on Vail is at greater risk than V1, so in that sense less reliable.  As for cost, yes there will be additional work to be done.  I'm not privy to MS's budget for developing Vail, we've all been merely asked to provide feedback.  As I've said before it's MS's decision as to what they develop or do not develop - but some official feedback would be nice instead of just opinions on the matter.

    Tuesday, August 17, 2010 4:55 PM
  • It's surprising to me that people are still complaining about the same old issues and I wonder if they even bothered to read the earlier posts.

    I don't have anything more to add, so I am going to do something a little off topic, in an effort to restore a little perspective to the complaining public.

    Top reasons I like Vail (over v1)

    1. Working shadow copies.

    2. No longer disables shares when removing drives.

    3. Working server backup and restore model.

    4. Client backups are protected from failure.

    5. No longer bogs down with demigrator activity.

    6. No more open file duplication warnings.

    7. Protects from silent failures (supposedly, but not tested, don't know any good way to do that without killing my hardware)

    Top reasons I prefer v1

    1. Able to port drives to other PCs for recovery.

    2. Slightly better efficiency.

    As I understand it, a lot of the things Vail does were only possible using the new block level DE.  Food for thought.

     

    Wednesday, August 18, 2010 4:35 AM
  • So in summary:: The WHS users with enormous multi TB movie collections are less happy than the actual target audience for the product.

    Looks like MS got DE just right.

    Where did MS exactly get it right?

    The latest beta came with at least two showstoppers so even site that fully supported WHS before does not recommend installing it. It is indication that Vail is broken, not that something is right.
    http://www.mediasmartserver.net/2010/08/16/opinion-thoughts-on-the-vail-rc0-release/

    So far what MS did was to cut off people with large amount of data from using Vail. One thing for sure is that Vail will require much more storage in order to operate on the same level of data safety as WHS v1.
    There is no indication of success otherwise. No major OEM announced Vail system, no public survey show desire to switch to Vail.
    Vail is too far from being ready to be released so it's too early to say if it is right or not. It can be as dead as Kin phone in a few months.

    I think WHS v1 combined with 3rd party tools is better product than Vail even when/if Vail gets its bugs fixed.
    All WHS v1 drawbacks can be mitigated with 3rd party tools and proper system configuration.
    Nothing can compensate for a need to buy ~130% more storage than needed in Vail and inability to read data in other PCs.

    Wednesday, August 18, 2010 7:10 AM
  • I just installed Vail Preview 'Refresh' (I skipped the first build of Vail) only to find I can not install the connector on a Windows Server 2008 (not R2) 32bit VM I have running. WHSv1 connector installs and backs up this machine fine.

    Is this a product limitation of Vail, or of the beta release? Currently (for me at least) this is a deal breaker!

    Wednesday, August 18, 2010 7:29 AM
  • Nic, backing up server operating systems has never been supported (although it's possible to install the V1 connector).

    And yes, this is by design. Microsoft would probably say (and I would agree) that Server Backup is more flexible, and offers better performance and reliability for the enterprise, than Windows Home Server backup for a server. And of course third party solutions are generally even more flexible.


    I'm not on the WHS team, I just post a lot. :)
    Wednesday, August 18, 2010 12:15 PM
    Moderator
  • Not just those with ripped collections.  Realize, a big perk - in fact, something built in to even WHS1 is the Media Center TV Archive.  As I point out above, a recording of HD in Windows MCE7 is 7.5GB/Hr.  So, a 2 hr show is 15GB (and can be more, depending on the content)


    So, for example, I went to a client using MCE and set them up with a Ceton 4-tuner cablecard solution last night.  They already have 400GB of TV archived, but since they have theirs set on "delete as space needed" (same here) that will grow considerably as they archive away TV shows they want to keep tons of.  Think:  Episodes of "Dora" "Blues Clues" "Oobi"  Etc.

     

    ____, at my house, I'm storing over 2TB of TV that has been recorded using MCE.  And, maintaining backups of our PCs - 4 of them plus a laptop in our house + an Itunes collection (currently 8TB in Itunes..) isn't small, and small business clients, who MS handed out tons of partner info for in the last shipment store way more then that.. I've got a client who stores more then 20TB in their WHS, and it isn't "movies"  It's X-Rays of people's teeth :)  As a two person dental office, it was a better solution for them the redoing everything with SBS, etc... and they migrated from a NAS box.

     

    But since Data corruption is still a serious issue in even the newest build, something here is amiss.  I agree with the above, I'm not as worried about removing a drive and reading it elsewhere.. it's a nice but not necessary touch.  But I am concerned that if you lose a single HDD, the entire thing falls apart like a Jenga puzzle crashing to the ground.   Because that isn't data security.

    As far as my personal concern:  Yes, I could lose everything in my WHS and I'd be pissed.. but it's just ISOs of BD, TV, etc. and I still have all the physical copies of those discs in my collection.  It'd be a lot of work to put them all back, but it could be done.  And anyone who worries about "it's not gettable" is just pirating stuff, so I don't see why MS should worry about that.  The backup for most of my data like that is that I have the physical copy of it in my attic.  The only reason I ISO and hide the originals is because I dont' want my autistic son near any of it, as he'll destroy the physical media (been through that twice before).  And having them all available for us is a nice benefit.  But that's the only real backup I need on that stuff.

    Wednesday, August 18, 2010 12:31 PM
  • PMFranklin, there's nothing magical about large files in moderate quantites. Your issue is large files in extremely large quantities. For users with extremely large collections of large files (multi-terabyte movie collections in just about all cases, I believe) I agree that the block-level allocation algorithms in Vail present challenges not present in V1, and I understand why such users don't like the Vail storage allocation mechanism. Unfortunately, those users represent a fairly small fraction of all Windows Home Server users today, and Vail's design doesn't put them first.

    For the average user Vail offers exactly as much (as little) reliability, absent duplication, as V1. For such users, a single share won't be more than 2 TB in size (probably the largest won't be more than a few hundred GB in size), so it won't have to span multiple drives. In this case, if duplication isn't in use, a single share will fit on a single drive, and you may lose all the files in that share if a drive fails. This is true for both versions of the product, and the size of an individual file doesn't matter. I think the documentation is quite clear for V1: duplication protects your files from the loss of a single drive. In any share where duplication is turned off, every file (100%) is potentially at risk if a drive fails.

    As for performance and reliability: Add in multiple storage allocation algorithms and code to choose between them, and you sacrifice performance. More code to be executed = more overhead = lower performance. Maybe it will be a measurable drop, maybe not. I can't say. I know that Vail performs as well as, or slightly better than, V1 on identical hardware for me today with code that I presume isn't fully optimized yet. YMMV. As for reliability, that same additional code introduces additional paths that need to be tested, and more opportunity for bugs.

    And no, I don't agree with you. To be clear, the statement to which I was responding was "... as Vail can operate on a single drive some logic must already be programmed to stop files over 1GB from being split across drives when duplication is turned off, ..." No such logic is required: whether there's one drive or 10, blocks will be allocated across all available drives.

    Now, to be very clear, what you and others are complaining about is that Microsoft has chosen to change undocumented internal functionality, the storage allocation mechanism. They've done away with file level storage allocation and moved to a block allocation algorithm. Undocumented features are always subject to change without notice, and should never be relied upon to remain "as is" in future versions of any product (remember all the times people have been reminded by Kariya21, Olaf, myself, and others that nothing you see on your server desktop is "supported" for end users?), so complaining about it isn't really a productive use of anybody's time. If this change is truly a deal breaker, Windows Server 2008 R2 Standard is right this way.


    I'm not on the WHS team, I just post a lot. :)
    Wednesday, August 18, 2010 1:12 PM
    Moderator

  •  

    For the average user Vail offers exactly as much (as little) reliability, absent duplication, as V1. For such users, a single share won't be more than 2 TB in size (probably the largest won't be more than a few hundred GB in size), so it won't have to span multiple drives. In this case, if duplication isn't in use, a single share will fit on a single drive, and you may lose all the files in that share if a drive fails. This is true for both versions of the product, and the size of an individual file doesn't matter. I think the documentation is quite clear for V1: duplication protects your files from the loss of a single drive. In any share where duplication is turned off, every file (100%) is potentially at risk if a drive fails.


    I'm not on the WHS team, I just post a lot. :)

    aHow do you figure this?  Let's take an HP MediaSmart Server.   2 500GB Drives in the basic from a few years ago.  Let's say hardware blows out; bad caps or a CPU burns up or power failure kills the PSU and you need your data off.  Pull the two HDD, put in an external box and move it whever.  How much did that user have?  Only their pictures from cameras/etc.  yes, a few hundred GB in family photos from the last two years (now that 10MP+ cameras are common, this is becoming more of a thing). 

    So, WHS hardware blows up, and they still can get their data.

     

    User with Vail, installs it on their HP MediaSmart to replace WHS.  Puts all their data back.  Something happens and their data is... poached.  Completely poached.  All lost because the drives can't be read elsewhere, and unless they run out restoring the OS on different hardware has been (so far in my tests using an attempted conversion from Hyper-V to physical) a complete crapshoot as to whether it reads data back in.


    So, complete and total data loss.  For someone who didn't have terrabytes of movies and the like.

    Compared to WHSv1, where they keep everything.

    Can we explain how this means Vail offers "exactly as much" reliability and safety?  Because it seems to this vendor like it's a ____ of a lot less safety, no matter what your end goal, the moment you go over 1 HDD, for no matter what purpose and file type.

    Wednesday, August 18, 2010 1:38 PM
  • You're still talking about undocumented functionality.

    In V1, Microsoft's answer to your scenario will be a server recovery, not moving drives around to copy files from a hidden location. That's a functional and flexible workaround that avoids server recovery, which can be problematic on home-built servers due to configuration issues. Instead of recovery you progress directly to a fresh installation...

    In Vail, Microsoft has two answers: 1) pull your server storage, or as much as still works, connect it to another Vail server (running a compatible version of DE), and copy files out of "Non Default Server Storage", or 2) reinstall, which will (in the present version) leave you in the same position: your data will be in "Non Default Server Storage".

    In addition, there's this suggestion on Connect, which (if implemented) will allow functionality equivalent to V1. (It's the number 1 suggestion for Vail by number of "up" votes, for what that's worth.) I admit that connecting drives one at a time to recover data is going to be less useful in Vail than V1, because of the large file issue, so server recovery becomes more important to end users.

    And you don't "install" Vail on your HP MediaSmart Server. If there's an upgrade path there (it will be up to HP to make that decision, but the EX49 series meets the minimum specs so it's at least possible), it will come from HP and will require wiping the server. It's unlikely that you'll receive any support for an upgrade performed any other way.


    I'm not on the WHS team, I just post a lot. :)
    Wednesday, August 18, 2010 2:17 PM
    Moderator
  • You're still talking about undocumented functionality.


    Most of us are talking real world here.  My data lives in the real world not in some MS Lab or specification.
    --
    Wednesday, August 18, 2010 2:45 PM
  • Nic, backing up server operating systems has never been supported (although it's possible to install the V1 connector).

    And yes, this is by design. Microsoft would probably say (and I would agree) that Server Backup is more flexible, and offers better performance and reliability for the enterprise, than Windows Home Server backup for a server. And of course third party solutions are generally even more flexible.


    I'm not on the WHS team, I just post a lot. :)
    Thanks for the confirmation Ken, I suspected as much. I shall investigate shoehorning the client connector on and see how I get on. As a developer I like to use the various server OS's MS offer, but I'd also like a single backup solution for my local lan, WHSv1 achieved this for me, I hope Vail can do the same. PS. I fully understand this is not going to be a supported scenario! Wish me luck... -Nic
    Wednesday, August 18, 2010 5:49 PM
  • PMFranklin, there's nothing magical about large files in moderate quantites. Your issue is large files in extremely large quantities. For users with extremely large collections of large files (multi-terabyte movie collections in just about all cases, I believe) I agree that the block-level allocation algorithms in Vail present challenges not present in V1, and I understand why such users don't like the Vail storage allocation mechanism. Unfortunately, those users represent a fairly small fraction of all Windows Home Server users today, and Vail's design doesn't put them first.

    For the average user Vail offers exactly as much (as little) reliability, absent duplication, as V1. For such users, a single share won't be more than 2 TB in size (probably the largest won't be more than a few hundred GB in size), so it won't have to span multiple drives. In this case, if duplication isn't in use, a single share will fit on a single drive, and you may lose all the files in that share if a drive fails. This is true for both versions of the product, and the size of an individual file doesn't matter. I think the documentation is quite clear for V1: duplication protects your files from the loss of a single drive. In any share where duplication is turned off, every file (100%) is potentially at risk if a drive fails.

    As for performance and reliability: Add in multiple storage allocation algorithms and code to choose between them, and you sacrifice performance. More code to be executed = more overhead = lower performance. Maybe it will be a measurable drop, maybe not. I can't say. I know that Vail performs as well as, or slightly better than, V1 on identical hardware for me today with code that I presume isn't fully optimized yet. YMMV. As for reliability, that same additional code introduces additional paths that need to be tested, and more opportunity for bugs.

    And no, I don't agree with you. To be clear, the statement to which I was responding was "... as Vail can operate on a single drive some logic must already be programmed to stop files over 1GB from being split across drives when duplication is turned off, ..." No such logic is required: whether there's one drive or 10, blocks will be allocated across all available drives.

    Now, to be very clear, what you and others are complaining about is that Microsoft has chosen to change undocumented internal functionality, the storage allocation mechanism. They've done away with file level storage allocation and moved to a block allocation algorithm. Undocumented features are always subject to change without notice, and should never be relied upon to remain "as is" in future versions of any product (remember all the times people have been reminded by Kariya21, Olaf, myself, and others that nothing you see on your server desktop is "supported" for end users?), so complaining about it isn't really a productive use of anybody's time. If this change is truly a deal breaker, Windows Server 2008 R2 Standard is right this way.


    I'm not on the WHS team, I just post a lot. :)

    1.  Of course the number of files are relevant, but let's be more specific it's actually the number of drives that increases the risk.  Can you also back up your opinion with some facts on the number of users...the last time we had this discussion it was based on the limited number of people you had talked to and you had no actual stats on worldwide sales.

    2. We are not talking about people who only have a few small shares.  I, amongst others, have raised a specific scenario.  Comparing Vail to WHS1 in this respect is an insult.  What is an average Vail user and what is a target user?  I asked this earlier and it wasn't answered?

    3.  I don't have an issue with performance on either at present.  But the issue is whether changing the algorithim would have a massive hit on performance or not.  If it would then agreed, a decision needs to be made, but nothing has been said on the matter.

    4.  We agree in that a change would be required.

    5.  I am not complaining.  This thread is marked deal breakers/complaints - so I should be entitled to anyway.  But I am presenting an opinion in response to others dismissing a specific scenario, and would like some official response.  I am not talking about undocumented internal functionality - in fact I don't care as an end-user - and that went right over my head.  As an end-user I am talking about my expectations from one version of a product to another based on using WHS1 and that Vail has taken a step backwards in the specific scenario under discussion here.  And let me be very clear here, other than the fact my WHS is home built, I am not using it in any way that lends itself to being "unsupported".

    I really feel quite insulted by your last remarks.  I have not mentioned anything about accessing the server desktop - this scenario exists straight out of the box using it in a fully supported manner - your comments are totally unrelated to this discussion and misleading suggesting the issue only exists when using WHS non "supported" scenarios.  Clearly you do not understand my requirements for a simple to setup server to stream media to clients around the house, retain important information and backup clients.  Doesn't that tie-in with how MS advertise WHS on their WHS site?  Windows Server 2008 R2 Standard does not meet those requirements. 

    It's also very clear that unless posters' feedback and opinions matches with certain people's vision of WHS on this site, then they are not welcome.  That's a shame, I thought this was a beta where we were able to provide feedback, not be brow beaten into submission by non MS people to accept an inferior product.  And let me be clear again, I've fed into this issue a number of times, and generally only raised my head when some people are trying to say it is not an issue at all.  I will accept MS decision once they have analysed the problems and decided if it is something they will resolve, but I don't like to sit by when people dismiss it.  I don't dismiss other people's feedback simply because it doesn't fit within my own vision, e.g. I understand the desire for WMC on WHS, but it's not that important to me...

    So perhaps you could try and accept that when you next respond.  You will not convince me that it is not an issue, and that I am using it in an unsupported way and please respect my opinion.  Similarly I will respect your opinion that it is not how you would choose to use WHS and is not that important.  Likewise we should both accept MS will have to analyse the issue and see if it is something they can technically do with the resource and budget available to them.  It may not happen, I accept that, but I won't accept your opinion on MS's internal priorities just to shut us up - unless your sig is now out of date ;) 

    Wednesday, August 18, 2010 6:18 PM
  • You're still talking about undocumented functionality.

    In V1, Microsoft's answer to your scenario will be a server recovery, not moving drives around to copy files from a hidden location. That's a functional and flexible workaround that avoids server recovery, which can be problematic on home-built servers due to configuration issues. Instead of recovery you progress directly to a fresh installation...

    In Vail, Microsoft has two answers: 1) pull your server storage, or as much as still works, connect it to another Vail server (running a compatible version of DE), and copy files out of "Non Default Server Storage", or 2) reinstall, which will (in the present version) leave you in the same position: your data will be in "Non Default Server Storage".

    In addition, there's this suggestion on Connect , which (if implemented) will allow functionality equivalent to V1. (It's the number 1 suggestion for Vail by number of "up" votes, for what that's worth.) I admit that connecting drives one at a time to recover data is going to be less useful in Vail than V1, because of the large file issue, so server recovery becomes more important to end users.

    And you don't "install" Vail on your HP MediaSmart Server. If there's an upgrade path there (it will be up to HP to make that decision, but the EX49 series meets the minimum specs so it's at least possible ), it will come from HP and will require wiping the server. It's unlikely that you'll receive any support for an upgrade performed any other way.


    I'm not on the WHS team, I just post a lot. :)


    Do you know another way to go to Vail on an HP MediaSmart, or any hardware without installing it that I have never heard of?  Realize, even if an "Upgrade" is not an option, the inbuilt audience for this product is the people who already have a WHS who will want to upgrade.  If they have to copy all their data off and reformat, several of them will do it.  I used the HP MediaSmart because a bunch of them shipped with 2 500GB drives, so an end user can easily backup the data to a single external and re-install.  Same with Acer and Asus WHS boxes.

    But, let's say you have a real hardware failure on one of those.  I'm not talking a drive, I'm talking something else.  Again, I will use the example of a local photographer I work with who has an HP MediaSmart.  Motherboard failed (overheat/bad caps) after a likely PSU failure.  So, we were able to get all of his data quickly.  Yes, undocumented feature but real world.

    When we moved the drives over to another machine and re-installed Home Server (drives were fine) it had NO ability to read back the tombstones and data was gonzo.  Which was something we considered which is why we backed up the data first.

    All is well.  And this is from someone who planned.

    Imagine home user John Smith.  The have an an HP MediaSmart running WHS1.  Two drives.  They run into the exact same problem.  They have never connected an external backup device or whatever because they viewed WHS as their backup.   Still, they can come to someone like us or anyone and get their data.  All is right with the world.

    Imagine home user Jane Smith.  They have an HP MediaSmart running Vail.  2 drives storing their pictures, etc.   Motherboard goes bad for whatever reason.  And their ability to quickly recover data is... sucky.  Like really sucky.  If it doesn't find the markers they are screwed.  They need to replicate hardware.

    At this point, taking a WHS Vail install on a Single Hyper-V VHD and then removing it from Hyper-V 2008R2 server and opening it up in VirtualPC in Windows7.. and all the data goes gozno.  So, how far am I going to trust that on real world hardware?

    And why is the concept of the ability to make sure you preserve data without loss in multiple drives so foreign to MS.   That seems ridiculous.  IPU, you've got products like FreeNas that are.. Free.. and will balance 100 drives if you want it to, and the whole thing could blow up and you'd still have your data.

    What I'm seeing is a product that gives no one who has WHSV1 a real reason to upgrade.  Time Machine is nice.   Welp, doesn't impact me.  Heck, I'm trying to think of what benefit can trumpcard wasting tons of drive space and offering very little data security along with that loss as an upgrade.

    Wednesday, August 18, 2010 8:11 PM
  •  

    Imagine home user Jane Smith.  They have an HP MediaSmart running Vail.  2 drives storing their pictures, etc.   Motherboard goes bad for whatever reason.  And their ability to quickly recover data is... sucky.  Like really sucky.  If it doesn't find the markers they are screwed.  They need to replicate hardware.

    Should this be in the realms of pure speculation?
    Who knows the final specifications of Vail and future MediaSmart servers (and the like)?


    A WHS v1 machine in the basement + Vail on an AMD Athlon 64 X2 4600+, 3 GB RAM, a mixed setup of harddrives in and outside the storage pool...
    Wednesday, August 18, 2010 9:55 PM
  •  

    Imagine home user Jane Smith.  They have an HP MediaSmart running Vail.  2 drives storing their pictures, etc.   Motherboard goes bad for whatever reason.  And their ability to quickly recover data is... sucky.  Like really sucky.  If it doesn't find the markers they are screwed.  They need to replicate hardware.

    Should this be in the realms of pure speculation?
    Who knows the final specifications of Vail and future MediaSmart servers (and the like)?


    A WHS v1 machine in the basement + Vail on an AMD Athlon 64 X2 4600+, 3 GB RAM, a mixed setup of harddrives in and outside the storage pool...

    as

    As there is no release candidate yet, of course, this is speculation based solely on the evaluation of the current build.  If this forum is in part a means to provide feedback to those working on it, then we should feel the need to point out our attempts with the current build, using hardware we provide and run through scenarios we have. 


    I have tried the ISO on 6 different variations of hardware and tried different situations to see how it holds up for stability/speed/performance/reliability/etc.  Which is the same that I have done with every MS product that I have ever Beta'd.   Now, I'm not a "beta" on this one, as I was with others, but I have to think ahead on how I would pitch this to clients.

     

    And right now, I'm not seeing the selling point or advantage over WHS1.  I'm just seeing disadvantages.  And we are late in the development game for that to be true.   At least in Aurora, you have Cloud based apps, functional domain controls, etc. that you can say "well, that's a potential big upside".  I'm not sure what the big upside is here.

     

    Thursday, August 19, 2010 1:27 AM
  • Al, in the real world you don't rely, as an end user, a developer, or an IT pro, on an undocumented feature as the foundation of your system. If you do, it's a safe bet that it will turn around and bite you on the ____ like an alligator
    I'm not on the WHS team, I just post a lot. :)
    Thursday, August 19, 2010 3:36 AM
    Moderator
  • If the HP MediaSmart Server EX49 series is "upgradeable" to Vail, I'm sure it will be handled the same way Hp has upgraded old MSS's to newer releases of their software: factory reset, wiping the entire server (all storage).

    The rest of what you say seems to be theory-crafting. There's already a robust data recovery mechanism in Vail: connect your drives to another Vail installation, and they appear as "Non Default Server Storage". From the desktop, this is mounted to a folder on your server's C: drive. That is Microsoft's plan for data recovery. Personally, I think Microsoft is smoking some fine crack there, thus the product suggestion I linked above.


    I'm not on the WHS team, I just post a lot. :)
    Thursday, August 19, 2010 3:39 AM
    Moderator
  • Has anyone done any actual tests of scenareos around a single drive failure, and would they like to share their findings?

    I can see alot of speculation and assumption on this thread, and not a lot of actual test resuls. If I had a rig spare, I wold do this myself, but sadly I don`t.

    If you guys want to make any meaningful progress on getting changes made to DE, I think thats the only way to ensure your feedback is taken on board by microsoft. Speculating won`t cut it.

     

     

     

     

    Thursday, August 19, 2010 8:11 AM
  • Has anyone done any actual tests of scenareos around a single drive failure, and would they like to share their findings?

    Yes. I've tested, and others have as well. Some tests have suggested that data loss for large files (larger than the DE block) if a single drive fails and duplication is off will be approximately the same as with DE V1. Some have suggested that it will be significantly worse.
    I'm not on the WHS team, I just post a lot. :)
    Thursday, August 19, 2010 9:40 AM
    Moderator
  • Thanks Ken,

     

    Is there any technical documentation available which describes exactly how (and when) blocks are allocated across drives for large files?

    Surely as soon as people understand how DE will allocate blocks in any given storage scenareo, then mitigating risks becomes easier (and dispelling the illusions people have becomes even easier).

    It may well be that there are more data storage risks than in v1 (for large movie collections) but I for one would like to know *exactly* what those risks are, and exacly when they are likely to occurr.

     

     

     

    Thursday, August 19, 2010 11:50 AM
  • The Diskeeper blog contains the best technical information available so far. And there's not a lot there. What is the risk? Any file larger than 1 GB will, in the Vail implementation, be split into multiple data blocks. Those blocks may reside on different disks. if any portion of a file is lost (through disk failure) the whole file is lost. People who have invested significant time and personal effort in accumulating extremely large data sets (which are, as I've observed before, almost entirely DVD and/or BD rips) are understandably concerned, particularly if they've made use of undocumented features (UnRAID to protect physical disks without 100% data duplication, for example).

    Bottom line for the topic of large (multi-terabyte) data sets: Costs will be higher for people with very large data sets if they use Vail and require protection from the loss of a drive. To hold 1 TB of data, you will need (in round numbers) 2.3 TB of storage, assuming you use duplication. Options for these people include:

    • Use a different tool on a different OS.
    • Use RAID, with a good hardware accelerated RAID HBA that supports OCE.

    There are other possibilities, of course. Revamping Drive Extender to suit them is not going to be one of them; I suspect that Microsoft would wind up killing Windows Home Server entirely if that became necessary. Any alternative that doesn't use plain Windows Home Server on plain hardware is going to cost significantly more, and frankly that's what I think the real complaint is; People want to build enterprise data storage systems, but don't want to pay enterprise prices. Possibly there's also a WAF at play: "You spent how much on this? And now you have to spend even more?! Are you mental?!?!" (I've had this conversation. It's not much fun, and you can't win no matter what you say. :) )

    The other thing that people with these extremely large data sets need to consider is force majeure event protection: fire, flood, theft, storm, earthquake, etc. How exactly does one back up 20 TB of data today? Again, the enterprise solution is very pricey, and there is no reasonable consumer solution at all. Nor does there need to be; the number of consumers today with that amount of data is absolutely miniscule.


    I'm not on the WHS team, I just post a lot. :)
    Thursday, August 19, 2010 1:32 PM
    Moderator
  • I get the distinct impression that it is not yet known when a file consisting of multiple 1gb blocks will be written to mutliple disks, and why .

    I would also put my neck out and say thats its pointless discussing if DE v2 is doing a bad/worse job of dealing with these storage situations than v1 until we know the definitive answer to that question.


     

     

     

    Thursday, August 19, 2010 1:40 PM
  • I can't give you the sort of specific information you're looking for, I'm afraid. Even casual testing will show you, though, that if you put a large file in a Vail server with 2+ drives it doesn't always get split among multiple drives (which would result in the loss of the file when you removed the drive physically, then removed it from server storage using the dashboard). Elsewhere in these forums, Microsoft has said that there will be some sort of balancing mechanism which will try to do initial allocations so all drives are sort of equal in terms of "fullness". That practically guarantees that this situation will occur without duplication.

    But no, you don't need to stick your neck out. There's no doubt, and no argument from me, on the subject: very large files are more at risk in Vail without duplication because they may get split between two or more drives. It doesn't have to be a common occurence, just the possibility makes increased risk obvious to me.


    I'm not on the WHS team, I just post a lot. :)
    Thursday, August 19, 2010 1:56 PM
    Moderator
  • One more thing: A "deal breaker" is something whose presence or absence will result in you choosing another product with no room for flexibility ("I need this feature to use your product." rather than "I need most of this list of features."). The people who have referred to the DE V2 block storage allocation as a "deal breaker" have polarized this whole discussion far more than anything I've said (go back and read the whole thread, and I think you'll probably agree). If block allocation of data storage as implemented in Vail is truly a deal breaker, it's time for these people to start analysing alternatives because I don't think it's going to change.


    I'm not on the WHS team, I just post a lot. :)
    Thursday, August 19, 2010 2:21 PM
    Moderator
  • I`d like to see the concept of a balancing mechanism in DE v2 challenged. Its obvious what risks such an approach creates, but its less clear to me what the benefits are. Drives don`t need to be "equally full". Is the intention that this approach improves performance?

    It almost sounds like DE v2 will come with a "block fragmenter" when a "block defragmenter" would appear to be a safer way to store blocks - assuming you don`t know how many blocks any one file will occupy.

     

     

    Thursday, August 19, 2010 2:30 PM
  • You're still talking about undocumented functionality.

    In V1, Microsoft's answer to your scenario will be a server recovery, not moving drives around to copy files from a hidden location. That's a functional and flexible workaround that avoids server recovery, which can be problematic on home-built servers due to configuration issues. Instead of recovery you progress directly to a fresh installation...

    It is perfectly documented. Please stop misinforming. Quote from WHS v1 Drive Extender technical brief:

    "If the home server fails completely, all the surviving drives can be attached to a computer that is not even running Windows Home Server Drive Extender, and you can copy the files from the drives to that computer".

    http://www.microsoft.com/downloads/details.aspx?FamilyID=40c6c9cc-b85f-45fe-8c5c-f103c894a5e2&DisplayLang=en

    Thursday, August 19, 2010 5:22 PM
  • One more thing: A "deal breaker" is something whose presence or absence will result in you choosing another product with no room for flexibility ("I need this feature to use your product." rather than "I need most of this list of features."). The people who have referred to the DE V2 block storage allocation as a "deal breaker" have polarized this whole discussion far more than anything I've said (go back and read the whole thread, and I think you'll probably agree). If block allocation of data storage as implemented in Vail is truly a deal breaker, it's time for these people to start analysing alternatives because I don't think it's going to change.


    I'm not on the WHS team, I just post a lot. :)

    Yes, it is.  Admittedly, this is why many of us will likely ditch WHS and recommend our customers to look at other products, or more likely, why we will stock up on WHSV1 OEM-10packs.

     

    For us, yes, the way it is being written is a deal breaker.  It is unlikely to change.  Sometimes it goes that way.  I was a beta tester for Windows 95, a Beta Tester on Office For Windows For Workgroups, and all the way up.   I've followed the MSDNs and the development trees for years.  Sometimes, the truck leaves and Microsoft is just wrong.  It happens.  Mobile Phone6.  Millenium.... most recently KIN, etc. 

    It happens to the best of us.  It doesn't mean that MS always gets it wrong - the first beta I received of NT 3.5 WS was one of those that made me go "wow", and the quick response to a near free upgrade of NT3.51 which was thrown out to us was great stuff.    Even MS DialerNet and ClassServer may have been wrong technologies, but when I was there in Seattle and walked to the technology presentations on both of them I could see where they were going and I appreciated it.  There was a real understanding of the market they were trying to reach and what they were trying to sell.

     

    That's what's so befuddling here.  This is a miss of what the market wants of epic purportions.  "No one has that much files"  "data security *eh*" is a horrible response.  If you're rolling out release candidates (the refresh labels on install RC0) then something is significantly wrong. 


    I recognize this is the direction Microsoft intends to go.  That however, should not in any way mean that those who are eagerly testing the betas are not saying "this is the wrong way to go".  I'm also playing with Aurora, which uses the same base system.  But Aurora overcomes the hitches by offering functionality that works for small business, like cloud based apps, etc.  so there is a stick and a carrot. 


    But in comparison to WHS1, this is all stick.   I find the concept of "it's undocumented" in relation to storing large data completely ridiculous when Microsoft handed out to all of us at TechConnect meetings fliers advertising WHS as a perfect storage for Windows Media Center.   Microsoft is out there selling HD-recording cameras themselves, which generate video at 10.2GB/hour..

    So, the idea of "large files, who uses that?!?! is preposterous.  They say this at the same time that their own ISO for this is 4GB, which would mean 4 segments based on their system, which means a single drive failure and *poof*.

    ____, even for those who own one of the ill-fated and now-dead KIN, you'll find that the KIN's data storage method generates about a 4GB DataStore right out of the gate for those who use the zunemarketplace function. 

    So, yes, maybe it is flame.  But users who are expected to buy and sell this should be able to go on the record.   While we say "well, this is hypothetical"  I've done 6 Vail installs on varying hardware to test.  Because I've got time and clients who will support it.  And I'm struggling to think of why I'd recommend it.

    And it is not as though MS has never listened to people saying "this idea is just wrong".  See: Microsoft OneCare for SBS.  Enough people came out into their forums and basically said: this is a very bad idea, here are the significant problems.. and where is it now?  Basically dead.

    That's the thing.  Many of us are consumers and we are also those who are expected to sell this product.  So, when something seems absolutely contrary to why we'd ever recommend it, then I'm not sure what to think.

    A person who uses WHS as a store for music and pictures and video from the Windows MCE will find this as a matter of protection is worthless.. ____, even an Iomega NAS box, which is trash, does a better job of assuring against data loss.

    I've used six machines, a few VMs.  I've tried repeated installs and tests and failures.  And out of a set of tries every day, I have yet to see Vail recover from losing A SINGLE DRIVE.  In a 2 or 3 drive setup, even.  The moment you lose a single drive, everything everywhere.. outside of very small files.. is lost.  Even MP3s, which are not "chunked" can't be reached because "this Volume cannot be read".

    So, what do I make of this?  I make of it that unless my results - which are using both major OEM hardware as well as home hardware - are either a complete fluke and it just hates the test data I'm throwing at it (dental x-rays) or there is something very wrong still.

    If you design a system that bets it's life on the end user doing their own backups and that there won't be drive or hardware failure, you are setting up an end user for a heck of a lot of grief later.

    Thursday, August 19, 2010 6:38 PM
  • I`m not in a position to contradict your experiences, as I`ve done exactly zero testing, but others appear to have had much better luck.

    See this quote from:: http://forum.wegotserved.com/index.php/topic/14782-allow-disabling-of-file-chunkingstriping-for-non-duplicated-folders/

    "After dealing with a "serious" drive failure that appeared to be more of beta issues than HDD, I was able to recover 99% of my files. That's approx 3TBs worth of data. Most of that videos. It wasn't pretty, but it looks like it worked just fine."

    Thursday, August 19, 2010 7:17 PM

  • I`m not in a position to contradict your experiences, as I`ve done exactly zero testing, but others appear to have had much better luck.

    See this quote from:: http://forum.wegotserved.com/index.php/topic/14782-allow-disabling-of-file-chunkingstriping-for-non-duplicated-folders/

    "After dealing with a "serious" drive failure that appeared to be more of beta issues than HDD, I was able to recover 99% of my files. That's approx 3TBs worth of data. Most of that videos. It wasn't pretty, but it looks like it worked just fine."

    adsf

     

    That link doesn't work for me.  But the links that do are all people requesting basically what we are:

    http://forum.wegotserved.com/index.php/topic/14691-allow-disabling-of-file-chunkingstriping-for-non-duplicated-folders/

     

    There is a method by which you can recover, and I've followed the way they have done it on WeGotServed.but if you check, most of the threads talk about people getting the exact same results as I have without the hack.  And, we can't be alone since MS warns about the drivecheck in the logs.. and you can't replace from failure a drive without it auto-starting.


    So, MS clearly knows about it, they put it right in the release notes.

    Thursday, August 19, 2010 10:31 PM
  • I've read this whole thread and come to these conclusions -

    1. The DE changes are here to stay. There is no point in hoping that 'MS will add a better way' in the future.

    2. Cold hard facts - enable duplication, use 66% overhead for all data, or you WILL lose everything (disk failure is inevitable).

    3. There were a number of technical issues in WHS v1 caused by the old DE (which lets face it is an ugly hack on top of NTFS which is a fs never designed for this). These issues may or may not have affected users but clearly the team felt they had to be fixed.

    3. WHS team decided to favor a better 'technical' rather than a 'user' solution.

    4. I don't think the H belongs in WHS anymore. You can sell it to home users but the product is no simpler to use than a competing NAS/Drobo. The attraction of WHS for many was aggregating a bunch of disks and building their own on a platform which was simple yet powerful. The simplicity is gone.

    5. I don't trust drastic filesystem level changes. And WHS has an infamous track record (the data corruption bug). Vail is taking longer to get out and I just don't see myself entrusting all my data to it when the price of failure is total catastrophe.

    WHS has 2 audiences -

    People who buy an HP Mediasmart to backup their laptops nightly and share their pictures and photos don't care what the internals are. I know many who don't even bother adding > 2 disks, for them it really is a 'home' server without any server issues.

    Then there are those with muti TB data, building their own WHS boxes, and who are worried about the implications. For these, the ONLY reason to choose Vail over say Raid6/ZFS is it allows mixing arbitrary size disks. For this you pay a heavy cost in data size compared to other systems.

    If you are going to do chunking and striping, not using parity gives no benefits at all. Basically you get the worst of both worlds - 100% extra space for duplication (+ more for CRC) as well as data not being in native format on a physical disk. 

     

     

    Friday, August 20, 2010 2:17 AM
  • 2. Cold hard facts - enable duplication, use 66% overhead for all data, or you WILL lose everything (disk failure is inevitable).

    Im sorry, but I still don`t buy this as a statement of fact.

    Its far from clear how DE v2 exactly works in this area. I have seen posts saying that DE v2 will activley spread chunks across drives (for reasons which are a complete mystery), and I`ve also seen posts suggesting DEv2 will attempt to keep chunks from the same share together on the same drive where possible. Depending on which is true (or perhaps something else is true), DE v2 is either slightly less reliable than De v1 or ALOT less reliable than De v1, depending on the number of drives in the pool (for unduplicated folders, of course) Some may think that the difference doesn`t matter, but I think it matters a great deal.

    Until this is cleared up once and for all with some proper, official tehncial documentation from the drive extender team, the fact of the matter is, that we simply don`t know the final outcome.

    This may come across as pendantic (and yes I know lots of people hate the new block based system, its been disussed at length) but how can people make assumptions about the actual level data security DE v2 will offer, when they don`t actually know how the system is supposed to work?

    I know people have tested this. They appear to come up with varing results (from what I`ve seen: lots being bad, some being good) . Not knowing how the system is supposed to work means we have no basis for comparison of actual results vs expected results.

    I would never test code at work without having access to the specs, because otherwide, all youre doing in fumbling in the dark, and making assumptions. That adds no value.

     

     

    Friday, August 20, 2010 8:56 AM
  • ...
    I've used six machines, a few VMs.  I've tried repeated installs and tests and failures.  And out of a set of tries every day, I have yet to see Vail recover from losing A SINGLE DRIVE.  In a 2 or 3 drive setup, even.  The moment you lose a single drive, everything everywhere.. outside of very small files.. is lost.  Even MP3s, which are not "chunked" can't be reached because "this Volume cannot be read".
    ...


    This sounds like you're running afoul of a known issue in the current build. See my reply to Jonas' post for more on that issue. I'm trying to put together a consistent repro case; can you drop me an email (address in my profile here, but if you don't remove the anti-spam measure your ISP will probably get an automated complaint) so I can understand the failures you've seen? Also, have you bugged them?

    But what should happen is nothing. You don't lose anything that was in a duplicated share. You don't even lose access to any files when a drive fails, as long as that file is duplicated. If it's not duplicated, the errors you describe sound reasonable. When you remove the "missing" drive, DE should re-duplicate any files that were duplicated, and should remove any files that weren't and were completely or partially on the failed drive.


    I'm not on the WHS team, I just post a lot. :)
    Friday, August 20, 2010 12:16 PM
    Moderator
  • Micksh,

    Thank you, I forgot about that paragraph. You are correct, Microsoft has provided information about DE V1 internals in that "white paper". However, that's not a guarantee that DE internals will never change.


    I'm not on the WHS team, I just post a lot. :)
    Friday, August 20, 2010 12:25 PM
    Moderator
  • ...
    2. Cold hard facts - enable duplication, use 66% overhead for all data, or you WILL lose everything (disk failure is inevitable).
    ...

    56%, actually. The 12% for ECC is "off the top" of every disk, and reduces overall disk capacity. As for losing everything, that's not my experience, even in the current build (which has known issues in the area of server storage and drive removal). What is your test case?

    I disagree when you say Vail won't be simple for an end user (who will generally buy it preinstalled from an OEM either online or via e.g. Best Buy). End users will only see the dashboard. It has, however, become more complex for an enthusiast.

    And as for how storage is used, allocated, etc. Microsoft's goal is to allow a user to just buy a new drive when storage needs to be expanded, with little thought to drive size and no requirement to match other drives already installed. Vail will deliver that, just as V1 did. The trade-off for simple capacity expansion with good data protection without a high-end RAID HBA is that it's done through mirroring plus ECC, which has higher storage costs in the long run.


    I'm not on the WHS team, I just post a lot. :)
    Friday, August 20, 2010 12:36 PM
    Moderator
  • 2. Cold hard facts - enable duplication, use 66% overhead for all data, or you WILL lose everything (disk failure is inevitable).

    Im sorry, but I still don`t buy this as a statement of fact.

    Its far from clear how DE v2 exactly works in this area. I have seen posts saying that DE v2 will activley spread chunks across drives (for reasons which are a complete mystery), and I`ve also seen posts suggesting DEv2 will attempt to keep chunks from the same share together on the same drive where possible. Depending on which is true (or perhaps something else is true), DE v2 is either slightly less reliable than De v1 or ALOT less reliable than De v1, depending on the number of drives in the pool (for unduplicated folders, of course) Some may think that the difference doesn`t matter, but I think it matters a great deal.

    The actual implementation of DE v2 is not important. Its clear that it WILL chunk files > 1GB and it may or may not chunk smaller size files. This means there is no guarantee or more importantly, configurable settings that control how your data is spread. It really doesn't matter if the likelihood of a file being split across disks is 99% or 0.01%, both are equally unsafe as far as data loss is concerned. Like you said yourself, 'less reliable' or 'lot less reliable'. WHS v1 is 100% reliable for any data not on the failed disk.

    The point is in Vail you cannot trust your data without duplication. This is the same problem shared by all striped storage solutions and is really not up for debate.

     

    Saturday, August 21, 2010 12:34 AM
  • ...
    2. Cold hard facts - enable duplication, use 66% overhead for all data, or you WILL lose everything (disk failure is inevitable).
    ...

    56%, actually. The 12% for ECC is "off the top" of every disk, and reduces overall disk capacity. As for losing everything, that's not my experience, even in the current build (which has known issues in the area of server storage and drive removal). What is your test case?

    I disagree when you say Vail won't be simple for an end user (who will generally buy it preinstalled from an OEM either online or via e.g. Best Buy). End users will only see the dashboard. It has, however, become more complex for an enthusiast.

    And as for how storage is used, allocated, etc. Microsoft's goal is to allow a user to just buy a new drive when storage needs to be expanded, with little thought to drive size and no requirement to match other drives already installed. Vail will deliver that, just as V1 did. The trade-off for simple capacity expansion with good data protection without a high-end RAID HBA is that it's done through mirroring plus ECC, which has higher storage costs in the long run.


    I'm not on the WHS team, I just post a lot. :)

    I think we are both saying the same thing. The user who buys off the shelf doesn't care about the internals. For people with a lot of data, the storage size tradeoff is now a big concern (since duplication is almost mandatory, isn't it on by default too?). e.g. I have 5TB of data out of which I only duplicate <1TB. The rest I backup offline if needed. I've had disks die in WHS, I simply replace the bad disk and lose at most 1TB availability, which I then rip back from source or copy from external backups.

    To move to Vail I'd need ~11TB which I simply can't afford.

    Is capacity expansion through any size disks the holy grail? That (and pc backups) are the only advantage Vail has over Raid. At the very least, DE could expose policies about which data sets are to be  kept on single disk and which ones are to be striped (on a share basis). This sort of thing can be done without making increasing product complexity because most people won't ever go into advanced settings.

     

    Saturday, August 21, 2010 12:52 AM
  • I understand the trade-off for data security is a pain point. I've said that enough that I probably type it in my sleep. :) I just don't think you're going to be able to get away from it easily. As I'm pragmatic, I'd rather worry at something that I can change. :)

    One note: Vail should deal fairly gracefully with RAID. It's probably not going to be a "supported" scenario, but if you buy a RAID HBA that supports OCE, you can turn duplication off, install new disks when needed, and roll out a new "volume" on your array. (I don't know what will happen if you just grow the existing volumes.) In the short term, it's expensive; you'll pay as much for the HBA as you will for 2 or 3 2 TB drives. But in the long run, probably cheaper for really large data sets. And you're still protected against the loss of a single disk. Unfortunately you'll still have the ECC overhead.

    Best of luck backing up any 10 TB server, though. :)


    I'm not on the WHS team, I just post a lot. :)
    Saturday, August 21, 2010 2:32 AM
    Moderator
  • The actual implementation of DE v2 is not important. Its clear that it WILL chunk files > 1GB and it may or may not chunk smaller size files. This means there is no guarantee or more importantly, configurable settings that control how your data is spread. It really doesn't matter if the likelihood of a file being split across disks is 99% or 0.01%, both are equally unsafe as far as data loss is concerned. Like you said yourself, 'less reliable' or 'lot less reliable'. WHS v1 is 100% reliable for any data not on the failed disk.

    The point is in Vail you cannot trust your data without duplication. This is the same problem shared by all striped storage solutions and is really not up for debate.

     

    I have to disagree.. Its VERY important. How can you even talk about how your data is spread when you don`t understand the intended design? Bottom line: We don`t know. But I do appreciate whatever the documentation says in the end, you will be less happy than you were with V1. Fair enough.

    And I totally agree that unduplicated data cannot be trusted. Even less so with DEv2 as both of us have pointed out. Thats why you have backups (or orignal media to re-rip). If you want to keep your data, it has to be backed up. Clearly having to resort to this approach makes DE v2 unacceptable to some people. But I still don`t think those people are the ones that MS are pitching WHS at. Hence why we are where we are. We`ll never agree.

    Its a reasonably safe bet that DEv2 will make it to RTM relativley unchanged from its current design. And yes, that leads to more data loss on unduplicated folders. But data loss on unduplicated folders was always a possibility. So by your own rationale it makes no difference. "Its equally unsafe as far as data loss is concerned"

    I know its harsh, but just like the unpopular decision on MCE integration being left out, it looks like we will have to deal with it, or go buy another product. In some ways its the exact same thing playing out - a bunch of enthusiasts are being left out in the cold because the product is going in another direction. As a keen MCE user, I hated that decision (still do) but I do understand why it was taken.

    It appears highly likely that Microsoft are not going to change the design in the fundamental way you and many others want at this (very) late stage of the game. RC1 isn`t that far away. I recon RTM by christmas. Theres not chance in ____ you can make such a fundamental change as this at this stage of the project.

    I for one am glad of that, becuase I want the benefits De V2 will bring. Clearly were never going to agree on that, and I appreciate why.

    Having said all that, I would hope that microsoft feel very much obligated to build a tool to read vail drives on non vail computers AND take a long hard look at how DE V2 will allocate blocks across drives.

    Saturday, August 21, 2010 7:39 AM
  • Who is the target market for Vail and Aurora?

    - Home users who buy an appliance for their photos/music won't notice any difference. But they can't expand their server as storage needs grow. They probably care more about the scheduled backups.

    - Enthusiast community is being abandoned. WHS v1 was the recommended choice for even 50TB servers because of its feature set.

    - Business users don't care about the 'mix different sized drives'.

    I agree that data needs to be backed up. Data loss on ALL unduplicated shares is impossible in WHS v1, and that is very likely in Vai, as a result of the striping. WHS/Vail/Raid are just high availability solutions with additional benefits like unifying the disk pool and growing it. But Vail combines the worst of WHS and RAID - total data loss if unduplicated, and horrible efficiency. 

    So the way I see it -

    - if you want to mix/match drives, stick with v1

    - if you want realtime data safety and duplication, use RAID.

    Why didn't MS choose to implement striping with parity checks? That's a much better design with no tradeoffs. Vail offers none of the read advantages of RAID. With the tech in Drive extender, it would have been possible to implement Raid 5/6 and work with drives of different sizes.

    I'm curious, what benefits (in data storage) does Vail bring over WHS1? I know plenty of people running WHS, I have a 6+TB server in daily use and I've had to replace bad disks and restore from backup, but I haven't ever had catastrophic data loss, and the performance is fine.

    What is the business case for Vail? Why as a user should I even think of upgrading? I see no possible benefits.


    Thursday, August 26, 2010 8:25 PM
  • Btw, I know the things Vail enables such as backing up the OS drive, shadow copies etc due to a much more robust file system. But none of this affects the main purpose which is data storage.
    Thursday, August 26, 2010 8:35 PM
  • ...
    - Enthusiast community is being abandoned. WHS v1 was the recommended choice for even 50TB servers because of its feature set.
    ...

    I think mostly recommended because it's Windows Server 2003 (a robust and mature server OS) under the hood, so enthusiasts saw themselves as buying Windows Server 2003 "Lite" with 10 CALs for cheap. And design decisions made (probably) in the interests of getting V1 out the door resulted in the appearance of the "logical unit" of data protection being the file, because files are exposed though that accident of design. Since the level at which you make data protection decisions is the share, the real "logical unit" is the share. This is true in both V1 and Vail, it's just more obvious in Vail.

    As for 50 TB servers, they're a disaster waiting to happen. They aren't backed up; there's no possible disaster recovery plan for them that an average homeowner can come close to affording. Back up to 25x 2TB disks? C'mon, pull the other one! That's $2800 for bare disks and a week or more to back everything up. LTO tape is worse for price, and about the same for time... And the people with this kind of data just keep piling more on.

     And where do these 50 TB collections of data come from? I own more DVDs and BDs than most people I know (and more CDs than anyone I know personally), and if I ripped every single disk I own today it would only be 5-6 TB of data. I have thousands of CDs, several hundred DVDs, and dozens of BluRay disks. I've ripped a fair fraction (20% or so) of the CDs to Apple Lossless and put it on my server, but I don't want to rip the DVDs and BDs. We have a rider on our homeowner's insurance that explicitly covers our media library instead (do the math, you'll understand why). If the house burns down, I know what I own (I use cataloging software that resides on my server, and the database goes off-site regularly with the rest of the data in the shares) so the media library is replaceable.


    I'm not on the WHS team, I just post a lot. :)
    Friday, August 27, 2010 1:29 PM
    Moderator
  • HD recorded TV - digital cable tuner and cable card. 2 hour movie can be 12
    gigs. People that want libraries, like me, of movies and series recordings can
    easily accumulate this amount of data.
     
    I've been saving recorded shows in HD since it first became possible in Vista.
    I've got an extensive library.
     
    And I also have a collection of mkv files.
     
     
     
    On Fri, 27 Aug 2010 13:29:08 +0000, Ken Warren [MVP] wrote:
     
    >And where do these 50 TB collections of data come from?
     

    Barb Bowman

    http://www.digitalmediaphile.com

    Friday, August 27, 2010 1:58 PM
  • ...
    I've been saving recorded shows in HD since it first became possible in Vista.
    I've got an extensive library.
    ...
    Serious question: How do you protect it? Feel free to email me (address in my profile, just remove the anti-spam device); it's kinda off-topic for this thread. :)

    I'm not on the WHS team, I just post a lot. :)
    Friday, August 27, 2010 2:42 PM
    Moderator
  • I use the NNTP bridge, so don't see an email address or your profile, but to
    quickly answer your question, I don't exactly have a good back up strategy for
    this library. I can't afford to duplicate the library
     
    On Fri, 27 Aug 2010 14:42:01 +0000, Ken Warren [MVP] wrote:
     
    >Serious question: How do you protect it? Feel free to email me (address in my profile, just remove the anti-spam device); it's kinda off-topic for this thread. :)
     

    Barb Bowman

    http://www.digitalmediaphile.com

    Friday, August 27, 2010 3:09 PM

  • What is the business case for Vail? Why as a user should I even think of upgrading? I see no possible benefits.



    Unless the DE team are living in a cave, I think they have heard the message on striping loud and clear ;-)

    You don`t see any benefits to vail because it appears that you`ve already decided that striping (in any form) makes everything else about the product unattractive. Not everone has that narrow view.

    Has anyone re-tested the single drive loss scaenareo on the new build?

     

     

     

     

     

     

     

     

    Friday, August 27, 2010 4:44 PM
  • I have tested this build with 2 disks and a mix of data.

    With duplication disabled, I copied ~150GB of mixed data to the machine.
    I shutdown and unplugged the second disk. Removing the 'missing' drive resulted in 95% data loss. I presume the loss wouldn't have been so large if it had been spread over a larger pool of disks.

    At first I was very concerned about this, but the more I thought about it, the less it seemed to matter...

    Either you have enough storage to duplicate and/or backup what you care about or you're already prepared to lose it. This is no different than keeping it all on one disk.

    -timotl

     

    Friday, August 27, 2010 5:08 PM
  • "Either you have enough storage to duplicate and/or backup what you care about or you're
    already prepared to lose it. This is no different than keeping it all on one disk."
     
    So.. what then is the real value of the backup system in Vail? I would feel a lot more
    comfortable with all valuable data duplicated (as in full complete files saved twice) on
    two (or whatever number it takes to duplicate them all) separate external drives than to
    trust it to some backup method that chunks the large files over several drives which even
    with duplication results in the files spread over several drives (without any external
    method of retrieval), making it near impossible to fully retrieve individual large files
    in the event of even one drive failure. It sounds totally insane to me.
     
    Aside from backed up (saved) shares data, in the event of even one drive failure, can I
    even restore any of the client system backups successfully, or are they also chunked over
    several drives?
     
    Art [artfudd] Folden
    I'm a PC and Windows 7 sucks less!
    ----------------------------------------------
    "timotl" wrote in message
    news:eb9bebdb-f8ef-496a-919e-ec264d89ed70@communitybridge.codeplex.com...
     
    I have tested this build with 2 disks and a mix of data.
     
    With duplication disabled, I copied ~150GB of mixed data to the machine.
    I shutdown and unplugged the second disk. Removing the 'missing' drive resulted in 95%
    data loss. I presume the loss wouldn't have been so large if it had been spread over a
    larger pool of disks.
     
    At first I was very concerned about this, but the more I thought about it, the less it
    seemed to matter...
     
    Either you have enough storage to duplicate and/or backup what you care about or you're
    already prepared to lose it. This is no different than keeping it all on one disk.
     
    -timotl
     
     
    Friday, August 27, 2010 6:11 PM

  • ......than to trust it to some backup method that chunks the large files over several drives which even with duplication results in the files spread over several drives (without any external method of retrieval), making it near impossible to fully retrieve individual large files in the event of even one drive failure. It sounds totally insane to me.

    You appear to be saying that, on a multi drive pool, large files which are duplicated are "near impossible" to retreive from a single drive failure. Have you actually bothered to test this and record the results - or are you just making assumptions?

    If you have done this test, then you need to post a bug on connect.

     

    Friday, August 27, 2010 7:27 PM


  • ...
    - Enthusiast community is being abandoned. WHS v1 was the recommended choice for even 50TB servers because of its feature set.
    ...

    I think mostly recommended because it's Windows Server 2003 (a robust and mature server OS) under the hood, so enthusiasts saw themselves as buying Windows Server 2003 "Lite" with 10 CALs for cheap. And design decisions made (probably) in the interests of getting V1 out the door resulted in the appearance of the "logical unit" of data protection being the file, because files are exposed though that accident of design. Since the level at which you make data protection decisions is the share, the real "logical unit" is the share. This is true in both V1 and Vail, it's just more obvious in Vail.

    As for 50 TB servers, they're a disaster waiting to happen. They aren't backed up; there's no possible disaster recovery plan for them that an average homeowner can come close to affording. Back up to 25x 2TB disks? C'mon, pull the other one! That's $2800 for bare disks and a week or more to back everything up. LTO tape is worse for price, and about the same for time... And the people with this kind of data just keep piling more on.

     And where do these 50 TB collections of data come from? I own more DVDs and BDs than most people I know (and more CDs than anyone I know personally), and if I ripped every single disk I own today it would only be 5-6 TB of data. I have thousands of CDs, several hundred DVDs, and dozens of BluRay disks. I've ripped a fair fraction (20% or so) of the CDs to Apple Lossless and put it on my server, but I don't want to rip the DVDs and BDs. We have a rider on our homeowner's insurance that explicitly covers our media library instead (do the math, you'll understand why). If the house burns down, I know what I own (I use cataloging software that resides on my server, and the database goes off-site regularly with the rest of the data in the shares) so the media library is replaceable.


    I'm not on the WHS team, I just post a lot. :)


    People who build 50TB servers build for capacity and future expansion, its not necessarily full. And some of them have hundreds of blu-rays :)

    You're wrong about why WHS is used. Its recommended because its the best solution possible for large data sets, with very good failure handling and dynamic expansion. None of that has to do with it being based on Server 2k3 or CALs. Most people who use WHS don't even know its Server 2003 and why should they? They don't use any server features. And if they did then they'd want Server 2008 even more.

    There absolutely is a disaster recovery plan, its re-ripping from source. If I have a hundred movies stored on my server, a single disk failure in v1 can be fixed by ripping movies on that disk. In Vail my entire collection would have to be re-ripped, a process that takes weeks. I can understand you not wanting to rip your discs, but for many that's the primary reason for a home server (and not the pc backups).

    Its clear there is a big disconnect between the way the WHS team (and you) think of the product and the way its being used. Not by a majority, but the people who do are the most vocal supporters of WHS and evangelize it. So its ok for you to say that Vail is not meant for us, WHS team can say that according to their surveys most people don't have that much data, but that's alienating and abandoning your biggest support group.

    WHS is invisible. No one sees it in stores, there are no ads for it. If not for word of mouth and forum posts (through people like us) do you think it would sell as much?

    Monday, August 30, 2010 8:12 PM
  • Its clear there is a big disconnect between the way the WHS team (and you) think of the product and the way its being used. Not by a majority, but the people who do are the most vocal supporters of WHS and evangelize it. So its ok for you to say that Vail is not meant for us, WHS team can say that according to their surveys most people don't have that much data, but that's alienating and abandoning your biggest support group.

    Microsoft gathers a large amount of telemetry on WHS, I think they have a pretty good picture of typical usage patterns.

    Youre taking a commercial decision and describing it as an emotional one. It isn`t.

     

    Monday, August 30, 2010 10:15 PM
  • Its clear there is a big disconnect between the way the WHS team (and you) think of the product and the way its being used. Not by a majority, but the people who do are the most vocal supporters of WHS and evangelize it. So its ok for you to say that Vail is not meant for us, WHS team can say that according to their surveys most people don't have that much data, but that's alienating and abandoning your biggest support group.

    Microsoft gathers a large amount of telemetry on WHS, I think they have a pretty good picture of typical usage patterns.

    Youre taking a commercial decision and describing it as an emotional one. It isn`t.

     

     I don't doubt their usage figures and I never said it was an emotional decision. If they can't address the concerns raised, they should identify the market segment Vail is aimed at (i.e. not power users) and clearly state that its not advisable or recommended outside said 'typical' usage patterns. Claiming that this is an 'upgrade' to WHS v1 can be seen as intellectual dishonesty when its a different product for all practical purposes.

    I don't mean to come across so critical but the WHS team does a really poor job at communication. Why for instance do they have 2 blogs (http://blogs.technet.com/b/homeserver/, http://windowsteamblog.com/windows/b/windowshomeserver/) ?

    Monday, August 30, 2010 10:35 PM
  •  If they can't address the concerns raised, they should identify the market segment Vail is aimed at (i.e. not power users) and clearly state that its not advisable or recommended outside said 'typical' usage patterns. Claiming that this is an 'upgrade' to WHS v1 can be seen as intellectual dishonesty when its a different product for all practical purposes.

    In reality, I think that power users will be expected to read the DE white paper (when it comes out) and take their decision based on that. Big stickers saying the equivalent of "Dont buy this" may not be popular with the Marketing Department ;-)

    Is it a different type of product? Really? For you, then of course yes it is, but for the majority (and its the majority which will sustain WHS going forward) the answer is certainly no.

    I don`t think DE is set in stone, yet - at least I hope its not. They certainly need to address how blocks are written across drives. If they go to RTM with DE Spreading chunks evenly across multiple drives, then they are quite mad, and I`ll be joining in with the vitriol.


    Tuesday, August 31, 2010 8:24 PM
  • Wow, I just finished reading this thread...Talk about losing the wind in my sails towards the Vail release...

    I currently have 26TB of data with about 20TB of it being non-duplicated data. I have personally recommended WHS to several others that purchased it and they now have similar setups.  In fact, I can honestly say that I don't know anyone that ISN'T using WHS as a backend to a significant media collection. If MS and Ken Warren think that these users are the 'rare exception' than its obvious that they have no clue about the current WHS market.

    I can't believe that the Vail development team is for a minute thinking about alienating some of their biggest supporters. Prior to WHS V1, it was pretty much impossible for movie enthusiasts to have their entire movie collections at their fingertips especially from multiple computers. Adding additional storage space was also pretty much a nightmare and not something to be approached casually. WHS V1 came along and basically solved these problems in one glorious swoop. Now because of a couple of obscure scenarios like copying ginormous 500GB files and handling of locked files, it appears that they are about to throw the baby out with the bath water.

    Both of these scenarios could be handled in less drastic ways than replacing the entire underlying data structure. For really huge files they could simply offer to archive and split it into multiple pieces during the copy or simply set an upper file size limit of say 60GB (larger than the largest dual-layer blu-ray). For open files you could just tell the OS to 'ignore' the files on a case-by-case basis or allow the user to create a special share that can better deal with open files. I also like the suggestion that someone made of being able to set an upper file/folder size limit for duplication. 

    The next successful version of WHS will be the one that caters to ALL segments of its user base:

    1. It should be configurable and not try to be a one-size-fits-all product that is targeted to the lowest common denominator. 
    2. It should be smart enough to give you the OPTION to keep all parts of file/folder on a single disk if you want to, without needing duplication turned on. 
    3. If/when you lose a disk, it should be smart enough to generate a report of exactly what you lost, without needing to wait for drive extender to generate a 20-item subset every few hours. 
    4. It should give you the option to optimize your storage for large files/folders  by allowing you to 'fill-up' disks instead of constantly trying to balance the data across all of the disks in your pool.
    5. It should be smart enough to put being a 'server' as a priority and allow you to restrict when housekeeping tasks (like demigrator) are allowed to run

    While WHS V1 wasn't perfect, it worked really well most of the time, all it really needed was some tinkering and a bit of polish. It hardly needed the complete overhaul that seems to be occurring with 'Vail'. 

    It looks like WHS V1 is poised to become the Windows XP of the home storage industry. I hope that Microsoft is prepared to continue supporting it for the next 10 years or so. Its something they will almost surely have to do until they realize that Vail and it successors will be huge flops if released without large file support that is equal to or better than what you currently get in WHS V1.

    -eB

    MS MCP, MSCE, MCSA

    Thursday, September 2, 2010 10:28 PM
  • It appears that people issues with DE v2 are quite clear by now.

    A solution for this does not apear to be that far from a reality.

    Currently Vail supports the use of NTFS drives in a normal fashion for the use of a removeable backup drive.

    For consumers that do not want to use Folder duplication it could be feasible that drives could be added to a WHS machine without being added to the drive pool and used in a normal C:, D:, E: fashion and at the same time left as a NTFS disk.

    Thus if the drive is removed from the server and plugged in to a normal machine, all is normal. Also if a drive dies only info on that drive is lost.

    For me the whole reason for running vail is for the Drive pool and Folder Duplication running at the same time. Seeing as my storage requirement are not that large, currently 1TB of data across 3x 1.5TB disk with a seperate system drive.

     

    Cheers

    Raenier

    Friday, September 3, 2010 10:06 AM
  • The next successful version of WHS will be the one that caters to ALL segments of its user base:

    The next successful version of Windows Home Server will be the one that reaches market, and that appeals to consumers. Consumers don't want to tweak every aspect of their servers, they want the vendor to deliver a product that does what's advertised. (Arguably, V1 fails at that; there are a lot of points of pain even for buyers of OEM units.) Consumers are the vast majority of sales of V1, and will be the vast majority of sales of V2. Not enthusiasts. Microsoft knows this because Windows Home Server collects a fair bit of usage data (non personal) which gets sent back to Microsoft on a regular basis. They know how the product is used, what kind of hardware it's running on, etc. I can only go by the dozens of people I've talked to about the product, 90% of whom have bought an OEM server they're happy with (because it sits quietly in a corner and "just works").

    For the most part, I agree to some degree with your reasons for wanting changes. That said, you're beating a dead horse: some minor tweaks to DE V2 may be possible (definitely are possible; Microsoft made some changes in DE V2 between the preview released in April and the one released last month) but a completely new storage architecture is a done deal.

    Some of your points are contradictory, and some reveal that you have not installed Vail anywhere; please install Vail on test hardware (or in a 64 bit VM) and learn about the differences between Vail and V1. Some you will probably hate, some you may like quite a lot. In any case, you will have an experiental basis for what you do and don't like. "I don't like the sound of that." is a second-hand opinion, I'm afraid...


    I'm not on the WHS team, I just post a lot. :)
    Friday, September 3, 2010 12:29 PM
    Moderator
  • Ken, 
     
    Most enthusiast most likely opt out of the auto reporting during install.  No hard data, just a hunch.  Consumers don't get the option as the vendor has already selected it they or click through it without paying attention.  Therefore the collected data is most likely skewed in favor of the consumer since the enthusiast opted out.  I usually opt in for beta boxes, but opt out on production boxes.

    --
     
    _______________
     
      BullDawg
      In God We Trust
    _______________
    The next successful version of WHS will be the one that caters to ALL segments of its user base:

    The next successful version of Windows Home Server will be the one that reaches market, and that appeals to consumers. Consumers don't want to tweak every aspect of their servers, they want the vendor to deliver a product that does what's advertised. (Arguably, V1 fails at that; there are a lot of points of pain even for buyers of OEM units.) Consumers are the vast majority of sales of V1, and will be the vast majority of sales of V2. Not enthusiasts. Microsoft knows this because Windows Home Server collects a fair bit of usage data (non personal) which gets sent back to Microsoft on a regular basis. They know how the product is used, what kind of hardware it's running on, etc. I can only go by the dozens of people I've talked to about the product, 90% of whom have bought an OEM server they're happy with (because it sits quietly in a corner and "just works").

    For the most part, I agree to some degree with your reasons for wanting changes. That said, you're beating a dead horse: some minor tweaks to DE V2 may be possible (definitely are possible; Microsoft made some changes in DE V2 between the preview released in April and the one released last month) but a completely new storage architecture is a done deal.

    Some of your points are contradictory, and some reveal that you have not installed Vail anywhere; please install Vail on test hardware (or in a 64 bit VM) and learn about the differences between Vail and V1. Some you will probably hate, some you may like quite a lot. In any case, you will have an experiental basis for what you do and don't like. "I don't like the sound of that." is a second-hand opinion, I'm afraid...


    I'm not on the WHS team, I just post a lot. :)

    BullDawg
    Friday, September 3, 2010 12:57 PM
  • Bulldawg, I doubt that most opt out. Some, sure, and it skews Microsoft's usage data somewhat, but I would bet that they at least know what percentage has opted out and apply a reasonable "fudge factor".

    In any case, Microsoft certainly at least knows numbers for OEMs (presumably mostly consumers) vs system builder packages (presumably mostly enthusiasts; I don't know of a lot of actual small system builders that do much with Windows Home Server) and Technet/MSDN downloads (presumably almost entirely enthusiasts). And that should be what they base "most users are consumers" on.


    I'm not on the WHS team, I just post a lot. :)
    Friday, September 3, 2010 1:26 PM
    Moderator
  • Raenier, that will work, as long as the user accepts that they drives not in server storage, and the data on them, is completely unmanaged from a Windows Home Server viewpoint: no duplication, no streaming, no remote access, etc.
    I'm not on the WHS team, I just post a lot. :)
    Friday, September 3, 2010 1:28 PM
    Moderator
  • yes!
     
    On Fri, 3 Sep 2010 12:57:13 +0000, BullDawg wrote:
     
    >Most enthusiast most likely opt out of the auto reporting during install
     

    Barb Bowman

    http://www.digitalmediaphile.com

    Friday, September 3, 2010 2:42 PM
  • And "no". :) I allow my computers to "phone home" if Mirosoft wants them to, and I defy anyone to say "Ken's not an enthusiast".

    In the long run, SQM provides additional data from which Microsoft can find issues that I don't even know are occurring, and it lets Microsoft adjust future versions of products to better meet the actual needs of actual users.

    For an example of the former, I understand that some of the changes in DE V2 are a response to issues found in SQM data that the users experiencing them are probably not even aware of. (No, I don't know what those issues or changes are.)

    For an example of the latter, I present the hated "Microsoft Office Ribbon Bar". It delivers the functionality people actually use, in an easier to use format if you take the time to get accustomed to it, so it can be said to better meet the needs of actual users. But it does so at the expense of requiring people who have been using Office products for many years to learn an almost completely new interface, which was egregiously bad planning if you ask me. :)


    I'm not on the WHS team, I just post a lot. :)
    Friday, September 3, 2010 5:12 PM
    Moderator
  • Ah.... but Ken is certainly not typical of 'most' enthusiasts... right?  :)
     
    Art [artfudd] Folden
    I'm a PC and Windows 7 sucks less!
    ----------------------------------------------
    "Ken Warren [MVP]" wrote in message
    news:7ed142e6-d525-4d91-9edb-bf1f8d10f152@communitybridge.codeplex.com...
     
    And "no". :) I allow my computers to "phone home" if Mirosoft wants them to, and I defy
    anyone to say "Ken's not an enthusiast".
     
    In the long run, SQM provides additional data from which Microsoft can find issues that I
    don't even know are occurring, and it lets Microsoft adjust future versions of products to
    better meet the actual needs of actual users.
     
    For an example of the former, I understand that some of the changes in DE V2 are a
    response to issues found in SQM data that the users experiencing them are probably not
    even aware of. (No, I don't know what those issues or changes are.)
     
    For an example of the latter, I present the hated "Microsoft Office Ribbon Bar". It
    delivers the functionality people actually use, in an easier to use format if you take the
    time to get accustomed to it, so it can be said to better meet the needs of actual users.
    But it does so at the expense of requiring people who have been using Office products for
    many years to learn an almost completely new interface, which was egregiously bad planning
    if you ask me. :)
     
    I'm not on the WHS team, I just post a lot. :)
     
     
    Friday, September 3, 2010 6:46 PM
  • Ah.... but Ken is certainly not typical of 'most' enthusiasts... right?  :)
     
     

    Whats your point exactly?

    If we are talking about the issue that enthusuasts don`t / won`t allow telemetry to be gathered from their test vail installations, then thats a pretty self defeating position to put yourself in.

    Friday, September 3, 2010 10:30 PM
  • Like I said before, people who go out and buy an HP MediaSmart may be the most common user (they have to be) but they didn't generate the immense positive buzz WHS v1 had. That was done for the most part by 'power users' who have massive data, found how powerful WHS was and wrote on blogs/forums.

    Seeing MS and some posters here act like those users don't matter now because they aren't the 'typical' audience is discouraging to say the least. "it lets Microsoft adjust future versions of products to better meet the actual needs of actual users" - doesn't really mean anything.

    Customers want what you give them. Before WHS their needs were being met by Raid and whatnot. Then WHS came along and did a better job. Vail is taking a step back when it should have been introducing yet more innovation. Some other product may come along that will serve those needs better.

    I obviously realize you can't please everyone. The backlash against Vail is there for a good reason, its not just a bunch of people with unrealistic demands. Its a bit like if Toyota killed off Lexus and only sold the Prius because it sells 10x more. Vail is designed and built for the lowest common denominator.

     

    Saturday, September 4, 2010 12:39 AM
  • Art, it's more like "Ken isn't given to paranoid fantasies." I hear two reasons for people to turn off the SQM tools in Microsoft (and other) products.

    One is bandwidth; some users with limited monthly bandwidth allotments feel that the SQM data takes up too much. They'd need an allotment of 1 GB a month or so for it to make much of a dent, but okay, it does use bandwidth, so I'll buy it.

    The other is that they don't trust Microsoft to collect only what they say they're collecting. So I have to ask: if you don't trust them to not collect personally identifiable data and use it in nefarious ways, why in heck do you trust them to actually turn off the data collection when you say to? C'mon, use some sense...


    I'm not on the WHS team, I just post a lot. :)
    Saturday, September 4, 2010 2:13 AM
    Moderator
  • Like I said before, people who go out and buy an HP MediaSmart may be the most common user (they have to be) but they didn't generate the immense positive buzz WHS v1 had. That was done for the most part by 'power users' who have massive data, found how powerful WHS was and wrote on blogs/forums.

    That may be true, but I don`t think that microsoft care that it was received well by people with huge storage needs - because they are not who the product is intended for.

    Seeing MS and some posters here act like those users don't matter now because they aren't the 'typical' audience is discouraging to say the least. "it lets Microsoft adjust future versions of products to better meet the actual needs of actual users" - doesn't really mean anything.

    Discouraging to you it may be, but I think theres a time when you need to appreciate commercial reality. Microsoft have a clear brief for WHS - to appeal to the storage needs of the non technical masses. They use telemetry to help isolate faults, and helpmake design decisions. Whats not to understand? Sometimes products follow development paths that isolate small groups of users, and they are unhappy. Thats life.

    Customers want what you give them. Before WHS their needs were being met by Raid and whatnot. Then WHS came along and did a better job. Vail is taking a step back when it should have been introducing yet more innovation. Some other product may come along that will serve those needs better.

    Before WHS their needs were *not* met at all. WHS is aimed at people who have no idea what a RAID set is.

    I obviously realize you can't please everyone. The backlash against Vail is there for a good reason, its not just a bunch of people with unrealistic demands. Its a bit like if Toyota killed off Lexus and only sold the Prius because it sells 10x more. Vail is designed and built for the lowest common denominator.

    Your demands are not unrealistic, they are just not as well catered for in the new version. You already know why that is. At least you have the skill set to go build a raid array. The people MS are aiming WHS at have no clue how to cater for their own needs.

     

    Sunday, September 5, 2010 9:31 AM
  • OK, MS has taken a target decision and a corresponding design decision. Here is blustered on "huge" amounts of storage and "target groups", and puzzled, if an application fits in the target scheme. Are 10 HDDs, i.e. 20 TB of storage, "huge" or still in the "target"?
    These discussions are completely useless in the case. You can have the sense to prepare whs2 have narrower limits than whs1.
    Now the question is, what limits? And Limits of what kind, only "not supported" or "sorry, no further hdd possible"?
    Unfortunately the target decision (lets say 5-6 TB user) allows namely a technical design decision that seems sufficiently secure to work only within this target use. Once again, what if the target user decides (because its so "easy-to-use") to upgrade up to 20 TB or 25 TB of storage? How is ms prevent it, by hard limits ("sorry, no ...") or MS waives the "extensibility-feature" in the future?
    Sunday, September 5, 2010 2:28 PM
  • Who exactly are the users whose needs were not being met by WHS v1? it was a perfect product for people with modest data storage, the only complaint was that restores used to hang (~80%) and from what I read this is *still* present in Vail.

    They removed streaming and transcoding features, didn't add media center support (which was in high demand), in fact haven't really added any features at all.

    Its not going to scale. Doesn't MS want people to start using Media center to replace their dvr's and use more digital media? Where do they want users to store all this data?

    I'd like to see some data (which I know I won't since its not public) about what %age of people expand the storage in their MediaSmart's and if they ever add more than 1 disk. I would bet its miniscule.

    You know what I think happened - WHS team knew DE v1 was a hack and wanted to overhaul it. After going through a bunch of PM's (usual at MS), the new design ended up being what it is now. Now its being backward rationalized that its meant to cater only to a very limited set of users. I would bet good money that the architects themselves are not happy with Vail.

     

     

    Wednesday, September 8, 2010 4:47 AM
  • You're still talking about undocumented functionality.

    In V1, Microsoft's answer to your scenario will be a server recovery, not moving drives around to copy files from a hidden location. That's a functional and flexible workaround that avoids server recovery, which can be problematic on home-built servers due to configuration issues. Instead of recovery you progress directly to a fresh installation...

    It is perfectly documented. Please stop misinforming. Quote from WHS v1 Drive Extender technical brief:

    "If the home server fails completely, all the surviving drives can be attached to a computer that is not even running Windows Home Server Drive Extender, and you can copy the files from the drives to that computer".

    http://www.microsoft.com/downloads/details.aspx?FamilyID=40c6c9cc-b85f-45fe-8c5c-f103c894a5e2&DisplayLang=en

    I love how there was no response to this. Although in the doubters defense it does clearly state "Although this works as a disaster recovery method, moving a secondary hard drive between computers is not a recommended or supported use of Windows Home Server Drive Extender." However I still stand by my statement that getting SOMETHING back is better than getting NOTHING back as in RAID scenarios. When Joe Blow's HP Media Smart kicks the bucket its nice to be able to save some of his grand kids pictures rather than saying 'Well its all gone you should have backed up'. The thing that is really laughable to me is the solution proposed is to 'Backup your Vail Server as you would a window file server', yet in the next breath you say your target audience is 'non-techy', what makes you think they're going to backup their Vail box?
    Wednesday, September 8, 2010 5:00 AM
  • You're still talking about undocumented functionality.

    In V1, Microsoft's answer to your scenario will be a server recovery, not moving drives around to copy files from a hidden location. That's a functional and flexible workaround that avoids server recovery, which can be problematic on home-built servers due to configuration issues. Instead of recovery you progress directly to a fresh installation...

    It is perfectly documented. Please stop misinforming. Quote from WHS v1 Drive Extender technical brief:

    "If the home server fails completely, all the surviving drives can be attached to a computer that is not even running Windows Home Server Drive Extender, and you can copy the files from the drives to that computer".

    http://www.microsoft.com/downloads/details.aspx?FamilyID=40c6c9cc-b85f-45fe-8c5c-f103c894a5e2&DisplayLang=en

    I love how there was no response to this. Although in the doubters defense it does clearly state "Although this works as a disaster recovery method, moving a secondary hard drive between computers is not a recommended or supported use of Windows Home Server Drive Extender." However I still stand by my statement that getting SOMETHING back is better than getting NOTHING back as in RAID scenarios. When Joe Blow's HP Media Smart kicks the bucket its nice to be able to save some of his grand kids pictures rather than saying 'Well its all gone you should have backed up'. The thing that is really laughable to me is the solution proposed is to 'Backup your Vail Server as you would a window file server', yet in the next breath you say your target audience is 'non-techy', what makes you think they're going to backup their Vail box?


    As a Beta tester of Vail, I did setup a backup of the Server with the built-in option to do so.
    I doubt that I'm gonna be able to do so with the entire storage managed by the WHS v1  - due to its size.
    But, as external USB-drives gets bigger and cheaper, that doesn't seem like an unclimbable fence, so to speak... :-)

    The downside (to me personally so far) is that I've got more "left-over" drives now, when I choose to upgrade the storage units in the WHS v1 machine from, let's say 250 to 400 GB-range to the more common 1.5 - 2 TB drives of today.
    Maybe I'll put all of those in a separate cabinet for backup reasons?

    And yes, I'm gonna buy the OEM license when it's released!


    One WHS v1 machine in the basement with a mixed setup of harddrives in and outside the storage pool. And now, next to it, a Vail Refresh brother for beta duties.
    Wednesday, September 8, 2010 6:50 AM
  • I love how there was no response to this.
    Actually, I did. I thanked Mick for the reminder that this was mentioned (because I had forgotten), and then pointed out that it wasn't supported or promised as permanent functionality. Documented <> supported, and unsupported = subject to change.
    ... what makes you think they're going to backup their Vail box?

    Vail makes backups of the server easier than V1. You can plug in a drive, configure backups once, and then your server will back itself up (including system state for bare metal restore) every 12 hours until the drive is full with no further interaction on your part. That said, I think you're correct that the average user of Windows Home Server won't back up their server, because they don't understand why they should. And frankly it's impossible for Microsoft to educate them on this point; it's hard enough to convince a good friend who's lost data to drive failures in the past(?!) to back data up.


    I'm not on the WHS team, I just post a lot. :)
    Wednesday, September 8, 2010 12:45 PM
    Moderator
  •  
    "Ken Warren [MVP]" wrote in message
    news:130499f1-1ac1-4248-a21d-b95dd6c2f8c6@communitybridge.codeplex.com...
    >  I love how there was no response to this.
    > Actually, I did. I thanked Mick for the reminder that this was mentioned
    > (because I had forgotten), and then pointed out that it wasn't supported
    > or promised as permanent functionality. Documented <> supported, and
    > unsupported = subject to change.
    >  ... what makes you think they're going to backup their Vail box?
    > Vail makes backups of the server easier than V1. You can plug in a drive,
    > configure backups once, and then your server will back itself up
    > (including system state for bare metal restore) every 12 hours until the
    > drive is full with no further interaction on your part. That said, I think
    > you're correct that the average user of Windows Home Server won't back up
    > their server, because they don't understand why they should. And frankly
    > it's impossible for Microsoft to educate them on this point; it's hard
    > enough to convince a good friend who's lost data to drive failures in the
    > past(?!) to back data up.
    >
    >
    > --------------------------------------------------------------------------------
    > I'm not on the WHS team, I just post a lot. :)
    >
    I have WHS Vail Refresh set to back up the server. You mention that ". . .
    your server will back itself up (including system state for bare metal
    restore) every 12 hours until the drive is full with no further interaction
    on your part."
     
    What happens when the drive gets full? Will it then start dropping old
    backups to make space for new ones?
    --
    Nancy Ward
    Windows 8 Beta Ferret
     
     

    Nancy Ward
    Wednesday, September 8, 2010 1:46 PM
  • Who exactly are the users whose needs were not being met by WHS v1? it was a perfect product for people with modest data storage, the only complaint was that restores used to hang (~80%) and from what I read this is *still* present in Vail.

    I doubt many want the product to stand still. Like I said before, this is a commercial venture were talking about here, MS will chase the money with features which appeal to the masses. Its not rocket science. WHS has to justify its existence as a product. Its not going to do that by pandering to the vocal minority who run self built multi TB monsters.

    By the way, have you tested the restore bug in vail you elude to and posted it on connect? Didn`t think so.

    They removed streaming and transcoding features, didn't add media center support (which was in high demand), in fact haven't really added any features at all.

     You can`t seriously claim there are no new features added, just becuase you don`t like whats been produced. Thats laughable. What about DLNA support? Remote Access upgrades? Error correction? Add-in Store? And they have certainly not removed streaming and transcoding - they`ve actually improved those fetures as far as I can tell.

    Media centre support was something I was very sore about (ask Ken). But the reality is, in the grand scheme of things, nobody uses it. Its yet another feature thats only popular with enthusiasts. Thats why it didn`t make the cut. And thats why DLNA support did. You can`t seriously criticise microsoft for moving with the times, can you?

    Its not going to scale. Doesn't MS want people to start using Media center to replace their dvr's and use more digital media? Where do they want users to store all this data?

    Media centre is dead, its all about DLNA now. Recording TV on your Media Centre/WHS is not a mainstream activity, and MS know it. Who makes standalone extenders anymore? Nobody. They all got dropped. They were not popular with the masses. 

    I'd like to see some data (which I know I won't since its not public) about what %age of people expand the storage in their MediaSmart's and if they ever add more than 1 disk. I would bet its miniscule.

    Not sure what you are getting at here, perhaps you could elaborate.

    You know what I think happened - WHS team knew DE v1 was a hack and wanted to overhaul it. After going through a bunch of PM's (usual at MS), the new design ended up being what it is now. Now its being backward rationalized that its meant to cater only to a very limited set of users.

    The "very limited set of users" you elude to, make up the vast majority of customers (in numbers).
    I would bet good money that the architects themselves are not happy with Vail.

    I`ll take that bet. What are your odds?

    Wednesday, September 8, 2010 6:45 PM
  • I would bet good money that the architects themselves are not happy with Vail.

    I`ll take that bet. What are your odds?

    I wouldn't. From "The Devil's Dictionary":

    Such an adjustment of conflicting interests as gives each adversary the satisfaction of thinking he has got what he ought not to have, and is deprived of nothing except what was justly his due.

    And from high school civics, the operational definition of a compromise is that everyone is equally unhappy.

    I'm pretty sure that the team is satisfied (it's "good enough"), but I doubt they're happy (it's "as good as possible").


    I'm not on the WHS team, I just post a lot. :)
    Wednesday, September 8, 2010 7:23 PM
    Moderator
  • "Personally, I think it would be great if Microsoft could find a way to make the people with enormous movie collections happy without compromising performance and reliability for the target audience. But if Microsoft has to choose one or the other, it's a decision that should really make itself."

    Just my .02.  I'm an average guy with a WHS.  I'm not in IT, I have a full time job, a wife, kids, dogs, and little spare time.  When WHSv1 first came out, I was using my computer to surf the net on my large screen TV.  I was fascinated by the idea of a server to hold all my movies, pictures, etc.  I liked the idea of backing up my home computers (what a concept!).  I got on Connect and tried it.

    Next thing I know, I've bought a license, and built a 9TB server using parts on sale from Newegg.  My entire movie collection is on there, and I had to turn off duplication as it was getting too big.  I want duplication, and I need to expand my storage even more.

    I always assumed I was the typical user.  I lurk on these forums and wegotserved, and learn a lot, but I don't post much, and I didn't have much to say.  Now I and those like me are paying for our lack of participation in the discussion. 

     

    Alan

    Wednesday, September 8, 2010 8:19 PM
  • Next thing I know, I've bought a license, and built a 9TB server using parts on sale from Newegg.
    Alan, by virtue of the above (the "built" part, not the "9TB" part) you are demonstrably not the average Windows Home Server user. The "average" user, in my experience (and I've talked with dozens of users) bought an HP MediaSmart server, an Acer, etc. and it sits in a corner and "just works". A vocal minority is (unfortunately for the minority) still a minority and isn't likely to be a focus when designing a consumer product.

    I'm not on the WHS team, I just post a lot. :)
    Wednesday, September 8, 2010 8:30 PM
    Moderator
  • That's really about the size of it, scoob. And I literally laughed out loud. :)

    Personally, I think it would be great if Microsoft could find a way to make the people with enormous movie collections happy without compromising performance and reliability for the target audience. But if Microsoft has to choose one or the other, it's a decision that should really make itself.


    I'm not on the WHS team, I just post a lot. :)


    Here's the problem that I see all the time with developers.   They use words like "target audience" which is the audience that justifies their design.  

    I find it ironic that us people with large movie collections are perfectly happy with the way WHS v1 works today DE wise, and most certainly if WE are happy so are the target audience.   Sure there's room for improvement.... but certainly not in the direction it's moving :)

    How about we just do what should have been done in the first place, rev WHS to 2008 64bit base, add 64bit client restore, some new fluff features and call it a day.  I know I'd be thrilled to death compared to the feeling I have about Vail so far.   I am completely torn between having a better faster 64bit OS and the utter fear of disaster that the DE v2 system creates in me.

    Friday, September 24, 2010 8:48 AM
  • One more thing: A "deal breaker" is something whose presence or absence will result in you choosing another product with no room for flexibility ("I need this feature to use your product." rather than "I need most of this list of features."). The people who have referred to the DE V2 block storage allocation as a "deal breaker" have polarized this whole discussion far more than anything I've said (go back and read the whole thread, and I think you'll probably agree). If block allocation of data storage as implemented in Vail is truly a deal breaker, it's time for these people to start analysing alternatives because I don't think it's going to change.


    I'm not on the WHS team, I just post a lot. :)

    Or they can continue to voice their concerns AND complain, loudly and even rudely if necessary, until MS gets it.   Many if not most of Vail adopters would be WHS v1 users and we deserve to be heard right up until and even past launch date, if that's what it takes.


    Nothing is more insulting that to be told by a company, it's representatives, or those affiliated with them to "go elsewhere if you don't like it".  

     

    I think MS is trying to improve the product, yes, but in so doing they are introducing a large amount of risk for a lot more customers than you imply in your posts Ken.

    Friday, September 24, 2010 9:00 AM
  • ...
    2. Cold hard facts - enable duplication, use 66% overhead for all data, or you WILL lose everything (disk failure is inevitable).
    ...

    56%, actually. The 12% for ECC is "off the top" of every disk, and reduces overall disk capacity. As for losing everything, that's not my experience, even in the current build (which has known issues in the area of server storage and drive removal). What is your test case?

    I disagree when you say Vail won't be simple for an end user (who will generally buy it preinstalled from an OEM either online or via e.g. Best Buy). End users will only see the dashboard. It has, however, become more complex for an enthusiast.

    And as for how storage is used, allocated, etc. Microsoft's goal is to allow a user to just buy a new drive when storage needs to be expanded, with little thought to drive size and no requirement to match other drives already installed. Vail will deliver that, just as V1 did. The trade-off for simple capacity expansion with good data protection without a high-end RAID HBA is that it's done through mirroring plus ECC, which has higher storage costs in the long run.


    I'm not on the WHS team, I just post a lot. :)

    You are correct, it will be simple for a layman.  They'll simply lose everything and not have a clue why where the enthusiasts will know it's due to the new DE system :)
    Friday, September 24, 2010 9:04 AM
  • You are correct, it will be simple for a layman.  They'll simply lose everything and not have a clue why where the enthusiasts will know it's due to the new DE system :)

    Totally false. Based on what we know right now, users who do not store large numbers of unduplicated >1GB files, DEv2 is far superior to DEv1 (for me, DEv2 is the main reason I`ll be upgrading to vail).

    Its not unreasonable to assume that a laymans (OEM) vail machine will most likely have duplication turned on for all  folders as default when shipped as a multi drive system.

    Now where is that DE white paper?? When that comes along at least we will have something tangible to talk about, rather than all this assumption and speculation.

     

     

    Friday, September 24, 2010 10:28 AM
  • You are correct, it will be simple for a layman.  They'll simply lose everything and not have a clue why where the enthusiasts will know it's due to the new DE system :)

    Totally false. Based on what we know right now, users not storing large numbers of >1GB files, DEv2 is far superior to DEv1 (for me, DEv2 is the main reason I`ll be upgrading to vail). Don`t forget that Vail is an OEM product, so they don`t control how many drives it ships with.

    Now where is that DE white paper?? When that comes along at least we will have something tangible to talk about, rather than all this assumption and speculation.

     

     


    How is your rebuttal any more valid in the absence of the DE white paper?  Claiming it is far superior is just another assumption and speculation on your part.

    As for losing the majority of large files on non-duplicated files when a single drive fails, this has been proven by other testers - search for other threads on the subject - so this is not speculation. You can question the reliability of those tests, and there are different degrees of data loss for different people, but to simply state it is an assumption and speculation that it is happening with DE2 is false.

    Friday, September 24, 2010 10:42 AM
  • A reorganized and condensed view of the issue:

    DE V2 is less "protective of data", in practice, than DE V1 for users A) with large volumes of data in large files (> 1GB), and B) with duplication turned off (presumably because of storage costs) because of the block allocation of space on drives in V2. As for testing, results have varied significantly. I lost no more than I would have with V1 (within a reasonable margin of error), some lost a lot more, and some lost less.

    At the end of the day, though, the root of this is that the physical unit of protection for files is changing from the entire file to a 1 GB block. Loss of a single block within a file should be counted as loss of the entire file and losing a disk could cost a single block from every large file that isn't duplicated. All of this is not subject to argument (by me, anyway :) ).

    However, the physical unit is not what people should look at. The logical unit, which is what data protection requirements will have been written for by Microsoft, is not the file, or the block, it is and always has been the share. Microsoft has made clear for years that if you turn off duplication for a share, you put all the files in the share at risk. Just because the physical unit is documented as the file for V1 doesn't mean that it won't change, because the physical unit doesn't really matter to Microsoft except technically (they need to manage those physical units, so they need to know about them, manipulate them, track them, etc).

    End users really shouldn't concern themselves with internal (and hidden from the end user in any "supported" sense) implementation details, so end users should never have been looking at the physical units in the first place. ISVs may base their products on internals, whether supported or not (it's horrifyingly common practice for products to include functionality based on some undocumented Windows API call that was found by trolling through decompiled DLLs), but they're taking a risk when they do so. It's possible to build a solution around the internals of this (or any) product, but if you do so you should do so expecting that you are locking yourself in to a specific version of the product.

    Personally, I will be moving to Vail (I already have, in effect, because Vail is running in parallel with V1, and most of my home computers are connected to Vail), because I like the improvements for the most part, and I use duplication everywhere (except the backup database). Someone who doesn't use duplication, who's built a large volume data storage solution on the internals of V1 (maybe using some file or disk parity software to protect data at a smaller cost than RAID 1 equivalents), won't move forward. These people made a choice that has, it turns out, locked them to a specific version of the product. In order to move forward to a new version, they'll have to re-engineer their solution.

    Choices have consequences, it's that simple.


    I'm not on the WHS team, I just post a lot. :)
    Friday, September 24, 2010 1:30 PM
    Moderator

  • How is your rebuttal any more valid in the absence of the DE white paper?  Claiming it is far superior is just another assumption and speculation on your part.

    As for losing the majority of large files on non-duplicated files when a single drive fails, this has been proven by other testers - search for other threads on the subject - so this is not speculation. You can question the reliability of those tests, and there are different degrees of data loss for different people, but to simply state it is an assumption and speculation that it is happening with DE2 is false.

    Are you suggesting my rebuttal is not valid? I was simply responsing to the (frankly crazy) proposal that laypeople will "simply lose everything and not have a clue why where the enthusiasts will know it's due to the new DE system :)" Seriously, you don`t beleive that statement, do you? Though, I suppose you do agree if you replace "laypeople" with "people with huge movie collections".

    All I`m saying is that DEv2 is far superior to DE v1 for storing <1gb files based on its advertised features. Thats a pretty reasonable basis for the claim. Funnily enough, the dissenters store much larger files and many more of them (how many times do we need to go over this?).

    I`m not disputing the test results - although its already been pointed out that theres a pretty mixed bag of results out there. Do you know why that is? No. And neither no I. And frankly, I don`t care either. It doesn`t affect my intended usage of the software.

    These arguments have been done over enough times already - theres nothing new to add here. If you want to secure your data - duplicate it. If you cant afford the diskspace then it looks likely that DEv2 isn`t for you.

    By the way, I do still want to see chunks from the same share on the same drive (where possible). Activley spreading them all over the place appears to add no value for extra risk.

    Friday, September 24, 2010 7:05 PM
  • Vail makes backups of the server easier than V1. You can plug in a drive, configure backups once, and then your server will back itself up (including system state for bare metal restore) every 12 hours until the drive is full with no further interaction on your part. That said, I think you're correct that the average user of Windows Home Server won't back up their server, because they don't understand why they should. And frankly it's impossible for Microsoft to educate them on this point; it's hard enough to convince a good friend who's lost data to drive failures in the past(?!) to back data up


    I don't really see the point of having a complete backup of the server in the same place as all the other data, why should one have duplication on if one just could tell the server to take a new backup every 30 minutes or so? Then one would have tripled the data in the same machine. If it was able to get the backup offsite by automation I would see more use for this but for my needs duplicated folders and an occasional offsite copy when more data has been changed than I want to lose is sufficient. Shadow copies should ideally handle most of the other scenarios, accidental deletes or overwrites... I haven't read enough to really have a good insight on this matter but it seems a bit odd to me as a user that are more than happy keeping my data on just two disks. I think there are more users like me than those with >8TB storage needs although I think a good storage system should handle a larger data volumes than just some TB's as HD video are growing in popularity.

    Friday, September 24, 2010 9:14 PM
  • All I`m saying is that DEv2 is far superior to DE v1 for storing <1gb files based on its advertised features. Thats a pretty reasonable basis for the claim.
    Where did this assumption come from? Who advertised that files smaller than 1GB would not be split across different drives?

    Suppose you have 1GB chunk filled with 900MB of files and you add a new 200MB file. Where would it go? From what I know nobody guaranties that it won't have 100MB part in that chunk and the rest 100MB in different chunk on another drive.

    And this is likely to happen when drive is close to being full. If developers had a goal to minimize storage overhead they might allow even small files to split across drives.

    I`m not disputing the test results - although its already been pointed out that theres a pretty mixed bag of results out there. Do you know why that is? No. And neither no I.

    It is because data distribution scenarios were different during these tests. Nobody says that Vail deliberately puts every file piece on different drive. It just may happen. It does not have to be proved by every test in order to be considered as a problem. According to Murphy law it will happen to you when you are least prepared for it.

    Testing this is like playing Russian roulette. Test results can only be consistent if
    1) Vail developers state that mechanism exists to prevent file pieces from being placed on different drives. (didn't happen)
    2) This mechanism works well. And such mechanism is what really would make sense to test if it existed.

    • Edited by micksh Sunday, September 26, 2010 12:06 AM minor clarification
    Friday, September 24, 2010 10:21 PM

  • How is your rebuttal any more valid in the absence of the DE white paper?  Claiming it is far superior is just another assumption and speculation on your part.

    As for losing the majority of large files on non-duplicated files when a single drive fails, this has been proven by other testers - search for other threads on the subject - so this is not speculation. You can question the reliability of those tests, and there are different degrees of data loss for different people, but to simply state it is an assumption and speculation that it is happening with DE2 is false.

    Are you suggesting my rebuttal is not valid? I was simply responsing to the (frankly crazy) proposal that laypeople will "simply lose everything and not have a clue why where the enthusiasts will know it's due to the new DE system :)" Seriously, you don`t beleive that statement, do you? Though, I suppose you do agree if you replace "laypeople" with "people with huge movie collections".

    All I`m saying is that DEv2 is far superior to DE v1 for storing <1gb files based on its advertised features. Thats a pretty reasonable basis for the claim. Funnily enough, the dissenters store much larger files and many more of them (how many times do we need to go over this?).

    I`m not disputing the test results - although its already been pointed out that theres a pretty mixed bag of results out there. Do you know why that is? No. And neither no I. And frankly, I don`t care either. It doesn`t affect my intended usage of the software.

    These arguments have been done over enough times already - theres nothing new to add here. If you want to secure your data - duplicate it. If you cant afford the diskspace then it looks likely that DEv2 isn`t for you.

    By the way, I do still want to see chunks from the same share on the same drive (where possible). Activley spreading them all over the place appears to add no value for extra risk.


    No, your rebuttal may be entirely valid.  The point is your opinion is based on the same assumptions and speculation that you dismiss of someone else.  You can't have your cake and eat it.

    Why is DEv2 far superior?  Again this has to be an assumption and speculation in the absence of the DE paper, unless you trust everything as advertised.   However there is sufficient evidence to question the reliability of DEv2 in certain circumstances that you'd be far more crazy to trust  your data to a new technology than an old one until thoroughly tested and understood.

    You've never answered a previous question over what the target audience is, if this was made absolutely crystal clear then it may clarify a lot - because the WHS website suggests it is an expandable storage solution  which can be used for media throughout the house - I'm sure having any movie collection, small or large fits that description.  In any event during Vail's lifetime it would not be surprising if large files, including movies increased as the move to more HD content digitally delivered becomes common place.  As long as the option to turn duplication off is an option, then there is no reason to believe that "laypeople" will not be affected.

    So the software suits you fine, DEv2 suits you fine.  I'm not sure why you are trying to convince people in a thread entitled "deal breakers/complaints" that it isn't a valid concern for them.  If it's not a deal breaker/complaint for you, then move on and let people who have concerns voice them.  The arguments continue, because people like yourself dismiss them, or try to convince people they are not important. 

    And actually, your last point sums up exactly what I would like to see happen and if implemented would bring Vail to near WHSv1 risk profile for unduplicated shares - so if you want to see this happen, why be so dismissive of the conversation as to the reason for wanting it?

    Friday, September 24, 2010 11:42 PM
  • I just lost one full year of data due to a server failure.  With the released version I was able to read the harddrive on another machine and could retrieve the data.  With Vail I could not.  I had to rebuild the server from scratch because the server restore function is still not working.  I am seriously considering just builing a file server and use a third party for machine backups.
    Saturday, September 25, 2010 1:54 AM
  • WHS (both v1 and Vail) backup strategies are ridiculous and that's why a lot of people don't use them.

    1st of all, like Ken says, Microsoft wants us to treat a share as the logical unit. Then they tell us the share must be duplicated. Fine, that's good. But on top of that I have to take backups of the entire server. So now I have 3 copies of data. Why do I need 3 copies? I don't. The only reason folder duplication is mandatory is because of DEv2 design limitations. Taking backups on external media is the best way to keep your data safe (its a real backup). DEv2 is theoretically 'superior' because it implements a number of advanced features such as crc and block level correction, however none of that means anything if your data is gone.

    Duplication is high availability - not that important for most users. Server backup is backup - critical.

    WHS has been promoting a false sense of security in most users, by teaching them that turning on folder duplication was the way to keep your data safe. Since they didn't even have proper server backup until PP2 (I think) and the risk due to drive loss was mitigated, this was fine. In Vail this makes no sense at all and I'm effectively forced to allocate not 2 but 3x the storage space.

    There is so much song and dance about who Vail is meant for. Microsoft will never come out and say 'don't use/upgrade if you more than 3 disks/2tb data because its not meant for you and we don't care about you'. Even though that's exactly what they mean.

    Ken you keep saying that Vail is more reliable, you like the improvements and you will be switching to it. That's fine, its your choice. But your posts also strongly imply that Vail is fine for the general user, and only power users who are 'suffering the consequence of their choice' will avoid it. As moderator and the unofficial voice in this forum, I would expect a more measured view - if anything one would err on the side of caution and warn prospective users about the high risk of data loss (for everyone) and the unknowns in a beta product.

     

     

    Saturday, September 25, 2010 6:12 PM
  • ...
    Duplication is high availability - not that important for most users. Server backup is backup - critical.
    ...

    This - mostly. In addition to high availability, duplication offers some protection from the most common data loss scenario: a single hard disk failure. If a user chooses (through reasoned decision or ignorance) not to take backups off-site, duplication protects that user's data from the failure of a disk. So it's better than nothing, but not hugely. As I've been saying for years, the only way to truly protect your data is to take a copy and store it elsewhere regularly. V1 makes that possible, but difficult, since you have to take the backup manually. V2 automates the backup process and uses the much more robust Windows Server backup tool (which, I'll note, has it's limitations, but it's still enormously better than the V1 equivalent).

    As for the rest: I suggest you look at who wrote the announcement (titled Please note that the use of this preview is not supported in a production environment.) about not using a beta for production up at the top of this forum. Then come back and apologize for accusing me of not warning people about the potential issues in a beta.

    I have repeatedly told people that if they only have one server, it should not be running Vail. Unfortunately people tend to think of beta software as "time-limited but otherwise free software for me to use". Plus, people mostly can't be bothered to read the warnings and take a moment to understand them (or ask what the warnings mean if they don't understand them. Not my fault, not my problem, but I'll feel bad if someone loses a decade of family pictures to a bug anyway.


    I'm not on the WHS team, I just post a lot. :)
    Saturday, September 25, 2010 9:18 PM
    Moderator
  • All this chat strikes me as people trying to use the wrong tool for the job, plus Microsoft perhaps missing a trick or too.  What VAIL appears to be offering is software RAID 0 or 1 - so striping for performance or duplication for security?  It's also trying to abstract away from users seeing multiple disks.  

    The strange thing is that W2k8 supports both mirroring AND extending partitions, so could have used the native functionality???  However, the fact is that extending a volume doesn't guarantee that large files remain on single disks - and this in the case in any similar system - e.g. linux LVM.  The only way to guarentee data remains on single disks is to put it there yourself!  You can abstract this with something Microsoft DFS though you'd need AD to do this.

    So if i were running TB of data at home, I'd be looking for a RAID 5 or 6 solution.  An apparently cheap way of doing this could be run a second server as a NAS unit using linux to provide software RAID 0/1/5/6/whatever and present the volumes to the WHS as iSCSI drives and ignore the options in WHS?  But by the time this is done, you're probably close to a hardware RAID solution.

    TBH, I'm a bit torn with WHS either v1 or v2. Whilst having a simple gui is nice, and the remote control/internet sharing features are nice, it's not really adding much to the W2K8 server party?  


    For example, use a vanilla copy of W2K8 and a NAS box, roll in AD and roaming profiles - WHS really should have this - family has a couple of laptops and a big desktop - want to roam between the lot with IE/Outlook settings etc.  Backup server using Windows Backup to a removable disk.  Use Windows Backup to an iSCSI/Share for PCs/laptops if required.

    Remote control, photo/media sharing via MESH (Live Sync now), logmein, dropbox etc, etc.  So just though in a media server somewhere and it's done.

    Anyway, bottom line is RAID 5 AND a good backup is the bare minimum for storing large amounts of data even if you've got the original data on DVD or whatever - who's got time to recover TB's of data by re-riping CDs and DVDs?

    So would praise MS for what WHS is trying to do, get a backup device into a home where there's no real techie knowledge to save the family photos, I'd say there's a load of users on this thread that's pushing it's boundary's way too far and should be looking at stepping up their solution!

    Paul

    PS: Back to work - my team only has 200TB of SAN disk  to manage!!

     

     

     

     

     

    Wednesday, September 29, 2010 9:24 AM
  • Part of the problem here may be defining what WHS should and really shouldn't do. I think most people have the expectation that if they can load an app in the intended meathod for the server it should just work. WHS is advertised as a central point for a home/familys various document, files, Media, to provide bare metal backups, and remote access. It is being made (by OEMs) in ways to make it very energy efficient, and productive, and other then the 10 drive limit(for vail preview) stated above I haven't seen a Max hardware supported. The basic platform and conept are good, but have many stated with Vail the new DE design will cause certain flexability to go away and add some new flexiablity.

    I would like to build a little bit of perspective. First I am running a MSS 485 with WHSV1 and a DIY system with Vail between them I have 2 1.5TB drives 2 2TB drive and 1 750TB drive. Most of the storage is in the MSS485 as I am still tring to figure out the best way to use vail in my enviroment. The things that drove me to look at the MSS 485 was a desire to provide a low energy solution for storing and operating at the time 1 1.5TB hard drive with my movie collection. That low energy solution is also what drives the desire for MCE in WHS, or most specifically a tuner pool in WHS. Over time my movie collection has grown a bit as I added Blu-Ray ISO's and Recorded content from Cable. Beyond the movie collection everything i store goes to the MSS. My PC's are simply for Processing power and to get access to what is on my MSS. Everything except the Movie collection is Duplicated. That is simply because if i duplicated the movie content I would need about 3TB more and not able able to use the MSS. Not only that but then once you start to mirror everything why not just run RAID1. The more i read this the more i am leaning toward a Larger server with many 2.5 drives. With a raid array

    The appeal of DE is the ability to get more storage from a set of drives that wouldn't be avaliable otherwise. When compared to raid where there is very clear math associated with the loss DE allows for that math to get rather dynamic. espcially if the majortiy of the files are smaller and really don't need a huge array to be raided to protect a small total amount of data. With Multimedia files residing on the server the flexability with WHS v1 allows for a expected loose of minimal data in the worst case scenerio i will loose the quanity on 1 drive. Where as there is a good chance with Vail every blu-ray ripp would be lost and my HD recorded Content to be lost. We can't loose sight of the fact HD recordings are typical now and many people are looking for new creative ways to store it.

    My expectation is two things first users that don't even know there devices can run more then one hard drive won't be effected by this as they will never have multiple drives and don't use it in that manor. Secondly the vast majority of those users that run systems above lets say  5-6TB will probably move to a raid level 5 or 6 of some sort with dedicated controllers. Lets be serious here, if you are running 24TB of storage you have to have a seperate controller and therefore can run raid most likely. I can't think of any motherboard with 12 SATA connections, can anyone else. If you are running raid 5 and have a drive failure you just replace the drive and there is no data loss at all. As much as I wouldn't want DE to loose the flexability I have to realize that Raid with many disks will offset the potential issues with DE in vail. The problem there is if the raid controller has issues, but that would be just like loosing access to a vail backup system for recovery as well.

    Ken,

    I just wanted to say keep up the good work but be careful about dissmissing people as a minority. Though i don't consider myself a typical user. I suspect that i am useing the MSS for media  sharing is a fairly typical. If it wasn't why would MS being copying somethig HP has had on the media server for a long time. Though the MS solution is much better. The WHS and espcially the HP MSS has a huge user base that are focused on media streaming and sharing so don't be so quick to dismiss. A few visits to the WHS forums away from MS will show that.

    I do think though that we are kinda of at a crossroads with how storage is used in the home. It won't be long till every home has TB's on a central server and the dumb terminials throughout the house.

    With all that being said. I don't forsee a huge dealbreaker for me. Just an adjustment as my needs grow.

     

    Mavrrick

    Wednesday, September 29, 2010 7:00 PM
  • I rarely post but thought it was time to throw in my two cents. I was a beta tester for WHS v1 now find myself doing the same for v2. I'm currently running a Pentium 4 3.2 GHz with 2 GB of RAM and eight hard drives for my WHS v1 machine. Three weeks ago I decided to jump into the fray and build a new system using Xenon 3450 processor for my WHS v2 machine. I went this route because this processor is cheaper than most Core i7 chips, allows me to build virtual machines, and is comparable in its ability for video rendering.

    Over the years the only difficulties I've had with my WHS v1 machine is poorly executed add-in programs and some hardware failures. Recovery from these issues has been more difficult than it should be. Life would've been much easier if I had duplication on all folders for recovery from hard drive failure. Also a simple method of system backup would have made recovery from bad acting add-in programs much easier.

    I guess I classify myself as a hobbyist. I applaud Microsoft's efforts to correct these shortcomings in WHS v1. The discussion here appears to the about the execution of these corrections not their necessity. If the cost of correction is more hard drive space, I am willing to pay that price which is currently about $55 per terabyte. I'm also willing to bet that we will see affordable 10 to 20 TB drives within three or four years. The real effort should be directed at the feature set that WHS v2 will have. I am much more concerned about the 8 GB memory limit, maximum of four cores, limited media streaming, and only one LAN.

    Only time will tell if Microsoft made the right choices in correcting some of these issues. I find it somewhat tedious to read these discussions with only a limited fact set into the decision-making process and no white papers to boot.

    Saturday, October 2, 2010 6:15 PM
  • All this talk of WHS being the wrong tool, doing more than intended etc is not applicable.

    There is NO difference between home and enterprise !! (in software, not hardware)

    An average digital family today can easily have multi-TB of data. Using data collected from studies done years ago, when the MSS was introduced, is flawed. Ask people how much data they want to store, esp with 3+ pc's and I guarantee its huge. And it's only going to grow. Unless the intended life of Vail is 1-2 years, they should at the minimum have planned for a 5TB dataset. In a few months 3TB drives will be common.

    Microsoft has insanely long development cycles (2-3 years) and unlike Windows 7, which was designed to take advantage of hardware which would be common at its RTM date, Vail was designed to support WHS v1 scenarios (if that) and demand 2011 class hardware.

    Besides DEv2 there is hardly anything innovative in Vail. I've proposed a number of features and each time the reply has been 'thanks for the suggestion, we'll consider it for the next version'.e.g. why was CDP not included? The tech is there - I can do it by installing Live Mesh on client pc's and WHS, and they could've built it in. Why is it not possible to select SIS for more than just backups? The answer is always the same - no one thought of it and if they did decided not to include it because it was triaged out at some meeting. I know the realities of software development, being a developer myself, but in the end as a consumer I'm disappointed.

    For all the time they've spent on this, they should've tried to deliver a product that made people happy. Instead they have to defend radical design changes of questionable value to a majority of the user base. That in a nutshell is why Google and Apple are gaining ground.

     

    Sunday, October 3, 2010 11:56 PM
  • nice, tell someone their suggestion is ____ and then ask others to vote for yours, kinda funny.
    Monday, October 4, 2010 7:16 PM
  • Well, what a read (every post)! I'm new to HomeServer and haven't even finsihed installing Vail. It seems form the posts/replies here and regarding the inclusion of Media Centre in Vail, that there is no point in Microsoft asking for feedback (excepting bug reports), they aren't going to listen.

    BACKUP + DATA RECOVERY: As so many others have already said, a single HDD failure out of a set-up of 10 will loose any large files (read video + PC backups) unless I have duplication active... and the overheads in that are ridiculously high. Essentially, I need to backup my backup. But hey!, "like it or lump it" seems to be the answer.

    MEDIA SERVER: Media Centre in-built... a simple "no brainer", after all, it's intended as a media server, it just seems that you source the media from everywhere else , so that you can then share it. Bah humbug!, we can share everything across the home network now (4x desktops + 1x laptop). True, we don't haver 24hr access, but there is zero extra outlay.

    I can't be certain,after all, the final code han't shipped, but it seems that once again Microsoft is not listening to their customers, rather, "we will tell you what you want and our way is the only way".

    BTW, who are the customers? I suspect that they are the big PC manafacturers who will supply ready-made 'HomeServer' boxes with most likely some enhancements of their own. Which is fine as far as it goes, let's face it, most people do not install and configure their own PCs (that's for the family member/friend or computer shop assistant). Right now, I really doubt if I will go ahead with a home server, the implemented file handling and the lack of Media Centre integration are for me, pretty much 'deal breakers'.

    Sunday, October 10, 2010 10:05 AM
  • mavrrick wrote:
    I can't think of any motherboard with 12 SATA connections, can anyone else.
    Gigabyte GA-P55-UD5, for one (http://www.gigabyte.com/products/product-page.aspx?pid=3159#sp), I'm sure there's others.

    But back to the topic: having installed Vail last night, spent a couple hours playing with it, and an hour reading this thread, and I have to say that I'm not enthused by Vail. I thought it would be the solution to my DE woes with WHSv1, but it seems to have solved one problem by replacing it with a worse one. DEv2 is a deal-breaker for me; I will not continue with the RC0, nor the final version whenever it's released. My WHS-promoting zeal of yesteryear has vanished.
    Wednesday, October 13, 2010 11:20 PM
  • Let's also be clear that dual disk failures will lose data on ANY storage system short of RAID 6/15 which are not in the same class economically.
    I've seen this 'no consumer solution handles dual disk failures' repeated sever times in this thread.  The Drobo FS NAS device will handle up to 2 simultaneous drive failures and it uses a parity solution instead of a duplication solution.
    Saturday, October 16, 2010 3:01 PM
  • Let's also be clear that dual disk failures will lose data on ANY storage system short of RAID 6/15 which are not in the same class economically.
    I've seen this 'no consumer solution handles dual disk failures' repeated sever times in this thread.  The Drobo FS NAS device will handle up to 2 simultaneous drive failures and it uses a parity solution instead of a duplication solution.
    The Drobo uses RAID 6. And no, Drobo isn't generally what I would consider a consumer solution; it's expensive. Consumers want everything, and they want it for pennies.
    I'm not on the WHS team, I just post a lot. :)
    Saturday, October 16, 2010 4:11 PM
    Moderator
  • The Drobo uses RAID 6. And no, Drobo isn't generally what I would consider a consumer solution; it's expensive. Consumers want everything, and they want it for pennies.

    The people complaining about the Vail overhead usually need a lot of storage, e.g. for a video library. If they are not considered "consumers" then I would agree. Otherwise, based on prices from a large Swiss store, a Drobo FS compares favorable to e.g. a HP EX490, if both are filled with disks (4*2TB in the Drobo, 3*2TB for the HP, since 1TB is pre-built). The Drobo has room for one disk more.

    Cost per TB are roughly:

    • HP: CHF 350, with protection against single disk failure (the price is "theoretical", because there is no HP EX490 with Vail available, I did assume that 3TB are usable from the 3*2 plus 1 built in).
    • Drobo FS: CHF 280 with protection against single disk failure, CHF 420 with protection against dual disk failure.

    Nowadays, CHF (Swiss franc) and USD are more or less on par.

    So, for a video library with single disk failure protection, a Drobo is actually cheaper than a system based on Vail. But: The Drobo is a single purpose device: Storage only. Vail also is a backup solution, a home web server, a music server (at least the HP version) etc.

    If Vail would only switch to a software parity based solution (Raid 6), it could also fill the need for a large storage pool. As it is today, it can't.

    Regards, Martin

    Sunday, October 17, 2010 8:06 AM
  • Drobo for storage only is probably a better solution for those with large data needs as long as you don't see needing more bays. Actually compared to a HP Mediasmart I'd say Drobo is better if you don't need the other features in WHS. But a single purpose device will always be less verstalite than an extensible solution like WHS, I just wish it hadn't been crippled by Vail for certain scenarios.

     

    Sunday, October 24, 2010 1:13 AM

  • The Drobo uses RAID 6. And no, Drobo isn't generally what I would consider a consumer solution; it's expensive. Consumers want everything, and they want it for pennies.
    I'm not on the WHS team, I just post a lot. :)
    Disagree.  Yes the Drobo is expensive, but it's absolutely a consumer solution.  The whole selling point of Drobo is simplicity: just add or remove drives whenever you want and you're covered.  No knowledge of RAID, storage, etc, needed.  Just stuff in some random drives and it does the rest.  
    Friday, October 29, 2010 7:22 PM

  • How is your rebuttal any more valid in the absence of the DE white paper?  Claiming it is far superior is just another assumption and speculation on your part.

    As for losing the majority of large files on non-duplicated files when a single drive fails, this has been proven by other testers - search for other threads on the subject - so this is not speculation. You can question the reliability of those tests, and there are different degrees of data loss for different people, but to simply state it is an assumption and speculation that it is happening with DE2 is false.

    Are you suggesting my rebuttal is not valid? I was simply responsing to the (frankly crazy) proposal that laypeople will "simply lose everything and not have a clue why where the enthusiasts will know it's due to the new DE system :)" Seriously, you don`t beleive that statement, do you? Though, I suppose you do agree if you replace "laypeople" with "people with huge movie collections".

    All I`m saying is that DEv2 is far superior to DE v1 for storing <1gb files based on its advertised features. Thats a pretty reasonable basis for the claim. Funnily enough, the dissenters store much larger files and many more of them (how many times do we need to go over this?).

    I`m not disputing the test results - although its already been pointed out that theres a pretty mixed bag of results out there. Do you know why that is? No. And neither no I. And frankly, I don`t care either. It doesn`t affect my intended usage of the software.

    These arguments have been done over enough times already - theres nothing new to add here. If you want to secure your data - duplicate it. If you cant afford the diskspace then it looks likely that DEv2 isn`t for you.

    By the way, I do still want to see chunks from the same share on the same drive (where possible). Activley spreading them all over the place appears to add no value for extra risk.


    It's hardly crazy (but if you think that makes your argument seem stronger have at it.)   The fact is that if a user is storing videos in HD, as MANY WHS users will be doing imo, then they are at serious risk of losing those videos if they turn off duplication and lose one drive out of a set.   Period end of story.  White paper not needed.  If you chose to ignore the obvious in order to maintain a position then I'd suggest your other previous personal attack would more aptly apply to yourself.

    Then there's the smart design side of things.  Guess what, not everyone that has a WHS will be a savvy techie.  This whole thread would lose some of them.   I think it's irresponsible to expect those users to inherently understand these obvious risk factors "intuitively" which is exactly what will happen with Vail.  Or do you think that MS is going to popup a dialog that says "WARNING: Without duplication you incur increased risk of data lost when using files over 1gb in size under Vail".   Oh and guess what, it might even be worse if it IS the share and not the files that are chunked.

    As for your intended use of the software not being affected by these issues... that's fantastic for you and I don't care.   Of course I'd be surprised if that really is the case.    And if it's not the case, and you don't care, why are you here arguing a position on this in the first place?

    Saturday, October 30, 2010 5:11 AM
  • I would just like to note, I've done many different real world (actually doing them with Vail beta and hardware) tests on DE, and I've never lost the entire volume, even when I pulled more than one device while it was running.  I'll document the details of what I did, but I used 2TB drives and Bluray image files (ISO format from AnyDVD-HD).  It does seem to stand up good when duplication is on though, the "resync" process doesn't take that long either, faster than most RAID options for sure.

    I honestly won't be using DE for my whole server, strictly because I don't want to loose 64% of my storage to Duplication and CRC.  I don't think I would trust it without duplication, even though it doesn't kill everything, as these days loosing even one disk means 2-3TB for many of us, and that's simply too much.  There is no reason you NEED to use DE for your whole server though, I have mine setup with a RAID volume outside of DE and it has been running great.  It takes about 1minute to setup a share and permissions (unsupported of course).

    Sunday, October 31, 2010 4:08 PM
  • I just read that DE is dropped and I was shocked.  This is a killer for me I won't be upgrading or buying anything new in the WHS front.  As far as I'm concerned this is a dead OS without it.  If I have to use raid which I dont want to there is better solutions out there.  Saying use raid is not an acceptable solution.  The number one reason to use WHS in my mind is now gone.  GJ guys!
    Wednesday, November 24, 2010 10:04 PM
  • Just out of curiosity...

    Whys is "RAID not an acceptable solution"? Have you used a modern RAID controller? With a nice GUI, and a dead simple wizard? (ie: LSI). Its pretty brainless, and provides FAR better protection than the DEv2 (RAID0 or RAID1) solution, ESPECIALLY for people with large amounts of storage.

    If you just want JBOD, you can do that with any version of Windows Server. Simply add the drives as dynamic disk, and create a volume set. Bingo, instant JBOD.

     

    Thursday, November 25, 2010 5:31 AM
  • Just out of curiosity...

    Whys is "RAID not an acceptable solution"? Have you used a modern RAID controller? With a nice GUI, and a dead simple wizard? (ie: LSI). Its pretty brainless, and provides FAR better protection than the DEv2 (RAID0 or RAID1) solution, ESPECIALLY for people with large amounts of storage.

    If you just want JBOD, you can do that with any version of Windows Server. Simply add the drives as dynamic disk, and create a volume set. Bingo, instant JBOD.

     


    You have to think about most average Windows Home Server user.
    I guess many of them don't even know what is RAID....

    Thursday, November 25, 2010 7:05 AM
  • Without Drive Extender, don't call the next version of WHS Windows Home Server.  Call it something else, but don't sullly the Windows Home Server name.   

    Thursday, November 25, 2010 5:36 PM
  • They won't need to understand what RAID is. It'll simply be an OEM provided solution that stripes all their drives together. The ONLY thing thats lost is the ability to add old hard drives to the pool, but you gain so much more. Storage is cheap.
    Thursday, November 25, 2010 6:57 PM
  • And since most WHS target customers don't know or care what RAID is, how many OEMs do you think will add the cost of a second hard disk to their BoMs and their price point? Hint: ZERO. And the expansion path for these customers 18 months out when their first hard drive is filling up will be what? All that talk about RAID may have some meaning in the context of SBS. For WHS? Not so much. It's just Microsoft trying to put a positive spin on total capitulation to a business case and development failure.
    Thursday, November 25, 2010 7:15 PM
  • For those that don't know the acronym, BoM stands for Bill of Materials, and the BoM cost for a component should be multiplied by about 3 to 4 to determine the impact on retail.
    I'm not on the WHS team, I just post a lot. :)
    Friday, November 26, 2010 2:30 AM
    Moderator
  • They won't need to understand what RAID is. It'll simply be an OEM provided solution that stripes all their drives together. The ONLY thing thats lost is the ability to add old hard drives to the pool, but you gain so much more. Storage is cheap.


    So how would this work?  I buy a server with one or two drives, let's say two, so RAID is possible.  They fill up, and now it's time to add another.  What will that look like?  What manipulations will the software need to do to have RAID on three drives?  A few months later, a fourth is added, etc.  That is more or less how my WHS v1 box grew. 

    Then, if I run out of disk slots, or if I want to periodically replce drives with larger versions, how will RAID handle that?  My WHS started with a few 750 GB drives, all of which have been replaced.

    Friday, November 26, 2010 3:22 AM
  • lafos,

    The vision of RAID as a Drive Extender alternative assumes that a user will accurately predict his storage needs well into the future, and will buy enough storage today to meet those needs. I think it's an unrealistic vision, because A) consumers can't accurately predict their storage needs, and B) if they could they won't spend enough to buy all the storage they'll need X years from now. No matter how cheap storage is, if you can predict accurately that you'll need 10 TB of storage in two years, but you only need 1 TB today, you aren't going to spend $500 or $600 now for storage you won't need for years.


    I'm not on the WHS team, I just post a lot. :)
    Friday, November 26, 2010 3:56 AM
    Moderator
  • Thanks, Ken, that's what I feared.  Vail has a lot of features I really like, especially the ability to back up the system drive and keep it out of the pool, not to mention the remote access tools.  I keep important data backed up off-site, but after losing data periodically over the years, the redundancy of DE is a great feature.  It may be a forlorn hope, but I wish Microsoft reconsiders their decision.
    Friday, November 26, 2010 11:29 AM
  • Well, nobody I know outside Microsoft, including the Windows Home Server MVPs, knows what the landscape will look like without Drive Extender. Microsoft has made some extremely vague statements to us, and there are some obvious things they could do, but who knows? Until we can see a build without DE, certainly the MVPs don't have a clue.


    I'm not on the WHS team, I just post a lot. :)
    Friday, November 26, 2010 1:34 PM
    Moderator
  • What's the bet that they will partially offset the loss of DE with a move to cloud storage and the "duplication" taking place there - after all MS are making a push to the cloud and it's part of Aurora. Features such as remote access to your data, streaming it could all be centralised on the Windows Live architecture, instead of rebuilding it within WHS.  Build once, re-use many times.

    Obviously that theory leaves WHS itself as not much more than a backup tool for multiple clients and effectively a LiveMesh interface.  A lot will depend on what MS see the target market for WHS is, something I don't think has been made clear a,t least during the Vail beta...I get the distinct impression with all the talk of drive sizes in the blog posts that it is for those (the vast majority of people who have multiple client PCs?) with quite small scale storage requirements  when compared to what many enthusiasts have built up on WHS1.

    I may be totally way off mark, and happy to be proved wrong, but after a few days reflection it makes sense to me that MS can quite rightly consider DE to be unimportant if their longer term strategy is to move their perceived target consumers away from local storage to cloud storage...

    Friday, November 26, 2010 2:26 PM
  • Cloud services: Zero chance of this for consumers. Certianly not in the short or medium term, and probably not for personal data in the long term either. I don't know about your ISP, but mine caps me at 400 GB of monthly data transfer. I have about 3x that amount of data in my shares, so the cloud doesn't do me any good as far as retrieving data after a disk failure. For businesses it's a different matter; they usually don't have the same bandwidth limitations, and frequently don't have anywhere near as much data (several years of family photos and video adds up).

    And I will not trust Microsoft, or anyone else, to preserve my data as well as I will do so myself. There have been a number of cloud based backup and data storage ventures that have started up, collected many terabytes of client data, then decided they weren't ever going to make a profit and shut down with only a few weeks' notice. End result: all the data people stored on these services was lost, because they didn't have adequate time to retrieve it. There have been other ventures that have suffered various storage failures, and it turned out that their guarantees of protection for customers' data were worth the paper they were printed on. Pfft! Data gone!


    I'm not on the WHS team, I just post a lot. :)
    Friday, November 26, 2010 3:31 PM
    Moderator
  • I'm not capped, though I do pay for that privilege - and even then I wouldn't look forward to the initial data upload!!!

    I couldn't agree with you more  - on all the points, I would have the same views as you.   But as you've told me many times, I'm an enthusiast storing way more than the "average user".  Whilst I agree it's extreme, what if they're thinking of lowering that line further - going for the really average users and tieing them in to cloud storage with MS?  After all if Windows 8 will have some form of cloud backup/restore as has been rumoured, it may not be too far fetched as alot of the same constraints may be true there - ISP Caps, size of backups, confidence in host etc.

    Anyway, just a theory of what might be in their plans for the future...not one that would suit me (or you by the sound of it)...but would be targetted at a far large market than WHS potentially and could be deployed on far smaller boxes, not much bigger than USB external drives.

    Friday, November 26, 2010 4:18 PM
  • Hi !

    Cloud isn't for us... end users customers.

    First, I can't imagine my ISP giving me my 4,7Go iso DVD of films every evening as well as my WHS on my gigabits home network does !

    Second, I would never trust someone enough to secure my datas... my wife will kill me if I lose only 1 photo of our new born child ;) And I don't want to be killed by someone fault ! lol

    Third, expanding my hard drive pool will always cost less than a monthly "tax" to store the same datas...

    That's just it !


    Friday, November 26, 2010 4:27 PM
  • I agree that cloud storage is not a lkely, nor, in my case, attractive scenario.  I recently upgraded my home LAN to gigabit because 100 Mb/s was too slow.  My cable company gives me 7 Mb/s down and 0.5 Mb/s up.  Those speeds aren't practical with even modest storage.  Then taking Ken's comments about disappearing companies or unknowable costs 1-3 years down the road, makes me shudder.  On the other hand, 2TB drives are in the $80-90 range, so the cost to support DE is knowable.  My WHS is 10 TB, and all but ripped DVDs are duplicated.  Since I have the DVDs, I consider them to be my backup, and if the home gets destroyed, there's Netflix and video stores.

    Hard disks are like light bulbs.  They can last for years or die in a day.  I see DE as a badly-needed safety net for data.

    Friday, November 26, 2010 6:27 PM
  • Stephane/Lafos,  I don't want to defend my theory - I would hate it as much as you guys...but look at some of the other things MS are doing, with our useage we would no longer be targetted as end user customers (if we ever were in the first place).

    Do you really think the entertainment industry want us storing our movies locally?  AppleTV, GoogleTV, NetFlix, XBox360, Streaming Internet built in to TVs etc - don't they all offer some form of download/rental mechanism?  They want us to pay monthly subscriptions and pump protected content to us.  Charge us again and again for the same content.  Or that seems to be the way of it.  Having multi terrabyte home servers is opposed to what the industry will want - and guess what, MS will want to emulate Apple's App store.  Yes bandwidth caps would seem to be an issue, but it doesn't seem to be slowing the major players in the industry from pushing ahead in this direction.

    For people with one or two PCs in the household, a few documents, some video, music and pictures a small WHS type box for local content and client backups, with a cloud sync could very well work and may even appeal to a wider audience.  It's just not what we know as WHS.

    All I'm saying is that's maybe the direction MS will head...I hope I'm wrong...let's wait and see when these "exciting announcements" regarding WHS are made next year :)

    • Edited by PMFranklin Friday, November 26, 2010 7:10 PM further clarification
    Friday, November 26, 2010 7:06 PM
  • Exactly. For those who is cloud storage adequate, the WHS was just a wrong choice.

    Stephane/Lafos,  I don't want to defend my theory - I would hate it as much as you guys...but look at some of the other things MS are doing, with our useage we would no longer be targetted as end user customers (if we ever were in the first place).

    Do you really think the entertainment industry want us storing our movies locally?  AppleTV, GoogleTV, NetFlix, XBox360, Streaming Internet built in to TVs etc - don't they all offer some form of download/rental mechanism?  They want us to pay monthly subscriptions and pump protected content to us.  Charge us again and again for the same content.  Or that seems to be the way of it.  Having multi terrabyte home servers is opposed to what the industry will want - and guess what, MS will want to emulate Apple's App store.  Yes bandwidth caps would seem to be an issue, but it doesn't seem to be slowing the major players in the industry from pushing ahead in this direction.

    For people with one or two PCs in the household, a few documents, some video, music and pictures a small WHS type box for local content and client backups, with a cloud sync could very well work and may even appeal to a wider audience.  It's just not what we know as WHS.

    All I'm saying is that's maybe the direction MS will head...I hope I'm wrong...let's wait and see when these "exciting announcements" regarding WHS are made next year :)


    Have a nice day!
    Saturday, November 27, 2010 9:47 AM
  • Yes bandwidth caps would seem to be an issue, but it doesn't seem to be slowing the major players in the industry from pushing ahead in this direction.

    My bandwidth cap is 100GB per month.  With 11.5TB of storage space, the cloud sync will never work.  Unless MS picks up the tab for a business class internet connection with no cap @ $670/month, the move to cloud storage is the wrong direction.
    Saturday, November 27, 2010 3:08 PM
  • I don't have any cap on my upload but I just have 1 Mbit/s speed up and I do audio recordings in my spare time generating a couple of gigabytes each time. I really depend on the duplication for short term safety and occasional offsite backups for more safety, cloud storage would not help me much as it takes several hours for each file to get uploaded. The amount of data people tend to have at home grows very quickly with the new HD cameras beeing cheaper and cheaper, I have several Gigs of videos that I'd rather not loose.

    I'm on a cable company and I may upgrade to 10Mbit/s but even that speed feels a bit to slow for me uploading and are eating bandwidh for other apps I regularly use such as Skype etc.

    My guess is that there will be more people searching for a solution as good as WHS1 with DE in the future for all personal photos and videos. My MP3 collection is just a fourth of the size of my own photos, video and audio recordings.

    At work I run several large Oracle (Sun) fileservers with zfs over a 100TB of total storage and I do see the good things it brings but as stated earlier in this thread you have to plan for your future needs which may or may not be met in the time the server is used. As for vendors adding their own solutions I really do not like the most likely scenario, cheap hardware raid. Almost every cheap HW-raid I have seen has been more liable to failures and when it happens in hardware you have often to find the exact same firmware and HW release to get it up running again, if it's located on the motherboard you are pretty much on your own. I do prefer software raid as it seems to be a bit more flexible in these occations.

    For _home_ use the DE per folder duplication and fs growth are far superior than raid. The ability as in V1 to get some of the data off the disks without having the server up running is outstanding, try that with a broken raid...

    Sunday, November 28, 2010 2:06 PM
  • I haven't been on the forums for a while since my WHS1 install is running flawlessly and I didn't like what I was seeing under the hood for the Vail release.

    I came back to "check in" just to see if some of my concerns were addressed and DE has been completely removed?   UNREAL.

    Microsoft's reasoning is "um yeah hard drives are really big now, you don't need DE anymore".... 

    I can understand some feature deprecation when moving to a new release\OS but the removal of DE is just a stunning event.

    I guess the next best alternative to WHS1 is the DROBO.    Yes it's more expensive but it seems to be the next best option to WHS1 moving forward.     I can keep throwing 2TB drives at my V1 but when prices come down If I want to use the larger 3,4 or 5TB drives I suspect WHS1 will have a problem with those disks?

    I can understand long term that if 10TB drives are $200 each the need for a storage pool will be severely diminished but not having to deal with drive letters and having automatic data duplication is a beautiful thing.   

    UNGH.

     

     

     

     

     

     

     

     

    Sunday, November 28, 2010 2:37 PM
  • Just a reflection over ISP's having caps, monthly limits on customers Internet traffic.

    Is this a North American way of business?
    Never heard of it here in Scandinavia, at least not regarding fixed household connections.

    On cell phones and similar WiFi-subscriptons, yes. But not on your normal "wire to the world from your house" type of thing.


    One WHS v1 machine in the basement with a mixed setup of harddrives in and outside the storage pool. And now, next to it, a Vail Refresh brother for beta duties.
    Sunday, November 28, 2010 10:30 PM
  • <non-constructive criticism>

    I have to agree with everybody in the "WTH" camp.  I have been using WHS1 since its first beta.  I anxiously awaited RTM and built my first box only to be blindsided by the nasty corruption bug.  Only some corrupt MP3s, re-ripped so not a huge deal to me.  Now FFW to PP3 and I love WHS, recommend it to all my friends.  Survived server crashes, failed power supplies, bad mobos, all recovered in one way or another with no zero data loss.  So then I downloaded and installed Vail the day the beta is available.  Looks great, looking forward to actually being able to restore my x64 machines without having to install a second nic.

    Then I head to this forum.  Read a few threads and slowly but surely lose all faith in this product.  Overhead for data protection is expected.  50% is expected in a "RAID1" environment.  That's an easy pill to take.  Now it's going to be closer to 70%!  Really?  Seriously? That's the best you can do?

    Then the next bomb hits and all I can think is "Are you freakin' high?"  Who the heck makes the decisions around there?  As others, I have multiple TBs of non-duplicated WHS1 data.  Lose one drive, lose all the data on that drive and it's an acceptable risk to me.  Now with Vail if I lose a single drive it's almost a certainty (higher than 10% is a certainty!) that I will lose more than just the data on that drive!?  Pass me whatever you're hittin'!

    </non-constructive criticism>

    I am married to Microsoft.  I have worked in IT for 10 years.  I have a 2000 and 2003 MCSE, 2008 MCITS, Exchange 2007 MCITP.  I owe my livelihood to MS.  Apple is the devil.  Linux is fun for someone like me but I could never recommend freenas or openfiler to somebody who isn't a geek.  This may be an early beta for Vail but given these two "features" I will never "upgrade" to a Vail WHS.  I feel like my spouse just cheated on me.

     


    +1!!!
    Art Zasadny
    Monday, December 20, 2010 5:29 PM
  • Just a reflection over ISP's having caps, monthly limits on customers Internet traffic.

    Is this a North American way of business?
    Never heard of it here in Scandinavia, at least not regarding fixed household connections.

    On cell phones and similar WiFi-subscriptons, yes. But not on your normal "wire to the world from your house" type of thing.


    One WHS v1 machine in the basement with a mixed setup of harddrives in and outside the storage pool. And now, next to it, a Vail Refresh brother for beta duties.

    While we in the US build a military, Scandinavia builds a high speed ubuiquitous computing infrastructure. 
    Tuesday, December 21, 2010 12:37 AM
  • 2) Technically speaking, NTFS locks the user in MS as well :) We understand the importance of the described scenario and are thinking about ways to make it easier. That said, similar technologies (e.g. BitLocker and all kinds of RAID) often don't allow seamless mounting on other platforms but customers are actively using and are reasonably happy with them.

    Speaking as a non-technical user, I put up with WSH's vagaries because of two unique (?) features.

    One is the ability to just keep adding drives of different sizes/makes/models: no hardware compatibility issues, no capacity compatibility restrictions... just add the drives.

    The other is that if the whole thing goes South, I can recover data from the drives just by mounting them on another machine.

    Without those, I might as well go to a smaller, simpler (on the outside, at least), and more trouble-free NAS box.


    -- PeteCresswell
    Friday, February 4, 2011 8:33 PM
  • I agree completely with PeteCress.

    For the record, I AM a Microsoft employee in DPE, and in my opinion the decision to drop DE is the most stupifying one that I have seen to date during my tenure at Microsoft.  Personally, this is the number one reason I became interested in WHS1 in the first place.

    From a discussion thread today with some of my technical compatriots:

    "No Drive Extender = Will never be installed

     Seriously, for a 3-4 drive setup like I’m looking at, I’d need a dependable RAID controller that would cost nearly as much as the core machine.  No thanks.  That’s not even considering the difficulty in adding more drives.  I can’t believe that anyone in the product group found this to be remotely acceptable.  My prediction is that, unless this changes, this will serve as the death knell for what is otherwise a really, really cool product.   Very sad.

    ____

     

    The “official response” as to why this was dropped = “well, with the size of hard drives now, it’s not that important of a feature.”

     

    Well, with the amount of media we all now have = proportionate with HD sizes…it still is relevant…not to mention the issue of “replacing” smaller drive with larger.

     

    Agreed…lame.

    _____

     

    Yeah, if that’s the official response, I will go so far as to suggest that it’s a stupid one.  The size of hard drive makes it all that much more relevant – there is much more to back up, and therefore having an easy way to have your data stored redundantly across multiple drives is key.  Not to mention the fact that since big hard drives are cheap, the chances of wanting to add one mid-stream go way up.  Any barrier to doing that reduces the usefulness of the Home Server product.  The point of this software is to help you consolidate and manage all of your content.  Backup/redundancy/expansion is part of that management process…so if that functionality doesn’t exist, then it’s not really helping manage content, and thus, in my opinion at least, it fails to fulfill its mission.

     

    Having to establish a set hardware environment that is not flexible is the mindset of over a decade ago.  1999 called, and they want their technology back.

    ______

     

    http://www.drobo.com :)

     

    ______

     

    Don’t get me wrong, Drobo is cool.  But, I shouldn’t be required to purchase a 3rd party product to make this work.  The whole point of home server is a solution-in-a-box.  I want to be able to get out the DVD (or stick drive, or whatever), install it, and have that be it.  If I want to add more internal drives, then cool.  If I want to hang a Drobo on there, then cool.  If I want to plug in an extra USB 3.0 enclosure, then cool.  I want us to live up to our being the company of “choice.”  If Drobo is my only option, that’s not much of a choice.  Just sayin’.  ;-)

    ______

     

    FWIW, I have this case laying around for building out this machine:  http://www.newegg.com/Product/Product.aspx?Item=N82E16811133132

     

    It’s discontinued, but a great case.  Aluminum for great heat dissipation, single-thumbscrew drive cage (for easy removal of drives if we need to evacuate for a hurricane), and, most importantly, the 3.5” drive cage itself holds five drives.  That’s not taking into account 5.25” bays that I could convert, or double-up 7200 RPM 2.5” drives into the 3.5” bays.  My point here is that I don’t need an external enclosure, even if I want a virtual mountain of drives.  And thus, I don’t feel like I should have to go buy an external enclosure to get it done.

     

    ______’s suggestion of the Rosewill RAID enclosure is a pretty good one for the average Joe on the street.  My only question is what’s the reliability of the Rosewill RAID controller?  Because a really good usually starts about $50 higher for just a card.  If you’re striping data and the controller dies…well, it can mean hasta la bye-bye for everything on there.  Most dedicated 4-port controllers are in the $250-$300 range, and it goes up from there for 5-8 port ones.  So, maybe I just have trust issues, but it seems like a stretch for me that the Rosewill is going to deliver that level of quality at that price.  It could just be me, but I think my doubt is justified.

     

    Again, it just brings me back to the fact that I have a mobo that has 8 drive controllers built into it, and with Drive Extender, Home Server (the first edition, at least) can manage all of those drives and ensure that my data is on at least two of them.  No additional cards needed and no external stuff hanging off of the main box.

    ______

     

    I’m with ya, it’s a bit odd to see this feature go away, yet the product itself remains, drive extender was really the big value prop for me.

    ______

     

    Anyone know WHY it was removed?  Seems to me like a deal breaker for most folks (including me)…"

     

    _____________________________________

     

    So, you can see that even internal Microsoft people are completely confused by this decision, which doesn't seem to make a single iota of sense.  I know that there isn't anything that Vail can offer me, short of having Angelina Jolie personally show up at my house and dance for my entertainment, that would make me install WHS2 with DE in there.

     

    I'll be at TechReady 12.  Seriously, if anyone from the WHS product team can shed some insight into the issue, I'd love to hear about it over a beer or three.  Because right now, it seems like we didn't listen to our customer or employee input regarding this product at all.

     

     

    Saturday, February 5, 2011 12:21 AM
  • It is grossly apparent that DE was removed due to incompatibilities with LOB (Line of Business) Applications.  When DE was included in one of the many Small Business 2011 editions, it did not offer the performance or reliability that businesses were use to with a dedicated RAID controller and thus was on the chopping block.  So I will spell it out.

     

    Drive Extender was removed from the product known as "Windows (HOME) Server 2011" due to its inability to be an effective "data storage solution for line of business applications". This decision had zero regard for its intended purpose to allow "home" users to simplify their storage needs.

    Neither this testing group, or the original focus group (home users) were consulted over the removal of this feature.

    Saturday, February 5, 2011 4:59 AM