locked
FUD about RAID RRS feed

  • General discussion

  • First let me say I would like to know who writes this stuff.  I read the white paper. 

    http://download.microsoft.com/download/2/F/C/2FC09C20-587F-4F16-AA33-C6C4C75FB3DD/Windows_Home_Server_Drive_Extender.pdf

    The stuff about hardware raid simply displays immense ignorance about how RAID actually works, unless the writer was referring to "Raid 0" which is not raid at all, in the same manner that JBOD is not RAID.

    http://en.wikipedia.org/wiki/RAID
    http://en.wikipedia.org/wiki/Standard_RAID_levels


    DISCLAIMERS:
    RAID IS NOT SUPPORTED BY MICROSOFT.
    IT IS NOT FOR GRANDMA (OR THE GENERAL NON-TECHINICAL PUBLIC)
    MS WILL NOT TALK TO YOU IF YOU HAVE PROBLEMS.
    GRANDMA WILL SCREW THINGS UP SO DO NOT BUILD A RAID SYSTEM FOR HER.


    and, oh yea:


    THERE ARE ADDITIONAL COSTS.


    My apologies but I added these disclaimers to this post in an attempt to regain control of my thread.


    Now, down to business...


    In the case of Raid 1 it is a simple "mirror" where exactly the same information is written to each disk, but it is done below the volume level.  If a drive fails, the alternate drive is used to supply the exact same information to the OS that is on the disk that failed.

    ANY raid 3 or greater writes data in stripes across the disks with "parity" information alternating in a round robbin fashion across the disks.  This means that if any single disk is lost, the parity information on the remaining disks absolutely, 100% ensures that the missing data can be recovered.  In this case the RAID software assembles pieces of physical disk surface and presents those pieces to the OS as a logical disk surface.

    To the OS a raid "volume" is exactly that, a LOGICAL formatted partition which presents to the OS a LOGICAL surface to be written to, divided into 4K or larger chunks.  The physical surface of the disk is NEVER presented to the OS at any time!!!

    It plainly states in the white paper that WHS writes to the volume, that the 4K chunk is the smallest storage location available etc.

    Yes, the replacement of a bad raid disk MAY BE slightly more difficult but only slightly.  In either case (RAID or WHS disk) the bad disk must be identified.  All raid systems have software dedicated to managing the RAID system and that software operates BELOW the volume level, not at the volume level.  I.e. if a raid has to be rebuilt, you replace the drive and the raid controller takes over and rebuilds the raid array PHYSICAL data storage.  It does the rebuild completely automatically, and it does NOT mess around with the LOGICAL volume per se, only with the MAPPING of logical to physical storage.  And I can tell you, without a doubt in my mind, that the rebuild of the raid array will take a miniscule fraction of the time that the "rebuild" of the newly inserted disk in WHS' methodology.  I have had disk failures in my Raid 6 arrays and the rebuild takes a few hours (for a 320g drive).  Rebalancing and rebuilding the shadow files in WHS could take days.

    You can think of RAID like a parking garage where you hand your keys to an attendant.  You have no clue where the car is parked, you just store your car in an available spot.  The attendant is the RAID controller, the parking lot is the raid array, your parking spot is on the volume, which DOES MAP to the parking garage, but you do not know, nor care how it is mapped.  What FLOOR (DISK) of the garage is your car on?  Who cares?  If a floor (DISK) of the garage has to be rebuilt the cars are just moved around but you don't know nor care where they are moved to.  In a perfect analogy COPIES of PIECES of your car would be parked on every single floor with extra copies of pieces of your car scattered around but we won't go there.  Read the link if you need to understand it better.

    Once you understand how RAID REALLY works, then some interesting options arise.  For example, file duplication can just go away.  The RAID controller is responsible for ensuring data integrity and does so MUCH more efficiently that a simple "duplication" strategy, although RAID 1 is in fact a simple duplication strategy, but even in that case (assuming a hardware controller) it does so MUCH (an order of magnitude) more efficiently.  If you have the luxury of defining a fully expanded RAID array (maximum possible number of drives already in place and formatted), then a single large volume can be used for WHS.  In that case there are NEVER any tombstones!  Tombstones only exist if a file is copied off to another logical volume and since the RAID array is presented to the OS as a SINGLE large volume, tombstones disappear (never created).  Balancing goes away.  "Smallest volume" goes away.  Calculations to discover which volume to place a file on goes away.

    Now, understand that if you then present additional volumes to WHS such as through a USB drive, you immediately start creating tombstones, balancing etc. But if you only ever present WHS with a single HUGE volume, then that entire issue should just never exist at all.  Files are written onto the main (and only) partition.  There is no place to copy them off to so they just sit there.

    At least that is my take on the thing.  It is really too bad that we don't get some of the design team talking to us on these forums.  They could attest to or dispute that interpretation but without insider knowledge one has to infer how it works and my inference says that RAID as a single huge volume managed by a hardware RAID controller is the fastest possible WHS server. 

    I KNOW how RAID works, and it ain't how the white paper would lead you to believe.  I KNOW that the OS does NOT poke down to the physical drive level on a RAID array.  The raid drivers simply do not allow the OS to do so!!! 

    Given the white paper, I INFER that tombstones are never created in the presence of only one single volume.  Since there can be no copy, I INFER that all the related overhead just goes away.  Unfortunately with the apparent complete lack of understanding about RAID that the white paper author displays, how can I assume that (s)he knows any more about the internals of WHS?

    Until a design team member comes along to dispute it, that is what I am left to believe.

    BTW I will be doing some homegrown tests to see how well a single RAID large volume write works.  the raw write speed might provide additional clues as to how much overhead is being avoided.

    Saturday, January 26, 2008 3:25 AM

All replies

  • I would be interested in hearing how your tests go.  I am running a 4 drive RAID in my VISTA ultimate system.  It is setup for both data redundancy and mirroring. I it nice to know if I lose a drive, I can replace it and the system will rebuild itself. I am using the Promise TX4000.

     

    Good Luck!

    Saturday, January 26, 2008 4:00 AM
  • I'll repost my reply here, as you've reposted yours as a new thread. From what I gather, you're concerned with this section from the DE whitepaper:

     

    Code Snippet
    Volume management technologies such as RAID (where drives are combined below the file system layer) potentially result in a directory on one disk that points to a file on another disk. In this case, the file can be lost if either disk fails.

     

     

    I do think that's a simplistic way to descrive RAID0 (which is what they're talking about), but an article on DE isn't the place to talk in-depth about RAID levels.

     

    My original reply:

     

    Hi John.

     

    I'm an infrastructure engineer by day, and as such I deal with RAID on a daily basis. Obviously, you have quite a bit of experience in that area as well.

     

    Even though I understand RAID intimately (and my desktop PC has a RAID1 data volume), I prefer the way WHS handles disks for bulk storage of media.

     

    Using DE and discrete volumes as mountpoints means I can slap whatever free disks I have lying around into the server without worrying about matching the existing drives or expanding a RAID volume. I can buy new storage a single disk at a time.

     

    With DE, I also get to choose which data to protect (at the share level at least), rather than an entire volume.

     

    Yes, DE has overhead. But so do cheap RAID controllers (and 'cheap' is nice way of describing the RAID controllers bundled with consumer motherboards). Expensive RAID controllers blow the pants off DE write speed, obviously, but we're talking about cheap, bulk, consumer-grade storage.

     

    I think we need to remember who this product is aimed at; it's not us.

     

    Saturday, January 26, 2008 4:35 AM
    Moderator
  • Sam,

    >I do think that's a simplistic way to describe RAID0 (which is what they're talking about), but an article on DE isn't the place to talk in-depth about RAID levels.

    No, but they could certainly point to an article about RAID for those who want to know.  That sentence is FUD if I have ever seen it!

    >I think we need to remember who this product is aimed at; it's not us.

    Of course I understand who the product is aimed at, but I also understand that you and I and others with our skill level are using it.  A white paper implies advanced knowledge.  Is grandma going to read a whitepaper? 

    Given that, I think that white papers should treat the intended audience OF THAT PAPER with a little respect and impart useful knowledge.

    >But so do cheap RAID controllers (and 'cheap' is nice way of describing the RAID controllers bundled with consumer motherboards).

    I understand that built-in raid, to put it bluntly, pretty much sux.  I tried it ages ago and came away shaking my head, and that was just with Windows XP Pro on a fairly modern motherboard.  The write speeds were ABYSMAL (as low as 5 mbytes / sec).

    >Expensive RAID controllers blow the pants off DE write speed, obviously, but we're talking about cheap, bulk, consumer-grade storage.

    Yes, the hardware controller is "expensive", but when taken as a % of total system cost it certainly is not.  The total system involves a chassis, power supply capable of supporting multiple disks, cpu, memory, motherboard etc. not to mention the disks.  If you throw a system like that at WHS you are talking about probably $1500 EASILY, $2000 or more is more likely.  So $350 for the cheap controllers or $500 for the expensive ones  represents "only" 15-25% of the total system cost, but it potentially provides an order of magnitude better performance in reads and double or triple performance in writes. 

    No I'm not discussing grandma, but I am in fact building such a system and I see posts from others out there who are spending big bucks on their WHS system and ending up with horrible performance and complaining mightily, when a raid card might just turn their very expensive "grandma" system into a power machine.

    Now, as for preferring one over the other, I suppose it could be argued either way.  For me the costs of the WHS way is a definite drag, and yes, a hardware controller is assumed.  Once you go there though you gain an immense benefit as I have written in other places. 

    1) The cost of actual storage is close to or even slightly cheaper than the WHS method (crude mirroring).
    2) The READ speeds of hardware raid, particularly if you have 8 drives as I do is in another universe.
    3) The overhead of all the balancing, tombstones etc should just go away.
    4) The data integrity is in another universe.  You simply cannot compare the WHS method to any kind of real raid for reliability.

    One of my main concerns is RELIABLE backup.  Have you read the "responses" to the "you lose the backups if a drive fails" messages?  Everywhere I turn WHS is touted as "RELIABLE, FAST AND EASY backup".  Well... Fast and easy YES.  Reliable?  NOT!  Buried in the fine print is "if you lose the system drive you lose your backups".  Not with RAID you don't because you don't lose your system drive, at least with a simple drive failure.

    So yes, you can "slap" free drives in to the WHS way.  Getting them back out is another story from what I am reading. 

    I am simply trying to point out that RAID is nothing to be afraid of FOR US techies.  And WHS implemented on a single large raid volume may provide huge benefits for those willing to do it this way. 

    How many of us are there?   I don't know but at least feed us useful information instead of FUD.  A simple "these are the options, these are the pros and these are the cons" would go a long way towards upping their believability factor.

    Saturday, January 26, 2008 5:22 AM
  • Well, MS made the call to say that WHS is not supported on a RAID array, it's not a "these are your options" scenario. That means they're targeting a different consumer and a different level of hardware.

     

    Yes, real RAID means better speed and better reliability (thats why we use it in datacentres), no one from MS is going to argue that.

     

    For your specific technical concerns about WHS losing backups, yeah that's an issue to a point. Power Pack 1 is due out "soon" that will have the capability to backup components of WHS to external disks automagically. That functionality should have been there for launch, but it wasn't.

     

    Use RAID if you want to, just know that it's unsupported. And, in my opinion, calling the lack of explanation of RAID in the "enthusiast" section of the WHS site FUD is a bit extreme; like you say, RAID could be argued either way.

    Saturday, January 26, 2008 5:53 AM
    Moderator
  • Why don't you try to think of the Drive Extender as an intelligent RAID controller that sits above the OS?

     

    I reckon once they iron out all the bugs in the "home" market, it'll probably get implemented in enterprise products.

    Saturday, January 26, 2008 7:15 AM
  •  smadge1 wrote:
    I reckon once they iron out all the bugs in the "home" market, it'll probably get implemented in enterprise products.

     

    My feelings as well, smadge.

     

    At the enterprise level, storage is measured in dollars per GB, not cents. All is not created equal, and being able to selectively protect files on the some "volume" is priceless in my opinion.

    Saturday, January 26, 2008 7:40 AM
    Moderator
  •  smadge1 wrote:

    Why don't you try to think of the Drive Extender as an intelligent RAID controller that sits above the OS?



    Think about this...

    I transferred a single 2.79 gb file from one of my servers into the personal folder for that server on WHS in 50 seconds!!!

    That would be 58.7 GIGABYTES / SECOND "burst" speed.

    Test systems:

    1 gigabit LAN switch, cat 4 cable (short - 6 feet).

    Source computer: Dual core AMD 3800, 4 gigs DDR2 800 RAM, Windows 2003 standard, Areca 1220 controller, (8) 500g Seagate hard drives configured RAID 6.  Raid configured as several volumes, total about 3 terabytes total. Measured streaming reads of ~400 megabytes / second.

    Destination computer (WHS): Quad core AMD, 4 gigs DDR2 800 RAM, Windows Home Server OS, Areca 1220 raid controller, (8) 320 gig Seagate hard drives configured RAID 6.  Raid configured as a single volume of ~1.7 terabytes.  Measured streaming reads (measured under under Windows 2003 Standard OS) of ~400 megabytes / second.

    I then transferred a directory of 98 files, total of 31.9 gigs.

    Begin: 02:41:00
    End: 02:56:40

    15 minutes 40 seconds. 

    That is 2.03 gigs / minute, 33.9 megabytes / second, sustained over 15 minutes.

    Me thinks I prefer hardware raid over the "WHS intelligent raid controller over the OS" solution.

    Please feel free to perform a similar test with your system and post the results here.  It will be interesting to see what kinds of results various systems get.

    Time for me to go to bed now.  I have young'uns that will be waking me up in about 5 hours.
    Saturday, January 26, 2008 8:06 AM
  •  John W. Colby wrote:
    Please feel free to perform a similar test with your system and post the results here.  It will be interesting to see what kinds of results various systems get.

     

    That is, of course, a rigged test. We all know a dedicated RAID controller will outperform software RAID, let alone the folder duplication and reparse point magic that WHS does.

     

    No one is saying that the WHS storage subsystem performs faster than dedicated hardware.

     

    Bulk storage of media isn't about write speed. Sure, when you're copying your various repositories to WHS the first time, things are going to be slow. But after that, are you really copying 31.9 GB to WHS every day?

     

    If so, what you're trying to do is not what WHS was designed for. Different market, different consumer.

    Saturday, January 26, 2008 8:30 AM
    Moderator

  • While my personal opinion is that it would be great for WHS to support RAID arrays, I don't feel it quite fits in with their target market.

    John, you're using a $450-$500 hardware RAID controller with WHS.  I would hope you'd get fast transfer speeds - in fact I'd expect higher than the ones you've experienced.

    I don't see many people spending that amount of money on a decent RAID controller.  If RAID was supported I could see a lot of people using their onboard RAID controllers with software RAID with WHS and having even slower write rates than if they didn't use a RAID at all.

    In addition the amount of QA and time required to test different RAID solutions would be immense.  Especially if you checking multiple RAID cards, and how they rebuild.  Driver support for RAID cards is not always great (for Windows), and time would be spent hunting down errors that could be faults of the cards.

    You can achieve similar read/write speeds that you're seeing with single drive WHS setups.  You lose duplication and the ability to swap out drives.  However WHS takes a proactive stance on drive stability and runs check disk a few times a day (every six hours?) so it can catch failing drives early on.

    But, if you already are running a RAID system with WHS successfully what's the problem?
    Saturday, January 26, 2008 10:53 AM
  • Never mind what the technical brief says about RAID (yes, it was oversimplified and I suspect written by a non-technical "tech writer"), the main reasons that RAID is unsupported are that:
    • Consumer-grade solutions don't deliver the performance they promise.
    • Consumer-grade solutions often have drivers that aren't fully baked. In a desktop PC, that might not matter (though I would beg to differ) but a badly written driver can cripple a server, even if it's not randomly corrupting data when stressed.
    • Consumers don't know how to specify and build a RAID subsystem: they don't understand the trade-offs between performance, reliability, and flexibility, and they don't understand the difference between different classes of hard drives.
    • RAID is expensive. You need at least one extra drive per array (whatever RAID level you choose) even if we're just talking about a cheap software/firmware (=motherboard) RAID controller that you already have.
    • Windows Home Server as an operating system is designed to be embedded in a device that one might not mind having in one's family room or home theater where guests will see it, and RAID arrays require large cases just because they use multiple disks.
    • Windows Home Server is limited to a maximum of 2 TB per volume by it's use of MBR disks.
    "Consumer" above doesn't refer to the people in this forum, who are tech enthusiasts and early adopters. It refers to the primary target market for Windows Home Server, Excel jockeys who like tech tools when they "just work" but who don't want to get under the hood.

    All that said, you absolutely can build your server on RAID. I've done it, though I eventually abandoned it (not because of any of the factors above, I just wanted to see how difficult it would be and when I was done, well, I was done Smile ). Just bear in mind that Windows Home Server makes particular use of various disks. Probably RAID 1 for the system drive would be a good idea, then one or more RAID 5 arrays in the storage pool, bearing in mind the 2 TB limit for MBR disks.

    As for overhead from the Drive Extender system, it's fairly high on multi-disk systems, and non-existent on a single disk. If you're satisfied with 2 TB of storage, you could build a system on RAID 5 with 5 x .5 TB disk and you'd be good to go.
    Saturday, January 26, 2008 2:24 PM
    Moderator
  • Sam and LLiam,

    I think both of you are missing my point entirely.

    Did you read the post from the guy who was trying to store something like 15 terabytes on his WHS? 

    "Sadly it's time to say goodbye" over in the WHS Software forum.

    His biggest complaint is that he couldn't stream video to 5 computers simultaneously.  Now I can't imagine wanting to do so, but he can.  And guess what, with my raid controller card he probably could (ignoring network bottlenecks of course).

    MY POINT (read this carfefully) is that RAID is NOT HARD, it DOES WORK, and with a HARDWARE CONTROLLER is blazing fast.

    MY POINT (read this carefully) is that while the WHS position is that RAID somehow makes the system more complex, THAT IS NONSENSE (for the readers of this forum).

    MY POINT (read this carefully) is that the official line, repeated over and over is that "raid is unsupported", yes we know, (and strongly implied... probably won't work...)....  NONSENSE. 

    MY POINT (read this carefully) is that scattered throughout these forums are TONS of "can I use RAID" questions.  I know because I searched the forums trying to see whether I could do this.  The questions are always answered, as you are trying to do, with the party line "it isn't supported", which strongly implies it is a BAD idea so don't try it.  NONSENSE.  It is not a bad idea, it is a wonderful idea (for readers of these forums) and it neatly solves the problems that they are trying to solve when they ask about RAID.

    Guys, I am not disparaging WHS as it ships, no raid, working exactly as it works.  I think it is a NEAT solution for Grandma.  But if you are HERE in this forum, it is highly unlikely that you are grandma.  And if you are reading THIS THREAD then you must have an interest in whether RAID really does work.  It REALLY DOES WORK!

    It is TRIVIAL to set up.  Do you have any idea what was required to set it up?  A floppy disk with the drivers on it!  At one point it stopped dead and told me it couldn't find the hard disk.  I had to use a little "explorer" like dialog to navigate to my a: drive and select the OEM file from the floppy and it took off and did a bunch of stuff.  At some point it rebooted, coming up to the BLUE DOS screens where it transfers files.  Look carefully at the beginning of that and it tells you to hit F6 if you need special drivers.  I hit F6 and a few seconds later I had to select the (only) choice from a one item menu.  Windows took off and finished install. 

    How tough is that?

    Guys, I UNDERSTAND that RAID is unsupported.  That translates to "don't call us if it doesn't work", but it does work.  And this is a custom install anyway, custom hardware with an OEM license.  Who am I goiing to call in any event?

    Am I talking to grandma?  No, I am talking to YOU and YOU could EASILY make raid work.

    Obviously YOU (personally as individuals commenting in the thread) don't want to spend the money for a hardware controller, and that is fine.  If you are happy with your performance and the way it works, then have at it.

    NOW, I have no issue with your responding to my thread, but if you are going to do so, then respond to MY POINTS.  You are just spouting the official position!  We all KNOW the official position, I know it, you know it, everyone knows it.  You do not need to repeat it here.

    WHS is a very cool system.  Raid puts the finishing touches on it and makes it PERFORM the way that all the people out there moaning about performance WANT it to perform.

    And NO, this was NOT a rigged test, it was just a test, read world example of how people use WHS.  It represents how I PERSONALLY will use my system, which is why I built the system the way I did.

    I absolutely encourage you to perform similar tests, and to post your results here in this thread.  If you do so please take the time to describe your network, and the two machines in detail so that anyone reading your tests can get a sense of how your WHS solution performs relative to what they have and relative to what I have described.

    Let's talk about how RAID can be done, and why you might want to do it.  Let's get some performance numbers from various systems so we can see metrics to get a feel for what various solutions bring to the table.


    **BTW, I will BUILD you (or your grandma) a RAID system just like mine for a fee, with WHS installed and running!  Turn it on and go.  I build all my systems from parts purchased from Newegg.  All the parts have warrenty.  You would be responsible for getting it fixed if it breaks just as I am responsible for fixing mine if it breaks.  I am not HP, but I can build you a SCREAMER WHS system.

    ****BTW2. I am a professional consultant and developer of database systems, not a box assembler.  To see who I am go to: www.colbyconsulting.com

    I have a strong background in electronics, having built all but two of my systems, starting with computer kits in the 1980s.  I am obviously an enthusiast.  From my readings in these forums, there are a lot more enthusiasts in here.  I am addressing you! 
    Saturday, January 26, 2008 2:35 PM
  •  Ken Warren wrote:
    All that said, you absolutely can build your server on RAID. I've done it, though I eventually abandoned it (not because of any of the factors above, I just wanted to see how difficult it would be and when I was done, well, I was done Smile ). Just bear in mind that Windows Home Server makes particular use of various disks. Probably RAID 1 for the system drive would be a good idea, then one or more RAID 5 arrays in the storage pool, bearing in mind the 2 TB limit for MBR disks.

    As for overhead from the Drive Extender system, it's fairly high on multi-disk systems, and non-existent on a single disk. If you're satisfied with 2 TB of storage, you could build a system on RAID 5 with 5 x .5 TB disk and you'd be good to go.


    Ken,

    Thanks for the validation that RAID does work.  Now I can tell people to see YOU if they have problems. 

    BTW I found this when trying to research the 2 terabyte MBR disk size limit:

    It is important that you use Service Pack 1 for Windows 2003 Server as that is what is required to get the >2TB array. Remember the old MBR table we have forgotten about? That has a 2TB limit to it so Windows supports something called Gui Partition Table ( http://en.wikipedia.org/wiki/GUID_Partition_Table ). This is a new partition type that allows much larger single partitions.

    So again, you can do this in 32-bit 2003 Server but you need SP1 to do it or 64-bit XP [not 32-bit XP].


    This post was from http://forums.nvidia.com/lofiversion/index.php?t18158.html

    All the way at the bottom.  In this post he discussed building a single volume 2.2 terabyte RAID array.  He was NOT discussing WHS so it would be important to actually try building such a thing.

    As for your point re using a raid 1 boot disk, I would NOT do that for the simple reason that you end up back at the "multi-disk WHS" solution with all the overhead of tombstones, balancing and all that.  Of course everyone has their own requirements.  If you want to be able to add disks over time, and are willing to put up with the overhead of the multi-disk solution then that is certainly a valid option.

    For those that don't know, the better RAID controllers allow you to seamlessly migrate raid configurations.  So you could start at raid 1 (two disk mirror) and later throw two more disks in for a raid 5, then later throw a few more disks in and even bump it up to raid 6.  ALL done "behind the scenes" by the raid controller, transparrently to the OS, live, while the system is in use.  Modern RAID is a fantastic solution to a very real problem. 
    Saturday, January 26, 2008 3:10 PM
  • Sam wrote:


    "Well, MS made the call to say that WHS is not supported on a RAID array,"

     

    I still haven't seen proof of that.  I've seen them say it's "highly recommended not to use RAID".  But that's different than not supported.   The statement says nothing about Drive Extender in particular, but that is what they are talking about.  If you read the DE whitepaper the cautionary note is pretty clear about that.

     

    As we all know, there is a big difference between RAID-1 and RAID-5/6.  The first technology fits well with WHS DE.  The second one not at all.  When talking in the context of DE, MS has in their whitepaper been specific in talking about *arrays*.  This is 5/6.  Not 1.

     

    Certainly running DE on top of RAID-5/6 is plain silly.  Whether MS supports it or not is unimportant, because anybody who does this has issues to begin with.  I know as an OEM system builder, I will not be providing this conifguration, and if my customers reconfigure to do this, they will not be supported.  Hopefully with an ounce of education, this will never happen.  I firmly believe the reason MS has been vocal about this is they REALLY want to make sure nobody does such a stupid thing.  In order to be clear, you need to keep the message short, and simple.  Getting into details about various RAID levels is not a short and simple message.  However, to give them credit, the whitepaper does specifically mention "RAID arrays".  This is not RAID 1.  It is not a RAID array.

     

    Now let's talk about when RAID arrays may make sense for WHS in general, not DE specifically.

     

    If you have a Windows-only network (90%+ of my customers), then you could really care less about samba shares.  As such, exposing a RAID-5/6 array as a standard network share on a WHS platform may make perfect, logical sense, and a sound alternative or addition to DE.  If you as the OWNER and PURCHASER of your WHS are willing and capable of managing the RAID-5/6 array independently of DE, then do it.  And if MS won't "support" that, then it is absolutely for Marketing reasons, not Technical reasons, and therefore, inappropriate.


    It would be nice for MS to come out once and for all and clearly state that DE component only is not supported on RAID arrays only.  That is what I infer from reading the white paper.  That is what technically makes sense.  There is no reason to not support people placing storage devices outside of the DE management pool of disks, and then exposing them via network shares.  This can be done out of the box in WHS with zero extra tools or software.  It may not be encouraged, but it absolutely should be supported.

     

    I for one will continue offering my classic product which leverages RAID-5/6 controller, but will be changing platforms from XP Pro to WHS.  I will also offer the option for DE configuration for lower-end systems, targetting those who simply do not want to purchase all of the drives up front.  My current option of providing RAID-1 for the boot disk will be valid for both RAID-6 and DE configurated systems, and will be *HIGHLY* recommended for DE systems!!!

     

    On a related note, there's been a lot of talk about performance in this thread wrt RAID vs. DE.  For one class of my customers, that's trivial as DE has plenty of performance for their needs.  The far larger issue is wasted space.  Assuming full duplication, DE is essentially a 1:1 or mirror technology.  Well, how many of my 10TB BigBoy customers are going to want to lose 5TB to redundancy with DE, when they lose a small fraction of that with RAID 5/6?  These are customers who do not plan to "grow" their data farm slowly over time.  They are moving 1000+ DVD ISO's onto farms *now* (note: I do not provide a way to do this, that would be illegal. ;-).  As such, they will be buying their disks up front.  So why would they want to use DE when they lose 3TB, it is slower, is less fault tolerant (can't handle 2 drive failures), and takes longer to recover from disasters?  They wouldn't.  So I will offer both. 

     

    On the other hand, the customer who only needs 1TB now and doesn't see their needs expanding beyond 3 or 4 TB total, DE makes far more sense.  If you look a tthe world as a whole, clearly this market is the vast majority.  If you look at my target market as a whole, it's 30%....tops, across both my automation controller and media server lines.  If you look at just the media server line, it's down to about 10%. 

     

    One of the things MS excels at is market analysis, and I feel they've hit the nail on the head here with WHS and DE.  But that doesn't and shouldn't preclude that targeted vision being the only way the server can be deployed and supported.  The way that small OEM's and system builders differentiate themslves from the big guys is by adding value to the platform and targetting niche markets.  That by definition means pulling away from the core that MS targeted.  If we didn't, there would be no reason for people to not just go buy a HP MSS or similar, because they'll be cheaper as the small OEM can't compete based on volume.

     

    - Ryan
    Saturday, January 26, 2008 5:11 PM
  • John,

    Re RAID: "RAID is unsupported for Windows Home Server." If you want to try it, you're on your own, and if your controller eats your data next week, umm, sorry about that, and I hope it was tasty data. Smile

    Re MBR vs. GPT: When you install Windows Home Server, it formats all disks as MBR disks. Some attempts have been made to convert them to GPT, with, IIRC, not a lot of success overall. It can be done, but the hoops you have to jump through make rebuilding the system in the event of a failure requiring reinstallaiton (and it can happen even with a reliable disk configuration) a severe exercise in frustration. So you're effectively limited to MBR disks no matter what.

    For the folks watching from the peanut gallery: When you start thinking about whether you're going to try RAID, or a GPT conversion (which can be done), or extending your C: partition, you need to think not so much about whether it can be done at all (because the answer is almost always that it can), but rather whether it will leave your server in a state from which you can't recover should you suffer a disk or operating system failure that requires you to reinstall. Taking the GPT conversion as an example, I haven't tried this, but suppose you converted your system disk/array to GPT, and then something corrupts your OS. When you reinstall, will setup recognize that you have a pre-existing WHS installation and let you do a server reinstallation? Or will it see that the "disk" is formatted in an unfamiliar fashion and only let you do a new installation?
    Saturday, January 26, 2008 5:20 PM
    Moderator
  • Ryan,

    Re RAID support: See this thread, in particular the post by Joel Burt who is on the team. A year ago the WHS team stated that RAID wasn't supported. I have checked with the WHS team, and that is still their position.

    If you want to sell your clients a system with RAID, go ahead, but you're not going to be able to call Microsoft for (paid) support. Windows Server 2003 would be a better solution for the vast majority of your customers, since you feel that RAID and support from Microsoft are both important.
    Saturday, January 26, 2008 5:30 PM
    Moderator


  • Hehe, do we qualify for support if we've bought the OEM version?

    I guess I'm missing John's actual point. Smile  Is it that Microsoft should go ahead and say that RAID is ok?

    As was stated, a big benefit of RAID is concurrent usage wouldn't drag performance down into the dumps.  As it is now, two or more people writing data at the same time brings abysmal performance.  Although I still haven't seen anyone posting numbers of multiple users accessing the same WHS with a RAID versus non RAID - I can only guess that it would be faster with RAID.

    Saturday, January 26, 2008 6:48 PM
  •  John W. Colby wrote:


    (1) MY POINT (read this carfefully) is that RAID is NOT HARD, it DOES WORK, and with a HARDWARE CONTROLLER is blazing fast.

    (2) MY POINT (read this carefully) is that while the WHS position is that RAID somehow makes the system more complex, THAT IS NONSENSE (for the readers of this forum).

    (3) MY POINT (read this carefully) is that the official line, repeated over and over is that "raid is unsupported", yes we know, (and strongly implied... probably won't work...)....  NONSENSE. 

    (4) MY POINT (read this carefully) is that scattered throughout these forums are TONS of "can I use RAID" questions.  I know because I searched the forums trying to see whether I could do this.  The questions are always answered, as you are trying to do, with the party line "it isn't supported", which strongly implies it is a BAD idea so don't try it.  NONSENSE.  It is not a bad idea, it is a wonderful idea (for readers of these forums) and it neatly solves the problems that they are trying to solve when they ask about RAID.

    Obviously YOU (personally as individuals commenting in the thread) don't want to spend the money for a hardware controller, and that is fine.  If you are happy with your performance and the way it works, then have at it.

    NOW, I have no issue with your responding to my thread, but if you are going to do so, then respond to MY POINTS.  You are just spouting the official position!  We all KNOW the official position, I know it, you know it, everyone knows it.  You do not need to repeat it here.

    (5) Let's talk about how RAID can be done, and why you might want to do it.  Let's get some performance numbers from various systems so we can see metrics to get a feel for what various solutions bring to the table.

     

    I'm not missing your points John, I'm disagreeing with the way you're pushing the line that MS is somehow deliberately obfuscating RAID for consumers in WHS. Having, many years ago, actually run the desktop support escalation team (if we're throwing credentials around) for MS (Win95/98/ME/XP) I know the customer they're targeting. Intimately.

     

    They are scared, they are confused, and they're like that 99% of the time when dealing with anything PC related besides browsing AOL.

     

    They should not be making any decisions about storage, because if it's confusing and they do something incorrectly that causes data loss they'll be blaming MS, regardless of who built the box.

     

    There's a reason that there is an "official position". The reason is that it makes sense for the market segment that MS is targeting.

     

    Now that you've tagged your points with specific scenarios ("users of this forum") I don't disagree with you. I've numberered your points for clarity.

     

    1) No argument

    2) No argument, to a point. RAID requires a lot more thought in the setup phase than throwing disks at DE. In that sense, yes you are complicating the setup.

    3) The official line is not nonsense. WHS is not supported on RAID.

    4) It's a wonderful idea for the people that need it, can understand it, and can fix it themselves.

    5) If we're talking about how "why you might need to do it" we should also talk about why you might not.

     

    I'm not disagreeing that RAID provides awesome performance. But being evangelical about it in a support forum for a consumer-grade operating system that explicitly doesn't support it should be tagged with disclaimers.

    Saturday, January 26, 2008 8:05 PM
    Moderator

  • Now that you've tagged your points with specific scenarios ("users of this forum") I don't disagree with you. I've numberered your points for clarity.

     

    1) No argument

    2) No argument, to a point. RAID requires a lot more thought in the setup phase than throwing disks at DE. In that sense, yes you are complicating the setup.

    3) The official line is not nonsense. WHS is not supported on RAID.

    4) It's a wonderful idea for the people that need it, can understand it, and can fix it themselves.

    5) If we're talking about how "why you might need to do it" we should also talk about why you might not.


    [quote/]

    1) No argument
    Thanks

    2) No argument, to a point. RAID requires a lot more thought in the setup phase than throwing disks at DE. In that sense, yes you are complicating the setup.

    ROTFL.  Of course it does.  There is a HUGE difference between paying HP $600 for a system and plugging it in, and building a system, installing an OEM version on it.  Even if you are comfortable building a system from parts purchased at Newegg, building a RAID system is a small extra  bit of knowledge.

    OTOH, have you actually READ the postings in this forum?  I know you have because you answer them all.  And I have too.  There are about a bajillion about how this doesn't work and that doesn't work and can't get a drive back out, and can't get a drive recognized, and can't transfer a large file because the landing zone isn't big enough, and how the disks run for 48 hours straight trying to redistribute the files.  C'mon, so much for simple. 

    I tried it myself before I went with raid.  I started with "too small" a system disk (a 120g drive for cryin out loud).  I added some in (never showed up) I tried to take others out (cryptic "unable to do that john" error messages.  The shear volume of these forums speak to how "easy" it is.

    Now I have a single large drive and it just works.  FAST I might add.

    3) The official line is not nonsense. WHS is not supported on RAID.

    I did not say the official line is nonsense.  You do not pay attention.

    (3) MY POINT (read this carefully) is that the official line, repeated over and over is that "raid is unsupported", yes we know

    NOTICE that I acknowledged the official line.  Now... I will break it down to make it easier to read!

    (and strongly implied... probably won't work...)....  NONSENSE.

    It absolutely is nonsense that it probably won't work.

    4) It's a wonderful idea for the people that need it, can understand it, and can fix it themselves.

    Yep, that would be us.

    5) If we're talking about how "why you might need to do it" we should also talk about why you might not.


    No, that is not necessary.  According to my  simple search there are 1595 posts with the word RAID in them.  You can bet MONEY that they have ALL been answered with the "why not", not supported, bad idea party line.  Personally I think 1595 times about "why not" is enough, we do not need to do that again.


    I think it is obvious that the discussion of RAID makes everyone very nervous and so, since I can get no support for objectively discussing how to do it I might as well cease and desist.  Let me say that I have a system running that uses RAID 6.  It works quite well thanks.  No, in fact it works OUTSTANDINGLY well thanks!

    If anyone is actually interested in running WHS on RAID, whatever you do, do NOT try to get help here.  It is quite obvious that "it is not supported" is all you are going to get.  If you are capable of setting up raid, then just rest assured that it works fine.  Just do it.


    Sunday, January 27, 2008 1:24 AM
  • Hi John, if you'd like a response to that last post, could you fix up the formatting issues? It's unclear in its current format.

    Sunday, January 27, 2008 1:31 AM
    Moderator
  •  Lliam wrote:


    Hehe, do we qualify for support if we've bought the OEM version?

    I guess I'm missing John's actual point. Smile  Is it that Microsoft should go ahead and say that RAID is ok?

    As was stated, a big benefit of RAID is concurrent usage wouldn't drag performance down into the dumps.  As it is now, two or more people writing data at the same time brings abysmal performance.  Although I still haven't seen anyone posting numbers of multiple users accessing the same WHS with a RAID versus non RAID - I can only guess that it would be faster with RAID.



    Liam,

    In a technical forum full of techies (which this obviously is) Microsoft should contribute real knowledge.  We have a bunch of people flailing around here.  Ken admitted that he used RAID, it worked, it did exactly what RAID does, and he stopped using it "just because".  Well golly gee.

    I tried to get a discussion going of how to actually do it and I got dog piled with the party line.  Well golly gee.

    I actually did it, am using it now, it works great, is fast, stable, failure resistant, and I simply tried to start a thread to discuss that and let people know that it can be done, quite easily actually. 

    Look at this thread.  What a mess.  I asked people to post actual results and did I get any?  No, I got more "party line".

    I get the point guys.  There are just things we don't talk about and successful raid implementation is one of them.  Let's discuss all the things that aren't working so well.



    Unfortunately that means I can't contribute because my system just works. 
    Sunday, January 27, 2008 1:42 AM
  •  Sam Wood wrote:

    Hi John, if you'd like a response to that last post, could you fix up the formatting issues? It's unclear in its current format.



    I don't.  A response is useless.
    Sunday, January 27, 2008 1:55 AM
  • Copy that. Good luck with your system.

     

    Sunday, January 27, 2008 1:57 AM
    Moderator
  • Thanks Ken,

     

    As an oem builder I'm used to that.  Been there done that with ramdisk's on XP Pro.  Frankly, I've never had to pay for support incidents ever anyway, as the only time I bring issues is when I know it's an OS bug, and not mine. ;-)

     

    It's really not that big of a deal anyway, as all front-line support has to funnel through me.  It's really only an issue when I can't figure something out.  In these cases it simpl ymeans I need to repro the issue on a "compliant" configuration first before contacting.  Part of diagnosing the problem is simplifying the test case anyway.  So it's not a big deal.

     

    Ryan

    Sunday, January 27, 2008 2:39 AM
  • Hey John,


    In the spirit of getting the thread on topic, I'm actually very curious about how you have integrated RAID onto WHS.  While clearly not supported, I'm wondering just how far you went. 

     

    My plans are to only use RAID 1 mirroring on HD0; that's it.  The goal is purely to make it more robust to disk crashes, and in the event there is a crash, no down-time.  I do not like the idea that during a crash of the C:/D: drive, not only does the system go down, but if you lose another during either reinstall or D: rebuild (which could take a VERY long time), you lose data.

     

    What I also haven't started hacking away at yet is to determine if I can put in a RAID 5/6 array and expose it, with full functionality of shared folders, etc.; without DE touching it.  Clearly it would not be supported, and I haven't found a native way to do this, but at the end of the day the code is there.  If it's doable, some tweaker will ultimately find a way to do it.

     

    - Ryan
    Sunday, January 27, 2008 3:07 AM
  • Hi Ryan,

     

    The most likely scenario would be to use a hardware RAID controller. You could then:

     

    a) dedicate two disks to a RAID1 volume and the rest to a RAID5/6 volume. WHS would see the volumes as "disks" and RAID1 would be your C:\ and D:\ drive, and the RAID5/6 volume would be the second "disk".

     

    or

     

    b) Depending on the RAID controller, you could build a single RAID5/6 array and break it into two volumes, one for C:\ and D:\, the other for the second "disk".

     

    Basically, the key is to get the hardware controller to present two volumes to WHS as physical "disks."

     

    Edit: Thinking about it, a single volume is probably a better way to go. You don't want to use duplication or DE in this scenario, so giving WHS a single "disk" is the better option.

     

    Sunday, January 27, 2008 3:12 AM
    Moderator
  •  ryan.rogers wrote:

    Hey John,


    In the spirit of getting the thread on topic, I'm actually very curious about how you have integrated RAID onto WHS.  

     


    Hey, how can I turn that down?

    The following is background as to why I am so adamant about RAID. 

    I am a consultant, a developer, and a one man show.  I work from a home office and I have a development laptop and two (at the moment) Windows 2003 servers.  I do business analysis, and resulting database development and application design.  I am absolutely dependent on my laptop for all of my business, and my servers which run SQL Server, multiple HUGE (100-200 gigabyte) SQL Server databases.  If any of these machines go down, I have to stop work and fix them.  If I lose the system or data disks it is catastrophic in terms of lost wages.

    Additionally my wife has a laptop and my son has a laptop (my old development laptop).  I am "tech support" for these machines, and again, if something goes wrong I spend my time fixing them.

    The single most common critical failure in any computer is the system disk failure C:.  Over the last 8 years I have had to replace several system disks as they died or were dying.

    I love WHS backup.  It is point and click, then it is automatic.  Because of the sector storage technology in WHS you really can do full backups, keep old backups around etc.  The storage overhead is extremely minimal.  However a dependable disk subsystem is even MORE critical due to that system.  If you are storing any given block on the disk once and pointing every computer backup to that one data block where the checksum and file name matches, you better be able to get at it when the time comes.


    While clearly not supported, I'm wondering just how far you went.

    So, the answer to your question is


    I first created a WHS using no raid.  It worked and I played around with it, and it worked as advertised, but one of my specific objectives is to get "one touch restore" going for all of my systems.  That WORKS BTW.  One of the very first things I did with WHS was to replace the hard drive on my Dell M90 laptop (the dev machine) when the disk started dying.  ONE year old and it was going.  Stuff happens.  Dell shipped me a new drive, I backed up the old to WHS, replaced the old drive with the new and did a restore.  1.5 hours later and I was back up and running.  BTW WHS took care of resizing my old disk from 80 gigs up to the 120 gigs on the new drive.  It just worked!


    However that was all with no raid.  It is unsettling to me that if the WHS system disk goes, the backups go.  It is unclear whether it is automatic that they do disappear, but what is VERY clear is that the folder duplication specifically DOES NOT apply to whatever is done with the backup data. 


    So next I used one of my Areca controllers to create a RAID system, using an existing server that I could take down and use for experimenting.  This system already has a pair of RAID volumes, a 200 gig volume and a 1.5 terabyte volume.  After getting the data off of the server, I just did an install right onto that.  Dead simple!  When I was done, WHS had formatted the two volumes, created a 20 gig system drive and the remaining 180 gigs on the first volume was now used as the infamous "landing zone" and the 1.5 terabytes was used as the "main storage".  Though maybe not due to the "smallest drive first" philosophy.


    The more I studied and read, the more I realized that I really wanted to just start over, have the RAID system tear down the two volumes and rebuild as a single big volume.  So I booted into the raid controller firmware (BIOS) and did just that.  Tore the volumes out and rebuilt a single RAID 6 volume.  As luck would have it, I was using (8) 320 gig drives in that server and the resulting volume (after taking two drives for "parity") left me with 1.7 terabytes, which just happens to be under the "2.0 terabyte limit". 


    I then installed WHS and that was that.


    My plans are to only use RAID 1 mirroring on HD0; that's it.  The goal is purely to make it more robust to disk crashes, and in the event there is a crash, no down-time.  I do not like the idea that during a crash of the C:/D: drive, not only does the system go down, but if you lose another during either reinstall or D: rebuild (which could take a VERY long time), you lose data.


    I hear you.  Raid one is very much doable.  Given what I am reading, I think I would go with LARGE disks for the primary volume, perhaps (2) 400-500 gig drives.  That leaves plenty of "landing zone" room, lots of tombstone space etc. 


    The nice thing about a hardware raid controller is that because of the coprocessor, the rebuild speed is very fast.  I have one 320 gig disk in this system that has dropped out of the RAID several times.  Not sure why, probably just unreliable.  I have ordered another 320g disk to replace it but in the meantime I just told the raid controller to put it back in.  It took about 1 hour to rebuild the entire array adding in the disk (going from Raid 5 back to raid 6 essentially).


    What I also haven't started hacking away at yet is to determine if I can put in a RAID 5/6 array and expose it, with full functionality of shared folders, etc.; without DE touching it.  Clearly it would not be supported, and I haven't found a native way to do this, but at the end of the day the code is there.  If it's doable, some tweaker will ultimately find a way to do it.

     

    - Ryan

    If I understand you correctly, you want to create a raid array that is not manipulated in any way by DE, no files copied onto it etc.  You would then be able to create Windows Shares and keep stuff out there without going through the WHS connector software.  I think if you tell WHS about the disk, then it will be used by DE and "pooled".  However IIRC if you add a new drive, WHS asks you if you want to add it to the pool.  If you just say no, then you have the drive available "outside of" WHS.  That is my take on the matter at any rate.


    OTOH, if you tell WHS to use the volume, then DE takes over and starts utilizing it in the storage pool.

    Whether you use a "logical" disk (RAID) or a real hardware disk is immaterial to that discussion.  A raid volume is presented to the OS as a volume, NOT a physical disk.  A volume is a logical construct, not a physical construct.  As you probably know, you can do (even inside of Windows) a JBOD where you take a bunch of physical disks, and create a logical "volume" out of it.  The logical volume is what the OS writes to.  Drivers map the OS reads and writes to sectors on specific physical disks.

    A hardware controller WITH a driver, or just a software driver without a hardware controller can be used to create raid arrays.  In the end, the physical hardware (if used) provides a disk interface (SATA or IDE depending on the interface) and MAYBE a coprocessor.  If it does provide a coprocessor, then that hardware takes over the "boolean math" intensive operations of creating the magical stuff that provides redundancy.  Not all hardware RAID controllers provide a coprocessor, in fact pretty much none of them do below about the $300 price point.  In that case the main processor chip has to do all that "boolean math" intensive computing, which is why the motherboard raid "hardware" is so slow.  the "below $300" raid controllers are really just a hardware interface to the disks, without a coprocessor, and as such are also pretty slow.


    So there ya go, the laymans guide to RAID.  I hope I have answered your questions.  RAID is definitely something that you have to wrap your mind around, but once you do, you can view it as just another volume presented to Windows, but one that is more resistant to lost data.
    Sunday, January 27, 2008 4:12 AM
  •  Sam Wood wrote:

    Hi Ryan,

     

    The most likely scenario would be to use a hardware RAID controller. You could then:

     

    a) dedicate two disks to a RAID1 volume and the rest to a RAID5/6 volume. WHS would see the volumes as "disks" and RAID1 would be your C:\ and D:\ drive, and the RAID5/6 volume would be the second "disk".

     

    or

     

    b) Depending on the RAID controller, you could build a single RAID5/6 array and break it into two volumes, one for C:\ and D:\, the other for the second "disk".

     

    Basically, the key is to get the hardware controller to present two volumes to WHS as physical "disks."

     



    Sam,

    Yes, except I would say "present two volumes to WHS as LOGICAL 'disks'."  Otherwise known as Volumes.  The OS itself NEVER sees a pysical disk.  Even in the case of a plain windows / non raid system, the physical disk is immediately partitioned into volumes by DRIVERS (not the OS), the volumes are formatted and the volumes are used as the write / read surface by the OS.  DRIVERS map the reads and writes to specific sectors on the underlying hardware.

    Which is precisely what makes the whole "raid is not supported" thing so bizarre.  A RAID driver simply replaces a SATA driver in terms of data mapping to physical hardware.  I would be astounded if in two years you don't see HP and the other biggies shipping RAID WHS systems.  It is pretty easy to do a "flashing red light on the drive means that disk needs to be replaced" thing.  Nothing complex about that.  The drivers take over and integrate the newly installed drive into the system.  Really pretty darned "grandma proof".

    Edit: Thinking about it, a single volume is probably a better way to go. You don't want to use duplication or DE in this scenario, so giving WHS a single "disk" is the better option.


    And yes, IMHO a single "volume" is a better option for several reasons as I explained in previous postings.  The biggest is that as long as you do not present additional volumes to DE and create a "pool", there is no DE overhead.  No landing zone, no tombstones, no balancing, and of course no duplication.  When a file is copied onto the system it just stays put.

    OTOH as soon as you do present another volume to WHS and tell it to add it to the pool you IMMEDIATELY start all the DE landing zone / tombstone / balancing stuff, including for the original single large volume that you so carefully created.  And you MIGHT have to start watching duplication IF you add non-RAID volumes.  If OTOH you just added disk drives to empty SATA ports on the raid controller and had the RAID drivers add the drives to the array, then you could create new raid volumes using the additional room and present those raid volumes to the DE to be added to the pool.  In which case you still do not need duplication.
    Sunday, January 27, 2008 4:40 AM
  • Yes, I put "disks" in quotes because that's how the WHS setup GUI is going to refer to them. You're correct that the more precise term is logical disk.

     

    And the 2 TB volume size is definitely something to consider as part of your RAID plan. Potentially, if you think you're going to use more than 2 TB of storage, it might be a better idea to present two logical disks in the beginning. If you have a ~2 TB D:\ volume and then add another 2 TB RAID volume, WHS is going to migrate everything off D:\ and on to the new volume anyway.

     

    That sort of operation, even without duplication, is most likely going to kill write speed despite the capability of the underlying hardware.

    Sunday, January 27, 2008 4:53 AM
    Moderator
  • John, I've appreciated reading your input on running a RAID with WHS. Now I'm going to ask some advice...

     

    What RAID card would you recommend? Right now I'm running with 2 500 GB Seagate SATA drives and really have a great deal of space left, so I don't imagine needing a great deal of more space. Cost really isn't an object (within reason). I like you, value my data and the whole purpose of the WHS (for me) is to protect my data.

     

    At this point, I'm thinking about a hardware RAID card and 2 more 500 GB Seagates and set them as a RAID 5 to give me 1.5 TB. Or would you recommend running them in a RAID 6 for the greater fault tolerance but lose the extra space (which I really don't need at this point)?

     

    Thanks for the input...

     

    Tim

    Sunday, January 27, 2008 7:21 AM
  •  

    Sam wrote:

     

    "Edit: Thinking about it, a single volume is probably a better way to go. You don't want to use duplication or DE in this scenario, so giving WHS a single "disk" is the better option."

     

    I'm right there with ya.  It's early and I may be missing something, but I just don't see the allure of running DE over a RAID5/6 array.  I am with MS on this.  Granted, I don't think it should be "unsupported", but it definately does complicate things for the average home user, even if you don't use the redundancy or expansion capabilities of DE.

     

    As a system builder, I wouldn't sell or support a system like that, becuase I would be afraid of what uninformed users would do.  Before you know it, they'll have duplicated files on top of RAID5/6 redundancy, which is not only a waste of space, a complete waste of performance.  Or worse, they won't duplicate the files b/c they know they "have RAID", but then they extend the pool with a USB drive (their backplane is full after all, ya know! ;-)

     

    DE was designed around the fact that those volumes are single, physical disks.  And while it may "work", it only works well under certain conditions, namely that WHS DE is operating in "single drive" mode and not duplicating, not balancing, etc..  You can't rely on any of the DE features.  This is why for systems I plan on offering RAID5/6 storage, I will offer that volume completely outside of the DE managed pool, and only support that configuration.

     

    The issue is the DE features are "in your face".  The RAID array is abstracted out of visibility.  Even exposing the ability to mix and match will lead  users to problems down the road.  They will add a USB drive, add it to the storage pool, and now you have a real mess, as your RAID array no longer contains all of the data.  Some real data is stored on the USB device, where it is one drive failure away from being *gone*. Why? Because the folder duplication isn't turned on; it wouldn't make sense when running DE over a RAID to use it.  Sure, you could have it off when it was just the RAID array, and turn it back on when you add the USB device, but even in this case you are back to wasted space, because you can't control which shared folder data is stored where.


    It's just easiier for MS to say "RAID is not supported", then to try to explain those nuances.  Even if you tell people, they either won't listen, won't get it, and will get themselves into trouble. 

     

    Now, for clueful individuals such as ourselves to do something like this on our own rigs when we know how to manage it, that's another story!   If you know not to use the DE expansion or duplication features b/c you have a RAID5/6 array underneath, then go for it!   If you know that when you run out of space, that slapping a USB drive in there as the WHS Gods intended is not going to work, then go for it!  You know how to manage the RAID array, so you can do this safely.   It may not be fine with MS, but as long as you are fine with it, go for it!  Just don't expect any support, either today, or down the road when you upgrade*.

     

    * Yep, that "no support" applies to WHS v2 built on 2008.  If you need to upgrade and have 5-10TB of data spread accross a RAID 5/6 array and the upgrade won't run b/c your configuration is unsupported, what will you do then?  Yeah it's unlikely, but I can't predict the future.  This is another reason why I'm not going to support this configuration for my users (DE over 5/6), but will support RAID1 on C: D: only.  An upgrade in that scenario is far, far more likely to run, and if for some god-awful reason it didn't, the work-around is trivial (reconnect single drive to non-array controller on mobo) and doesn't require me rebuilding the entire array and trying to figure out how to do that with all of that data.

     

    On the other hand, I have no problem supporting a RAID 5/6 array on WHS as long as it's not in the DE storage pol *at all*.  But I still have more research to do to see if this is plausible.  For example, if the WHS couldn't media-stream from such a volume it would be an issue.

     

    - Ryan
    Sunday, January 27, 2008 12:36 PM
  •  

    Thanks John, a great post.  It's interesting to see how things evolved for you.

     

    I concur re: raid controllers.  I don't bother with the junk.  If customers are dropping $2k to $3k on dries, it's worth spending $400 on a controller vs. $$100-200.

     

    As I said in my other post, for people like yourself who know what they are doing, I think this is a good idea.  What you are saying is you would rather manage the redundancy as opposed to DE, and given some of the issues that DE has today, that's an easy decision to make if you are going to invest in all of your storage up front.


    - Ryan

     

    Sunday, January 27, 2008 12:53 PM
  •  tneasham wrote:

    John, I've appreciated reading your input on running a RAID with WHS. Now I'm going to ask some advice...

     

    What RAID card would you recommend? Right now I'm running with 2 500 GB Seagate SATA drives and really have a great deal of space left, so I don't imagine needing a great deal of more space. Cost really isn't an object (within reason). I like you, value my data and the whole purpose of the WHS (for me) is to protect my data.

     

    At this point, I'm thinking about a hardware RAID card and 2 more 500 GB Seagates and set them as a RAID 5 to give me 1.5 TB. Or would you recommend running them in a RAID 6 for the greater fault tolerance but lose the extra space (which I really don't need at this point)?

     

    Thanks for the input...

     

    Tim



    The thing to understand in this whole conversation is that there a handful of different "problems" created by the WHS teams due to their system decisions.  Each has to be understood and considered to make sure that you have your bases covered and that you get what you want out of WHS.  Let's break it down:

    1) The system drive has no redundancy, and if it dies then the BACKUPS die.  For whatever reason, MS decided not to do redundancy with the backup files.  In my mind, that would have been the very FIRST thing to be made redundant!  However it is MS toy we have to live with their decisions.

    The SOLUTION to that issue is redundancy of the system drive, i.e. RAID.  That is a no-brainer.  Raid 1 (mirror) is fine or a RAID 5 volume on a larger RAID array is also fine.  Now if a system volume disk dies you simply take it out and throw another in.  You might have to tell the raid software to do a rebuild. 

    2) The DE coming into play creates a performance "problem".  Any time that you have a POOL of disks, DE will take over and start shuffling things around.  You have absolutely no control over the how and why, it does what it does and you sit down and shut up so to speak.  The only solution to that problem is to prevent DE from coming into play.  If you are going to play the RAID game, then the only way to prevent the DE from coming into play is to create a single large volume.  That in itself causes you to consider using RAID 5 (or 6) and not doing a RAID 1 system disk.

    3) Duplication of files is inefficient.  It is basically like using all RAID 1 volumes in a large raid system.  Raid 5 and 6 are specifically designed to minimize the impact of RAID by requiring only 1 or 2 "extra" disks to hold error correction information.  It really isn't quite as simple as I make it sound, in fact the data is striped and "rotated around" the physical disks but you don't need to know or care about that stuff.  What you need to know is that you can get more actual storage from a raid 5 than you can from a raid 1 for the same number of disks.  Storage for raid 5 is Size * (N-1) disks.  With Raid 1 it is N/2.  AFAIK you cannot do a raid 5 on just 2 disks.

    So... if you want to start with 2 disks you would start at RAID 1.  Adding a second disk would allow you to move to raid 5.  Many raid controllers will allow you to dynamically switch between RAID levels on-the-fly.  I.e. you take (2) 500g drives and create a raid 1 array and end up with 500 gigs of usable storage.  Add a third drive and tell the raid system to convert the raid 1 to raid 5 and you end up with 1 tbyte usable.  Add another drive and end up with 1.5 tbyte usable.

    It is important to understand that you will probably not be able to make your existing volume bigger, but rather have to make a new volume with the additional storage.  So in the example above, adding the next drive would add a second 500 mb volume, but both volumes would end up RAID 5.  Adding another disk would add a third 500 meg logical volume so now your system would have (3) logical volumes, 500 gigs each, RAID 5.

    To WHS this is all irrelevent.  As soon as you add the second volume, DE takes over and creates a pool out of whatever is there so to all APPEARENCES, you now have a single 1 tbyte drive or a single 1.5 tbyte drive.  And performance drops due to all the DE overhead.  HOWEVER... you STILL HAVE RAID.  Every single volume added, and all storage (so far) is RAID protected.


    Understanding this makes you decisions in the raid controller card critical.  Typically hardware controllers come with "sets" of 4 SATA (nowadays) connectors.  So you buy a 4, 8, 12, 16 or 24 port controller.  Each level up is approximately equivelent extra cost.  So if you buy a 4 port controller you are "stuck" at only 4 RAID drives.  If you buy an 8 port controller then you have up to 8 possible drives etc.  And of course the more ports, the more cost for the controller card.

    Hoewever since you can add more drives, the USABLE storage goes up in a non-linear fashion. 

    2 drives = 50% usable storage
    3 drives = 66% usable
    4 drives = 75% usable
    5 drives = 80% usable

    etc.  Basically it is N-1.  Raid 6 is N-2.  So the curve approaches but never reaches unity (100%) usable storage.

    For my purposes and budget, 8 ports was the optimum.  I figured (correctly) that at the time I was buying the drives I could just replace the drives with larger drives more cheaply and efficiently that paying more for the "bigger" controller (12 ports etc).  Remember that I started this process when the 320 drives were the sweet spot in the price / size curve and I had a limited budget.  Had I had a bigger budget AI would have gone with a 12 port card but it was ~$720 for 12 ports vs (at the time) $490 for the 8 port card.  I couldn't afford to populate even 8 ports with the 320 gig drives at that time.  I ordered 4 (I think) and then ordered more later.  I later built a second "identical" system except that I used 500 gig drives because they were at the sweet spot.

    The last thing to discuss is the u-controller part.  A micro controller will make the system anywhere from 3 to 10 times faster.  Writes are ALWAYS the killer, reads are not computationally intensive.  So a controller with a coprocessor will make the writes in particular up to 10 times faster, and will also increase read speeds.  However more importantly, the NUMBER OF DRIVES drives read speed more than any other single thing.  The reason is simple.  Each drive can only stream at a certain speed.  Whatever that speed is (let's say 25 mbytes / sec), you will (roughly) multiply your READ speed by the number of drives in the RAID Array.  Remember that the data is written across ALL of the drives.  So a single file will be written on 2, 3, 5, or more drives (depending on the number of drives in your array).  Thus all the drives contribute to the stream of data read off the raid array.  The bottle neck will eventually be the interface to the CPU (PCI card, PCIX etc) but you will not hit that bottleneck with PCIX until you get up past 8 drives.

    So which controller is better.  The simple axiom, the more you pay... does come into play.  Areca is a small company but they make killer controllers, some of the best in the business (at least that you and I can afford), but they are also the most expensive.  They are who I chose but I won't tell you to buy them.  You have to look at your budget but the BIGGEST single thing you can do is pay for a coprocessor.  Without that the speeds will suck because the computer itself will have to do all of the work.

    RAID is a journey.  It opens a new world of reliability as well as speed.  It brings its own set of complications, and it may not be worthwhile to you in the end.  It is to me, but I really just wanted to be able to present the facts so that everyone here can make an informed decision about how it will effect their own system and needs.
    Sunday, January 27, 2008 7:03 PM
  •  John W. Colby wrote:

    DE will take over and start shuffling things around.  You have absolutely no control over the how and why, it does what it does and you sit down and shut up so to speak. 

     

    ...

     

    Raid 5 and 6 are specifically designed to minimize the impact of RAID by requiring only 1 or 2 "extra" disks to hold error correction information.  It really isn't quite as simple as I make it sound, in fact the data is striped and "rotated around" the physical disks but you don't need to know or care about that stuff. 

     

    This isn't an attack John, but I have to point out the irony of those two statements Wink I agree with you entirely that everyone with the appropriate level of technical skill should be making their own, informed decision.

    Sunday, January 27, 2008 7:31 PM
    Moderator
  • LOL, SAM the difference is that DE makes a mess of things and has some horrendous consequences.  DE is not RAID, is an attempt to take the complexity out of adding a disk.  MS entire focus was taking the complexity out of things which is an admirable goal.  In the process they created some truly horrific issues which they blithly ignore.

    Raid's focus is not to take the complexity out of things but to provide redundancy.  DE doesn't hold a candle to the complexity of RAID. 

    But by and large, RAID very neatly solves problems.  In any "normal" environment it neatly solves a specific problem while creating none.  DE on the other hand very neatly solves one and only one and creates a TON of other problems, all of which are a royal PITA to straighten out.

    BTW, using RAID 1 in the system disk for WHS creates NO additional problems and solves one of WHS problem, the "there goes your backups" problem if the system disk fails.


    I don't know exactly what your problem is with my attempting to provide real, intelligent and useful information about RAID within the constraints imposed by WHS but these kinds of posts are a bit annoying.  If you would like to help out that would be wonderful.
    Sunday, January 27, 2008 9:22 PM
  • Well, you have about as much chance of convincing me that my posts are "annoying" as I do of convincing you that yours are evangelising RAID, at the expense of DE, to the point of hyperbole.

     

    If presenting the other side of the argument isn't attmpting to provide real, intelligent and useful information about RAID in WHS, I'll bow out of the discussion.

     

    As I said before, good luck with your system.

     

     

    Sunday, January 27, 2008 10:00 PM
    Moderator
  • To be fair, I don't think Microsoft is "blithly" ignoring any issues with DE.

     

    We know they are fixed (at least) one bad corruption issue during beta, and working on another now.

     

    Yeah, there may not be a person actively working on the inconsistent performance or inability to add/remove drives.  And if not, it's probably because they are all full-steam ahead on the corruption issues which are *paramount*.  But either way, we don't know.  It's not a transparent development effort.  We don't know the details of what they are and are not working on, so anything else is pure speculation.

     

    If MS wants this product to be a long-term success, it will have to address these issues.  They know this.  But that doesn't mean they will all be fixed this week.  Or even by the nexdt major release.  It may take some time.  Windows NT was not a good operating system until 3.51; some would argue 4.0.


    At the end of the day, nobody should be suprised by the bit of "spin" MS has released wrt RAID.  Getting widespread adoption of a v1 product, despite it's flaws, and then fixing them later is what MS does best.  I actually admire them for it.  I'd rather have it with flaws now, than perfect a year from now.  I can work around the flaws for the year (through use of RAID, for example. ;-).

     

    Ryan

    Sunday, January 27, 2008 11:33 PM
  •  Sam Wood wrote:

    If presenting the other side of the argument isn't attmpting to provide real, intelligent and useful information about RAID in WHS, I'll bow out of the discussion.

     

    As I said before, good luck with your system.

     

     


    Thanks, I appreciate that.
    Sunday, January 27, 2008 11:49 PM
  •  ryan.rogers wrote:

     

    <snip>As we all know, there is a big difference between RAID-1 and RAID-5/6.  The first technology fits well with WHS DE.  The second one not at all.  <snip>


    <snip>Certainly running DE on top of RAID-5/6 is plain silly. <snip>


    Ryan, I would not put it in quite those terms.  Let me throw out this scenario:


    Starting with a pair of 1 tbyte disks we set up WHS in Raid1, single volume.  Things work fine, we have 1 terabyte of RAID storage for WHS with the system disk fully RAID protected, we have no DE issues (balancing, tombstones etc).


    In a year we run out of storage.  We drop in another terabyte disk and convert to RAID 5.  We build a new logical 1 terabyte volume and present it to DE.  DE now takes over and "pools" the two drives to give an apparent 2 terabytes of storage.  Yes, we now have the DE overhead which is not good BUT we also have FULL RAID data integrity.  We do NOT need file duplication, simplifying things somewhat and getting rid of that overhead at least.


    A year later we add another terabyte drive, and so forth and so on.  We increment apparent storage a terabyte at a pop.  All storage is RAID protected.  Using an 8 port RAID controller card we could get up to 7 terabytes of actual data storage RAID 5 or 6 terabytes actual data storage RAID 6.

    That is a pretty good compromise if you ask me.  So IMHO DE and RAID 5/6 can indeed play together and still get benefits from each other.  You do end up with the balancing overhead but that is only the same overhead you would endure even without the RAID storage - if you used plain old raw disks.




    <snip>Now let's talk about when RAID arrays may make sense for WHS in general, not DE specifically.

     

    If you have a Windows-only network (90%+ of my customers), then you could really care less about samba shares.  As such, exposing a RAID-5/6 array as a standard network share on a WHS platform may make perfect, logical sense, and a sound alternative or addition to DE. 

    <snip>


    Ryan, this too is a very valid point.  It should be possible to provide volumes outside of DE if you so desire.


     

    On the other hand, the customer who only needs 1TB now and doesn't see their needs expanding beyond 3 or 4 TB total, DE makes far more sense.  If you look a tthe world as a whole, clearly this market is the vast majority.  If you look at my target market as a whole, it's 30%....tops, across both my automation controller and media server lines.  If you look at just the media server line, it's down to about 10%. 

     

    - Ryan


    And in this case, referring back to my top response, there is no technical reason that this can't be done on a RAID volume.  Just pop in another drive, create a new volume and let DE go to town.  But it is still raid protected, and since it is using RAID 5 you are getting much better storage % than the DE "file duplication".

    I contend that with a hardware raid with sufficient ports and using large hard drives, you could build out a very large, fully RAID protected, DE managed WHS storage component.
    Monday, January 28, 2008 1:38 AM
  •  

    OK, this is a more complex issue than it seems.  RAID is true and tested, DE is a brand new, V1 product, incorporated into a brand new V1 product called Windows Home Server.

     

    I have no doubt that Microsoft will be working on making a DE v2, hopefully to be incorporated with WHS v2.

     

    It'd be neat if the V2 product could also incorporate Single-Instance Storage and better redundancy for the system partition.  the reason I think a "green" product was released into the home market was so they could get this sort of feedback.

     

    I would have preferred a RAID solution, because I can see the benefits.  But I'm happy with the redundancy that DE provides (excepting of course the system disc)

     

    On my system, the speeds aren't that important.  I can see how some people would love extra speed, but it's not necessary in my setup.  I did over spec my CPU and RAM, and I'm confident my system can handle all the thrashing DE can throw at it.

     

    Now I'm waiting eagerly for the V2 product.

    Monday, January 28, 2008 5:08 AM
  •  

    John wrote:

     

    "We do NOT need file duplication, simplifying things somewhat and getting rid of that overhead at least."

     

    Understood, and as I mentioned in the other post, as long as duplication is not enabled, then it wouldn't be too bad.

     

    But my big concern with my users is they would add extsernal USB disks at some point, and when they do, what happens?

    For arguments sake, let's assume I've I've told them never to fuse folder duplication because the RAID array provides the redundancy.  Well, if they add an external USB drive, then they won't be protected.

     

    If the are "smart" enough to know the difference and turn it on, data is not being duplicated across the RAID array as well, as you have no control over where it stores the shadow copies and where it does not.  Even if you added two external drives at the same time, you wouldn't be guaranteed data wouldn't go to the array.

     

    I concur, this absolutely "could" be done.  And it would work great as long as you knew what you were doing.  This is why I've maintained that it's not quite fair that MS simply says that "it's not supported", when there should be no technical issues. 

     

    But there are concerns with doing this and mixing and matching external drives, and I do also have (albeit minor) concerns about what happens when you go to upgrade to the next major version.

     

    Ryan

     

     

    Monday, January 28, 2008 10:52 AM
  • Ryan,

    I do understand your concern.  IF you could build a system with large drives and empty slots like the HP system, and IF you could make the customer understand that there are additional empty slots and that they add drives to those slots when they need more storage, and IF you could make the customer understand they simply do not add USB drives THEN it works. 

    Can you tell I'm a programmer at heart? 

    In that case if the customer wants more storage then they pop in additional drives.  If the system is marketed as a "server" sitting in a server room, then adding usb drives is not an option.  You don't add USB drives to your current servers do you?  I.e. this is not a Windows Home Server, it is a XXX server.

    I am not advocating one way or the other, just explaining how it works.  We each have to decide what works for us.
    Monday, January 28, 2008 1:29 PM
  •  ryan.rogers wrote:
    This is why I've maintained that it's not quite fair that MS simply says that "it's not supported", when there should be no technical issues.
    I believe that there are several reasons why RAID is unsupported (which is Microsoft's apparantly final word on the subject, so if you decide to do it, you're on your own). All of these are issues for the non-technical user that is the WHS primary target market. (None of them are insurmountable for the enthusiast, but if Microsoft has to make a decision that favors one over the other, the Excel jockey will win every time. Smile ) Here are three of those reasons:

    One is a complexity issue. There are no RAID HBAs I know of (though something tells me that I probably just don't know about them, not that they don't exist) that automatically grow and change RAID level as drives are added without user interaction. In a headless storage device, which is what WHS is, that's essential. You don't want the (non-technical) end user to have to go through a long, involved process to add a drive to their server, and Microsoft undoubtedly doesn't want to be specifying RAID HBAs for WHS OEMs.

    One is a cost issue. I think we are all agreed that cheap RAID solutions are lacking in any number of areas. A good RAID HBA is $200 or more, depending on features desired, number of drives to be connected, etc. And the drives you use with it are not standard desktop drives, or at least the drives I use aren't standard desktop drives (because I don't want the drives going into an extended error correction routine and triggering a rebuild every month). WHS was intended to meet a particular price point, I'm sure. Adding several hundred dollars of additional hardware just wasn't going to make that happen.

    One is an expandability issue. The largest disk farm I'm aware of attached to a WHS PC was something like 16 TB. It was an experiment by Microsoft, but it apparantly offered adequate performance. With RAID, you are limited to the number of disks the HBA itself supports, typically 4 or 8 (more if it also supports port multipliers, but there's a performance penalty there). Once you max out the RAID HBA, you're done. And can you increase capacity in a RAID array by replacing a single drive, as you can with WHS?

    So DE is less complex (for the end user), less expensive, and more easily expanded. For the target market, it's an overall win. If you're comfortable with not being able to call Microsoft for paid (or free incidents with MSDN/Technet) support, go ahead and use RAID.

    Edit: an additional reason not to support RAID: Microsoft would like to sell Windows Server 2003 (soon to be 2008) in one flavor or another to people with more advanced needs. You give up the "ease of use" features of WHS, and gain all sorts of flexibility.
    Monday, January 28, 2008 8:24 PM
    Moderator
  • I believe that there are several reasons why RAID is unsupported (which is Microsoft's apparantly final word on the subject,...


    LOL, you've been around MS long enough to know there is no such thing as "Microsoft's final word".

    And believe I could build you a 126 terabyte RAID solution tomorrow if you are willing to pay the price. 

    http://www.areca.us/products/pcietosas1680series.htm

    And the price (in disk drives) would be about 1/2 what you would pay if you were to do the same using drive extenders and file duplication.

    None of which makes any difference.  My only intent with this thread is to provide people enough information to use raid if they want to, which I hope that I have achieved. 

    NOTICE BTW that this thread has had more views than the "please read our forum policies" thread.  That indicates a healthy interest in RAID to my way of thinking.
    Monday, January 28, 2008 8:48 PM
  • Well, yes, Microsoft changes their collective corporate mind about as often as I change my, umm, well never mind. Smile They change their mind lots.

    That said, though, the Windows Home Server team is fairly adamant about the RAID issue at this time. I wish they weren't (though not as much as you do), because RAID 6 and GPT are a better solution for data integrity than DE, if you're willing to pay the price. I really don't think they'll change their minds any time soon, though, because of the primary target market for WHS, where price is an overriding consideration.
    Tuesday, January 29, 2008 2:10 AM
    Moderator
  •  Ken Warren wrote:
    <snip> because RAID 6 and GPT are a better solution for data integrity than DE, if you're willing to pay the price.<snip>


    I just HAD to do that!!!
    Tuesday, January 29, 2008 3:59 AM
  • Ah, but you see, I've never said that RAID doesn't have advantages. It just doesn't have enough advantages for the Windows Home Server target market to be willing to pay the significant costs required to achieve those advantages.

    I happen not to be willing to pay those costs for a home server, either the monetary costs to purchase the additional hardware, or the time costs of managing that hardware myself. I do enough of that every day on the job; I don't need (or want) to do it at home too. My needs in a server for the home are modest: a couple of terabytes of protected storage will keep me for the forseeable future, and an HP MediaSmart Server is easily expanded to have a couple of TB of storage.

    For those who are willing to pay those costs, there are better solutions than Windows Home Server.
    Tuesday, January 29, 2008 5:55 AM
    Moderator
  • Well Ken, in my opinion there are costs in both scenarios.  WHS makes many things OTHER THAN just adding storage dead simple.  As long as you manage all your drives through the RAID then WHS has very real advantages in ease of use, particularly in backups.

    Now, move to "the better solutions than WHS" (I assume you are talking about plain old Server 2003?).  You STILL have to do all of the RAID stuff you do in WHS so you haven't gained any advantages there.  Now you have to go find, buy and install backup solutions on each machine.  You have to manually set up shares, get it all talking to each other etc.  You have to set up the schedule and so forth.  All that stuff is just "click, I am done" in WHS.  Not to mention the constant solicitation to buy the next version for an additional money.

    I am no expert on backup systems but I haven't seen anything out there that is as "point and click" as WHS.  AFAICT it does not provide the granularity of the other backup solutions in terms of specifying what files or directories to include / exclude but on the other hand, for the small office, a whole system backup just works.  Doing a re-install from the backups is DEAD SIMPLE, I know because I have done so.

    I have two large servers in my office (one of which is now WHS) which I created RAID arrays on, and then started doing the "manual" thing.  Creating shares, directories for each machine, going to each machine and setting up backups, setting up the schedule, and all that nonsense.  Then I never knew if a backup ran (and they don't always for various reasons).  Yea it worked and it was a PITA.  I did the "better solutions" and IMHO they were SO NOT BETTER.  Why do you think I even looked at WHS?  It was certainly NOT for the ability to add USB drives to my storage!

    As for the "additional costs and time costs", I have to tell you that I set up the raid and forgot about it.  I don't know what you do but raid is very "in the background" process.  I have been using it for about 3 years and don't "do" anything.  One of my RAID arrays has never made a peep in those three years, the other has peeped twice about one specific drive dropping offline.  Yea, there are additional monitary costs of course but a cheap card can be bought for under two hundred, that while not a coprocessor at least allows you to move the array should the server die.  At around $300 you start getting into the coprocessor assist boards. 

    So assuming WHS works as advertised I simply don't see the "better solutions".  What is it about doing all that stuff manually that attracts you to something else?  And why do YOU have WHS?  Surely you are technically adept enough to do the "better solutions"!

    Certain individuals in particular keep harping on "the intended market" and "not supported" but I am reading these forums and see a LOT of people who certainly appear to be technically adept, and TONS of inquiries about RAID.  There is another just posted today.  What I don't understand is why those of us who are interested in doing this can't just be left alone to sort out the issues without the constant posts in the middle of my thread about "intended market" and "not supported".  I made it VERY CLEAR, right up in the very first post that RAID is not for Grandma, and yet over and over there are posts about "yea but".

    Just to be clear, and so that no one feels the need to post it again, "RAID isn't for Grandma".  It is NOT supported.  There are issues that you have to understand in order to use it effectively. 

    DISCLAIMERS:
    RAID IS NOT SUPPORTED BY MICROSOFT.
    IT IS NOT FOR GRANDMA (OR THE GENERAL NON-TECHINICAL PUBLIC)
    MS WILL NOT TALK TO YOU IF YOU HAVE PROBLEMS.
    GRANDMA WILL SCREW THINGS UP SO DO NOT BUILD A RAID SYSTEM FOR HER.

    Oh yea...

    THERE ARE ADDITIONAL COSTS.

    Are we satisfied with my disclaimers?  Can I have my thread back?
    Tuesday, January 29, 2008 1:31 PM
  •  

    Okay,

     

    I've been mostly a silent observer during this and I just have to say I think it's funny that you both seemingly saying the same thing and then discussing (arguing) about it, but in a very civilized manner (which I applaud). 

     

    Boil all the posts down and you get, "RAID is better but more expensive and Microsoft doesn't support it."

     

    I really have enjoyed reading the thread.

     

    Thanks!

    Wednesday, January 30, 2008 6:54 PM
  • John,

     

    Since I'm pretty sure you are a sales rep for Areca (j/k), what do you think of the Areca 5030 unit? 

     

     

    Wednesday, January 30, 2008 7:19 PM
  •  tneasham wrote:
    I've been mostly a silent observer during this and I just have to say I think it's funny that you both seemingly saying the same thing and then discussing (arguing) about it, but in a very civilized manner (which I applaud).
    I generally refer to this as "being in violent agreement." Yes, John and I actually agree on the concepts, though we differ somewhat on the details. And yes, it's fun to watch from the outside. Smile
    Wednesday, January 30, 2008 8:13 PM
    Moderator
  •  tneasham wrote:

    John,

    Since I'm pretty sure you are a sales rep for Areca (j/k), what do you think of the Areca 5030 unit? 



    Nope, I just needed a fast and solid raid card for my business.  I researched the stuff that was available to our price range and all the reviews said that Areca was significantly faster than the competition so that is what I bought.  I own two Areca 1220 controllers and have both of them in use.  One has (8) 500 gig drives and the other has (8) 320 gig drives, both Raid 6.

    I do recommend them however I understand their tech support is bad, although I also understand that there is an american company that sells their stuff that has become their US tech support.

    I know nothing about anything other than the 1220.
    Wednesday, January 30, 2008 9:42 PM
  •  tneasham wrote:

    Boil all the posts down and you get, "RAID is better but more expensive and Microsoft doesn't support it."



    That is certainly my opinion.  I just read all of the RAID threads and saw nothing actually discussing RAID in depth.  I was forced to go do it with pretty much nothing to go on other than "it is unsupported".  I wanted to provide a thread where those who know the risks and are technically capable could go to discuss what they are doing.

    It hasn't gone that way.  Sigh.  But at least people can read my experience and know that someone out there is doing this successfully. 

    The sad part is that in my very first post I acknowledged that it isn't for grandma, but there were still lots of "party line" posts thrown in by those who feel compelled to protect all the grandmas reading up on RAID in this forum. 

    I have decided that if nothing else, such posts keep it bubbling to the top of the pot.  Kind of a "bump" that I don't have to go do.
    Wednesday, January 30, 2008 9:53 PM


  • TTT

    In all honesty you should write up a guide for the FAQ and request one of the moderators to post it.

    While I don't expect WHS to officially support RAID it would be nice to have more control over what the demigrator.exe does.  For instance controlling: balancing, duplication, when chkdsk was run, etc...  This would be nice for the 'power' users or those who have set up a RAID and don't need some of the protection that the demigrator provides.

    To some extent WHS does support RAID.  It does pop up a dialog for installing drivers when a new RAID comes online.  It also recognizes a RAID set as storage it can use.

    Personally, I'm trying to justify an areca arc-1210 controller. Smile

    Thursday, January 31, 2008 12:17 AM
  •  Lliam wrote:
    Personally, I'm trying to justify an areca arc-1210 controller.


    My advise would be to go with something like the Promise Supertrak EX8350. 

    http://www.newegg.com/Product/Product.aspx?Item=N82E16816102071

    The reason simply is that it is "fast enough" for the WHS environment, but it has 8 SATA connectors.  I think having MORE connectors than you need is critical in the WHS environment.  You do NOT want to be screwing things up by having to go with external drives because you don't have enough connectors to add more storage to the RAID.  IF you are able to go with at least (4) 500 GB drives right up front then a 4 port would probably suffice.

    In fact right now the Areca 1220 is on sale at newegg.

    However I would bet money that even a non-coprocessor board would suffice in this environment.

    I actually ordered (have in hand) one of the EX8350s to use in my WHS machine.  Basically the only reason I ended up using my Areca was pure laziness.  I had the server already built, with the drives mounted in the cabinet, motherboard and RAID controller, video etc.  I was using it for a big SQL Server backup to my main SQL Server machine.  It was "underutilized" so I decided to take it offline to try doing the WHS with RAID, just to see if it worked etc.  I was also trying to get WHS to run in a virtual machine on top of the Windows 2003 Standard that was already running on the machine.  I couldn't get WHS to load in the virtual machine and "see" the drive I gave it so I ended up just formatting and installing WHS on the raid volumes (two of them) that were already in place.  In the end I decided to go back and create a single volume instead of using the two volumes already on the RAID. 

    Well, after getting it installed and the machine connectors installed and stuff, I decided to just leave it in place.  Not an optimum use of an Areca 1220. 

    I may very well go back and move to the Promise someday soon so that I can have my Areca back for the SQL Server.  SQL Server needs and uses all the speed it can get.  I had a mother board go south and when I get that back I will be building a new SQL Server machine, new drives etc.
    Thursday, January 31, 2008 12:52 AM
  • I currently run a Windows 2003 server with an Adaptec 2820sa 8 port raid controller. I have the boot drives mirrored, and I have 5 400GB drives in a Raid 5. I started with 4 drives, and then expanded the raid 5 to 5 drives, and with the left-over data, I grew that partition under 2003.

    Example to Extend a Partition:

      • Type diskpart to start the utility
      • Type list to get an overview over all disks and volumes
      • Select the volume you want to work with i.e. select volume 2
      • Now this volume is in focus you can type expand to use all available space
    I ran this command in the Command Prompt, and immediately, my volume was larger. This allows me to grow my system, without having to worry about another drive letter. Would this happen to work in WHS?

    Also, I chose server 2003 64-bit for the single volume larger than 2TB deal, as I plan to migrate my raid setup to much larger drives when I need the space (online capacity expansion). With WHS, even if I expand the volume, I would still have to format it in 2TB intervals?? I think if this were the case, then if you had a single 2TB volume to the OS, and as you add drives to the Raid, you expand the next volume until it's 2TB, thereby reducing the number of volumes that DE has to worry about as you grow, even if it is one drive at a time.

    I really want to move my system to be WHS as I really like the web sharing features and the possibility of growing the system indefinitely, but I still am trying to decide if it's worth the change. Backup doesn't matter to me as I only use a Mac for my Desktop (and will never change that).

    Also, RAID is great and all, but it can totally fail. Just two weeks ago I had a process crash while data was transferring and then data became corrupt and I was left with a failed slice in my RAID. Solution, delete, recreate, recover from backup. Luckily, I have a 1.5TB USB device attached to my mac, and I do differential backups of my RAID 5 every night. After rebuilding my RAID, I reloaded the data, although it took about 12 hours to copy my 900GBs of data. What do you propose for backing up the WHS when it is used a primary storage device, and not a backup device. Remember, RAID doesn't save you from everything!!

    Comments are appreciated.
    Thursday, January 31, 2008 5:06 AM
  • I completely agree with John.  Unless you are doing some serious, performance critical stuff (DV editing, etc), you don't need a XOR engine aka microcontroller aka u-controller assist or whatever marketing term on your RAID card. ;-)  I just don't see the majority of home users, even techies, needing that kind of speed.  Certainly it isn't needed for backups, media streaming, etc.

     

    Given the WHS DE architecture, if funds are tight, it would be better to go with extra ports vs. coprocessor.  If you had to choose between the two, go with extra ports (assuming your case will have room to hold the drives).

     

    Of course, if money is no option, get both!  ;-)

     

    Ryan

    Thursday, January 31, 2008 11:54 AM
  • But, you know the real reason, MS lilkes to sell software not hardware.  Gosh what a great product we wrote, it does it all with software...  To bad hardware already exist that does a better job!!!!  But, really would a Consumer like a red/green light hot replacedable system, of course, and do they really want to mark which files / directories are protected and which are not (guess).  Seems a streach really, but hey they sell software not hardware I completely understand...  But the mom and pop that want to get into this system do have the money, it us nerds streaching our wallets to squezz everything we can out of our dollar for our personal use...  And guess what we want RAID!!!!

     

     

    Not for the real quetion,  Got an old server, ordered a controller  - five .5 GB drives (will raid 6 - 1.5TB single drive to OS).   What was the procedure again for the installation, could you make it clearer.  Insert disk, stand on one leg, reboot (that is always standard procedure)..face the East.... .  Just would like this a little clearer for me...  AND THANKS A BUNCH FOR THIS THREAD!!!!!! 

     

     

    Sunday, February 3, 2008 10:15 PM
  •  Home PC Guy wrote:
     

    Now for the real question,  Got an old server, ordered a controller  - five .5 GB drives (will raid 6 - 1.5TB single drive to OS).   What was the procedure again for the installation, could you make it clearer.  Insert disk, stand on one leg, reboot (that is always standard procedure)..face the East.... .  Just would like this a little clearer for me...  AND THANKS A BUNCH FOR THIS THREAD!!!!!!


    I'm trying to decide whether this post is for real.  If you do need assistance with installing RAID on your system I can certainly help, just post back again. 

    The general procedure is to:

    Setting up the raid hardware:

    If you are using built-in raid on the motherboard then you will need to go into the BIOS to configure the raid.  If you installing a RAID controller card then that will have a BIOS that presents itsefl AFTER the normal BIOS stuff.  You will need to go in there and set up a raid array.  I cannot be more specific on that stuff because it is veyr much specific to the motherboard or raid card but the book should help you to get that set up.


    Installing Windows:

    1) Build a floppy (yes, a FLOPPY) with the drivers for the RAID hardware (or software for that matter).
    2) Put in the A: drive
    3) Insert the install disk for Windows
    4) Boot off the Windows CD.
    5) IF YOU ARE SEEING BLUE DOS SCREEN (not the "blue screen of death, just a blue screen) then as soon as you start the install look down at the bottom for a question / statement something like "hit F6 to install special drivers".
    6) When you see that start tapping the F6 key.  I tap it twice per second or so until that question goes by because I have run into situations where the computer did not sense the F6.
    7) It will then ask some other question immediately after which you can ignore.
    8) Windows will continue to load and eventually will stop and present you a screen asking if you want to add special drivers.  Answer affirmatively and then select the driver from a menu that is presented next. 

    Windows will take off and install.  If you have done an install before then everything else will proceed as normal.


    OTOH if you were not actually askinng for assistance then... I just wasted a few minutes of my time.  ;-)
    Sunday, February 3, 2008 10:47 PM
  • When I first heard about WHS I was disappointed that RAID was "not supported".  I then was directed to the white paper on how the storage system works for WHS.  It looked (and looks) pretty darn good to me.  I can envision large storage needs in the future for all the digital video I take of the kids and such, not to mention pictures, and the duplication seemed to take care of the whole "I wish I had RAID 5 on this thing" problem that I initially had before.

     

    I have read through this thread somewhat carefully understanding nearly all of it.  I believe I read on a couple pages back that you would not want to run RAID 1 on the OS drive and JBOD on the storage end.  Did I misunderstand that?  Also, next question, from the white paper, it says that you'll get a warning on the client when one of the disks starts to get all faulty.  If you were running RAID 5/6 on the storage part, would you get that same warning?  Furthermore, what's to tell you that one of your RAID 1 OS drives failed?

     

    I am very glad this thread is here.  While I believe that the storage technology of WHS will indeed serve 90% of the population it's aimed at, I tend to sit in the 10% that likes to muck with stuff. 

     

     

     

    Matt

    Friday, February 8, 2008 7:27 AM
  • There isn't a problem with RAID 1 on system drive per se, it's just that some feel that if you are going to do RAID, then use a single RAID-5 array and then run the system as an entire volume.  This essentially disables the Drive Extender technology, and there will only be C: and D: partitions

     

    However, if you WANT to use the DE features, then RAID-1 on the system drive isn't a terrible diea.  Let's say you don't have the $$$ to build your fully populated RAID array today.  Or, let's say even if you did, you envision in a year you'll pass that storage limit and want to add drives anyway.  DE makes a lot of sense in this case, even with it's issues which will eventually be resolved.

     

    So let's say you go that route.  But...C: and D: have no redundancy.  You shouldn't ever lose "data" if that drive fails, but you have to reinstall and reconfigure everything.  Maybe you are fine with that.  Maybe you aren't.  If you aren't, then RAID-1 on HD0 is perfectly acceptible solution.

     

    Note that once Power Pack 1 is out, it lessens the RAID-1 argument somewhat.  Not significantly, but somewhat.

     

    I also am assuming that simply backing up the entire WHS sytem is not an option through some form of whole-system backup and future restore through bootable backup media.  If you were, then the lack fo redundancy of HD0 wouldn't be much of an issue, as you can restore the system from full backup and not have to go through the reinstall.  But realistically, I don't see this being an option for many of the WHS users out there.

     

    So, when does RAID-1 make sense?

     

    - you wan't to use DE over a RAID array for any one of a number of good reasons (support, cost, expandability,etc.)

    - you don't want to have to a full system reinstall, configuration, and tombstone rebuild if your HD0 fails

    - you won't be able to perform full system backups due to size of data array

     

    Hope that helps.

     

    Ryan

     

     

    Friday, February 8, 2008 11:55 AM
  • <duplicate post...deleted>

    Friday, February 8, 2008 11:55 AM
  •  Matt Greer wrote:

    the duplication seemed to take care of the whole "I wish I had RAID 5 on this thing" problem that I initially had before.


    yea but...


     

    I believe I read on a couple pages back that you would not want to run RAID 1 on the OS drive and JBOD on the storage end.  Did I misunderstand that? 

    That is not quite correct.  Running Raid1 on the system disk provides you a higher level of security against system disk failure.  The thing to understand is that WHS is based around "this is for mom n pop" and as such it tries very hard to stay away from complexity.  I have no argument against that, however for the technically capable RAID is not a problem to implement or monitor. 


    A failure of the system disk TRASHES the computer backups.  From what I have read, this is not a "may trash", but a WILL TRASH.  So protecting your system disk from failure with RAID1 is a positive step.


    Also, next question, from the white paper, it says that you'll get a warning on the client when one of the disks starts to get all faulty.  If you were running RAID 5/6 on the storage part, would you get that same warning?

    No.  If you have no RAID then the disk reports directly to windows (the OS) and the OS senses drive status.  With RAID in place, drives report to the RAID controller and the RAID controller senses drive problems.


    Having said that, if Windows finds a checkdisk problem on your raid array then it will tell you so.  But if a drive just dies or drops out of the raid array the OS will never know.  The RAID controller will know however.

    Furthermore, what's to tell you that one of your RAID 1 OS drives failed?

    All raid software has some method of alerting you in case of a RAID array problem.  It is usually a loud obnoxious beeping but it could be a popup in windows.  The method will depend on the RAID implementation, but it WILL be there. 

     

    I am very glad this thread is here.  While I believe that the storage technology of WHS will indeed serve 90% of the population it's aimed at, I tend to sit in the 10% that likes to muck with stuff. 

    Matt



    I just want more from WHS than it provides.  I like the concepts but I do not need the DE functionality, at least not yet.   In any event I understand RAID and I know that it will NOT cause problems for the OS or WHS.  It just serves up a  volume to Windows and that volume is used by WHS however WHS decides.  In the end though the RAID does what it is supposed to do and WHS does what it is supposed to do.  They do not interfere with each other.
    Friday, February 8, 2008 4:07 PM
  •  ryan.rogers wrote:

    So let's say you go that route.  But...C: and D: have no redundancy.  You shouldn't ever lose "data" if that drive fails, but you have to reinstall and reconfigure everything.


    That is certainly not my understanding.  The way I read it, if you lose the system disk you WILL lose the computer backups.  That is data and is quite important to me.  The "yea but you have your computer as a backup to the backup" argument simply does not hold water, because the SNAPSHOTS are gone which hold data tnat may no longer be on your computer.  You have lost the ability to get back to a previous state of the backed up computer.

    If I am wrong in my understanding I certainly want to be educated in the how and why.

    Friday, February 8, 2008 4:12 PM
  • If you lose the system disk, you lose your configuration (add-ins, remote access, users, etc). If you are in a multi drive system, you shouldn't lose your PC backups, as they are migrated to other drives in the pool. When you replaced the failed system drive, Windows Home Server setup will go through and rebuild tombstones for all the data, and when you re-add your users, it will re map them to their already existing user folders.

     

    Friday, February 8, 2008 4:44 PM
    Moderator
  •  John W. Colby wrote:

    A failure of the system disk TRASHES the computer backups.  From what I have read, this is not a "may trash", but a WILL TRASH.  So protecting your system disk from failure with RAID1 is a positive step.

    No, John, I'm pretty certain it's "may trash". If all the components of the backup database are on secondary disks in the storage pool (admittedly impossible to determine without digging into the file system) then the failure of the system disk will cost you nothing but tombstones (which are rebuilt during the reinstallation) and time reinstalling. See Saving a Copy of the Backup Database and Restoring a Backup Database in the Home Computer Backup technical brief for a little more information on how you can manipulate the backup database.

    Friday, February 8, 2008 4:55 PM
    Moderator
  • You are absolutely correct.  The backups are gone. When I said "data", I meant shares, etc.

     

    [Edit: Given the last two posts, clearly I'm mistaken.  I thought I remember reading that the backups are only stored on D:, unless you hack at the system.]

     

    Clearly different people view the backup feature differently.  That's life.  I myself do not see backups as "data", others will.  Neither view is less valid than the other.  It just means that some people are going to view some features a bit differently than the designers intended, and that's something the designers have to deal with if it ever becomes a big problem.  I don't think this one will, but it could.  The fact is, Power Pack 1 is actually addressing a lot of this, so this will no longer be an issue.

     

    I have seen that argument you refer to many times in the forums, and again, I believe there is no wrong or right side.  I happen to completely agree with you, but for different reasons.  MS also seems to agree as well, as if they did not, why add the "backup the backups" feature to PP1?  I'll tell you why.  Becuase the idea of consulting the source to rebuild your backups is, quite simply, flawed.

     

    The reason re-backing up from the source does not hold water is while I'm rebuilding my WHS, I'm *vulnerable*.  If I lose a drive or even some sectors on one of the other computers while I'm rebuildign my WHS, I'm hosed.  The backups are gone, and it may be hours or days before I can get them backed up again.  Shame on an archtiecture that, to begin with, didn't make the backups redundant.  They *are* important enough to warrant redundancy, whether you view them as "data" or not!!!

     

    And clearly MS now agrees that they are important enough to warrant redundancy, as we now have a way to backup the backups in case of total WHS failure.  Now you no longer have to worry.  You can shapshot your snapshots, so if you ever have to rebuild C: D: (because RAID isn't supported ;-), you will at least get your backups back WITHOUT having to consult the source!  Even if HD0 goes down in flames, you will know that your backups are secure.

     

    MS knew they had issues here, but hey,it's a V1 product.  These aren't stupid engineers.  You just can't fit *everything* into a V1 product.  The Power Pack goes a long ways to address these.

     

    If I had my choice, I would much rather have the option to target the backups for Duplication so they could have a copy off of HD0.  I understand why this wans't allowed, but on the other hand, it *is* a simpler model for simpleton users, and is more unified approach to how to redundantly store data.  In fact, I think the whole PP1 backup feature, while addressing a big problem, should be simply another way to provide redundancy, not the *only* way.

     

    I think WHS should be able to *competely* back itself up to the data pool, and restoring it due to HD0 crash is a matter of replacing the HD, tossing in the DVD, booting, and let it go.  When done, you are back up and running 100% configured.  Yeah, it would take 5-10GB or so + backup size off of your data pool, but TOTALLY well worth it!


    You still need the backup tool added in PP1 for true disaster recovery (fire, theft, etc).  But it shouldn't be needed for HD0 crash.  WHS reinstall should be able to completely bootstrap itself from the other drives, in an ideal world, as long as you as the admin have chosen that this capability is worth the loss in storage space.

     

    Ryan

    Friday, February 8, 2008 5:07 PM
  •  Ken Warren wrote:

    No, John, I'm pretty certain it's "may trash". If all the components of the backup database are on secondary disks in the storage pool (admittedly impossible to determine without digging into the file system) then the failure of the system disk will cost you nothing but tombstones (which are rebuilt during the reinstallation) and time reinstalling. See Saving a Copy of the Backup Database and Restoring a Backup Database in the Home Computer Backup technical brief for a little more information on how you can manipulate the backup database.



    Thanks for that clarification and for the link.
    Friday, February 8, 2008 5:44 PM
  • Thanks Ryan for and John for your kind replies.  Something else that *might* work is to use a program like Acronis to make daily incremental backups of your OS to the secondary storage area.  Might that be a somewhat acceptable solution to not using RAID 1 on the OS?  Seems to me that, based on the most recent posts, a restore of the previous night's backup of WHS from Acronis plus a rebuild of the database might get you going quicker than a full database restore.  Furthermore, if you time the Acronis backups to trigger after the nightly backups from WHS, you might not need to rebuild anything.

     

    Thoughts and comments?

     

     

    -Matt

    Friday, February 8, 2008 7:18 PM
  • Yea its real.  The installing Windows part was what I needed.   And thanks;  have the extra hardware coming in should be here next week.  I'm sure it will be a BIOS after the normal BIOS (already have a PC - RAID 1 - that I currently store pictures - will move the pictures to the home server).    It was the floopy part I needed  - But can I use another CD Drive USB Drive?  -  does it have to be a floopy - in Drive A?   It has been awhile since I did this kind of stuff...  But I already know the next problem I will have will be moving my DMR songs to this server (use Napster and  SONUS).  Already seen the blog at SONUS getting the DRM stuff to work off a shared drive....  You know I am about ready to go to Apple, but I would miss all of this fun!!!

     

     

    Friday, February 8, 2008 9:22 PM
  • Matt, yeah that would almost work, as would pretty much any backup solution.  If it weren't for just one teen-tiny issue. ;-)

     

    The problem isn't backing up HD0 drive.  That's easy.  The issue is when you restore the backup.

     

    Let's say your backup is 20 hours old.  But several files have been added, deleted, updated, moved, or renamed in your shares since that last backup.  There is now a disagreement between what your backed up D: says is in the pool, and what is actually in the pool.

     

    The core issue is you are restoring tombstones from point in time X, when in fact the data they represent (still in the pool) are from time X + 20 hours in the future.

     

    WHS makes no attempt to resolve this discrepancy (a bit of a shame, but understandable).  The reason it doesn't have to is the supported solution is to Reinstall, which rebuilds the tombstones, from scratch, perfectly.  As such, backing up just the HD0 is just as unsupported as RAID'ing HD0.

     

    This is why in my (long) post I was talking about *full* backups as an option.  Really the only way you can use a 3rd party backup tool to restore is if you backup *everything*, and restore *everything*.  Backing up and restoring either the system or data drives independently is not a good idea, at least how things stand today.

     

    Of course, if your data hadn't changed, at all, in the data pool there would be no issue with the restoration. ;-)

     

    One last note: one of the things on my very-long TODO list is to write a utility to find discrepancies in cases like this, and optionally fix them.  Sort of like a "chkdsk" for DE.  It could scan both primary and secondary shadow copies across the entire pool, and compare them with the tombstones.  It's not the easiest piece of code to be able to handle all of the possible deltas, but what I've designed on paper so far shows that it's theoretically possible.  It's just not trivial.  And running it could be painfully slow (but then again you'd only run it infrequently).


    In theory, if we had this utility today, and it worked like I think it could work, it would be fine to backup the system drive by itself, as if and you restored it, the first thing you would do is log in and run this theoretical "dechkdsk.exe /r", and it would update your tombstones (as well as your shadows if necessary) so that they were completely in sync with each other.  Of course, such a tool would be unsupported by MS.  But then again, MS doesn't support many 3rd party disk tools either.  But they still work.

     

    I don't know if anybody has started such a tool, but I will get to it as my customers will need it...it just won't be too soon.  Probably at least a few weeks.

     

    Ryan

    Friday, February 8, 2008 10:00 PM
  • Wow, hey thanks Ryan!  You're quite helpful.  I do appreciate it.

     

    I think, for me, the solution is as I usually do.  Set up the server and write down everything I do when I set it up, such as the software I install (iTunes server anyone?) and what settings I've manipulated, what computers connect, what folders or items upon which I've decided to do a "double" backup.

     

    A bit of a pain, but how often does a hard drive die?  Smile  /knocks on wood

     

    What would be really helpful, I would think, would be the ability to store the server "state" or settings, whatever they may be, that identify the computers that connect to them and whatever else would be important to remember.  Given that I do not currently have WHS (but am still planning on purchasing it) I don't know what all would need to be stored.  But if you could back that information up to a floppy or USB thumb drive and set it aside, that would make restoring/reinstalling a bit easier, or foolproof, or idiotproof I suppose.  I'm reminded of the network settings disk that you can make and just run on multiple computers to get them all on the same net.  Maybe some utility could make a "this is how you set up your server settings" type floppy.

     

    Perhaps I'm off base, or it isn't doable.

     

    Now, I just need a full tower case and a motherboard with 15 SATA ports... 

     

    -Matt

    Friday, February 8, 2008 10:22 PM
  • Heh.  It always seems that drives tend to fail when it can do pretty much the most damage to you.  ;-) 

     

    I don't know if you read much at the backup solution coming in PP1, but it looks pretty nice...

     

    http://blogs.technet.com/homeserver/archive/2008/01/06/windows-home-server-power-pack-1.aspx

     

    It isn't exactly what you are looking for, but if you did have all of this stuff backed up, getting up and running from a full Reinstall wouldn't be too horrible.  But unfortunately it doesn't back up an restore everything.

     

    Ryan

     

     

    Friday, February 8, 2008 10:47 PM
  •  Ken Warren wrote:
     John W. Colby wrote:

    A failure of the syste disk TRASHES the computer backups.  From what I have read, this is not a "may trash", but a WILL TRASH.  So protecting your system disk from failure with RAID1 is a positive step.

    No, John, I'm pretty certain it's "may trash". If all the components of the backup database are on secondary disks in the storage pool (admittedly impossible to determine without digging into the file system) then the failure of the system disk will cost you nothing but tombstones (which are rebuilt during the reinstallation) and time reinstalling. See Saving a Copy of the Backup Database and Restoring a Backup Database in the Home Computer Backup technical brief for a little more information on how you can manipulate the backup database.



    In any case, without RAID (again from my understanding) the loss of a single drive can cause loss of the backups.  It doesn't matter where the backups are stored, since they do not participate in the duplication they exist in one single place (drive) and the loss of that drive trashes your backup.

    Using a RAID 5 array, the loss of a single disk does not compromise the system.  Using Raid 6 the loss of TWO drives does not compromise the system.  Furthermore if you use RAID then duplication becomes a none issue and a whole ton of disk thrashing goes away as the DE no longer has to do as much balancing.
    Saturday, February 9, 2008 1:38 AM
  • If you can live with a limitation of 2 TB for your server, a single large RAID array is workable for WHS. But WHS uses the MBR style of disk partitioning, not GPT, so that's your limit for disk space.
    Saturday, February 9, 2008 4:15 AM
    Moderator
  •  Ken Warren wrote:
    If you can live with a limitation of 2 TB for your server, a single large RAID array is workable for WHS. But WHS uses the MBR style of disk partitioning, not GPT, so that's your limit for disk space.


    Are you positive about that Ken?  I tried to plug one of my ex-WHS hard drives into XP and it could not be read.  XP acknowledges that the volume was there but it could not access the contents because it was a GPT drive.  Vista was able to handle the drive without issues (Vista works with GPT drives, XP does not).  Conceivably I could have made a mistake, but I don't knowingly format drives as GPT drives.

    Saturday, February 9, 2008 4:51 AM
  •  Ken Warren wrote:
    If you can live with a limitation of 2 TB for your server, a single large RAID array is workable for WHS. But WHS uses the MBR style of disk partitioning, not GPT, so that's your limit for disk space.

    That is your limit for any single disk.  Again that has nothing whatsoever to do with RAID per se.

    If I choose to do so I can take my controller and place 8 terabyte drives on them.  I then make it a RAID 6 array and carve it up into (1) 500GByte volume, (2) 2 TByte volumes and (1) 1.5 TByte volume (total 5.5 TBYTES of actual data storage, 500 GIGS of landing zone).  The other two tbytes of course are used for the "parity" information.  I then feed those volumes to WHS.  WHS will happily take those six volumes and use them to create a single 5.5 TBYTE DE controlled data volume.  Again, to WHS, 6 volumes on a raid array and 6 physical hard disks are treated exactly the same.  The biggest difference now is that because of RAID 6 I have up to 2 disks which can die and still not lose my data.  Also I do not need file duplication, so DE does not need to do all the disk thrashing involved in moving COPIES of duplicated files around to other disks. 

    If you directly feed WHS those same EIGHT tbyte drives (no RAID now) and turn on file duplication then you get a maximum of FOUR tbytes of actual file storage (assuming that you duplicate everything).  You have no protection for the system drive, in fact you have no protection for any drive of course.  If the drive with your backup database happens to die... well... you just lost your backups.  If your system disk happens to die... well... you have to go do all the reinstall stuff.  If ANY disk dies WHS will spend the next eternity recovering tombstones and rebuilding your data partition.

    I do have a single volume ATM.  That volume is created on a RAID6 array out of (8) 320 gig drives (actual data storage about 300g).  So I get about 1.8 tbytes in my single volume.  Should I ever need more I will likely go for another RAID controller and start throwing TByte drives on it.
    Saturday, February 9, 2008 12:43 PM
  • Lliam, yes, I'm sure. Log in to your server with Remote Desktop and start up the Disk Management MMC snap-in. Click on the View menu, select Top, then Disk List. This view includes the partition table style. You will see that all WHS disks are listed as having MBR style partition tables.
    Saturday, February 9, 2008 2:01 PM
    Moderator
  • John, the points I was responding to were that with only a single "disk" in your server, you don't have a risk of file corruption form the DE bug, and you don't have the DE overhead dragging down performance. You are correct in those points, and they are significant advantages to running WHS on RAID, but you have to accept that your server will have a single "disk" of no more than 2 TB. When you take your larger array and carve it up into several smaller logical drives, then give those drives to WHS, I think you're giving up the advantages of RAID for a WHS PC. You will still have DE dragging down performance (whether you use duplication or not, DE will be actively involved with more than one disk in the storage pool), and you will still have the file corruption issue to deal with. You can't have it both ways.
    Saturday, February 9, 2008 2:09 PM
    Moderator
  •  Ken Warren wrote:
    John, the points I was responding to were that with only a single "disk" in your server, you don't have a risk of file corruption form the DE bug, and you don't have the DE overhead dragging down performance. You are correct in those points, and they are significant advantages to running WHS on RAID, but you have to accept that your server will have a single "disk" of no more than 2 TB.
    Sorry Ken, it wasn't obvious from your one line reply what you were responding to.

    If course DE will be involved as soon as you hande WHS more than a single volume. 


    When you take your larger array and carve it up into several smaller logical drives, then give those drives to WHS, I think you're giving up the advantages of RAID for a WHS PC.


    As for giving up the advantages of RAID, I would say you give up one of many advantages, but you keep the rest of the advantages.

    You give up:
    1) keeping DE out of the loop entirely.

    You keep:
    1) A disk failure is handled at the RAID level.  Discover the bad disk, replace it and stand back and let the raid controller rebuild.
    2) You will not have the "reinstall WHS" stuff to deal with as you would without RAID if the system disk died.
    3) You will not lose the backups as you would without RAID if the disk the backups were stored on died.
    4) Raid gives you more storage space for N disks.  Raid 5 is N-1 * size.  File duplication is N/2 * size.  Anything more than 2 drives gives an immediate advantage to RAID.
    5) Even with DE in the loop, without file duplication you lose all of the file copy  stuff (additional disk access) required to find room for and store the duplicate of each file, fill out the toombstone data to track the whereaboutsa of that duplicate etc.

    Those are significant advantages that you still have with RAID and are nothing to be sneezed at.

    You will still have DE dragging down performance (whether you use duplication or not, DE will be actively involved with more than one disk in the storage pool), and you will still have the file corruption issue to deal with. You can't have it both ways.


    True, DE getting in there will immediately expose you to the current file corruption issues.  And yes, DE will still be doing some stuff (landing zone management), but will no longer have to handle file duplication responsibilities and any speed issues that come along with the duplication process itself.

    Many people seem to view RAID as an "all or nothing" proposition.  It truly is not.  RAID is simply a storage system that handles physical level data duplication for you.  When you no longer need to worry about a disk dying and trashing your system you can spend your time worrying about other things, like your golf score or how to pay for the dentist bill.

    I have RAID and I do not worry at all about what happens if my system disk (or any other disk) fails.

    You probably know,but with 8 disks available to my raid controller I can create a RAID6 array with 7 of them and make the 8th disk a hot spare.  In that case, if any disk does fail, the controller IMMEDIATELY takes the hot spare and starts rebuilding the RAID array, without any intervention from me.  Now that is peace of mind.

    For the dreamers out there check this out:

    http://www.newegg.com/Product/Product.aspx?Item=N82E16816151027

    Now imagine 24 terabyte drives.  1 hot spare, raid 6 leaves 21 terabytes as a single volume, with full Raid 6 protection AND a hot spare.  No DE, no balancing, no duplication, no DE corruption...

    Total price...

    $1200 for the controller
    $6600 for the drives

    I can only imagine what the streaming read speed would be, but you can rest assured that you could never shovel the data out of the computer fast enough for the disk subsystem to be the bottleneck.
    Saturday, February 9, 2008 3:04 PM
  • RAID gurus,

    I plan to build or purchase a custom WHS box.
    Primary purpose: backup my personal laptop (qty=1).
    Secondary purpose: serve shared digital content to my personal laptop, work laptops (qty=2), and PS3.
    The PS3 is connected by 100BaseT.  Everything else is connected wirelessly (802.11g now, probably 802.11n later).
    I could easily live within a 2TB per volume limit.
    I don't expect to stream media to multiple clients concurrently.

    Based on the discussion you have had I think that my wireless connection would be the primary throttle on performance.
    So my questions are:

    1) Do you agree that RAID 5/6 w/hardware co-processor would be a waste of money for my application?

    2) Is RAID 1 likely to be either more secure or easier to recover from than Power Pack 1 (not to mention already available).

    3) John mentioned that I could "grow" storage inside a RAID array by adding disks and rebuilding.  But he also said that this added capacity would be presented as additional volumes.  Is there no simple way in RAID to rebuild the array with additional disks and still present a single higher-capacity volume?

    [added one]
    4) Does anyone make WHS whiteboxes with very low power consumption?
    For example the Apple Time Capsule is spec'd at 30W max consumption.  The HP EX475 w/2 disks burns 60W idle state, 73W full load.  Any way to get closer to 30W consumption with a 2-disk RAID 1 WHS solution?


    Sunday, February 10, 2008 3:41 AM
    1. Yes. For purposes of Windows Home Server, motherboard RAID support (from a major vendor) is likely to bemore than  adequate.
    2. Yes. PP1 doesn't include any way to back up the system drive; the "approved" recovery method is a server reinstallation. (I expect it will be for the foreseeable future...)
    3. Not every RAID controller supports this. Expect to spend a couple hundred dollars for one that does (John, do you know of a less-expensive one?). I would just buy the disks up front and not worry about it.
    4. Tranquil PCs in the UK has some very low power systems. Take a look, but you will probably not like the specs overall, if you're looking to have a 2TB storage pool. Frankly, it's just not in the nature of Windows servers to be low-power devices, much less with a RAID array. RAID is for performance and/or data integrity. If you want power savings install the smallest possible number of large efficient drives to meet your needs. (2x WD Greenpower 1TB drives, in your case...)
    If you're technically proficient, and like to tinker, go ahead and set up RAID.
    Sunday, February 10, 2008 5:54 AM
    Moderator
  •  Kevin Dezfulian wrote:

    Based on the discussion you have had I think that my wireless connection would be the primary throttle on performance.
    So my questions are:

    1) Do you agree that RAID 5/6 w/hardware co-processor would be a waste of money for my application?


    First let me say that I have used motherboard RAID for my system drives on normal XP and Win2K3 boxes and have been OK with the speed.  They are not stellar but they are free and adequate for that purpose.  However WHS uses the system disk for a landing zone and as such if you ever went to a second volume things would likely get miserable.

    Sooo....Maybe it would be a waste of money.  Unfortunately no one else has volunteered ANY transfer rate numbers to this thread so we don't have any data to go on.  I can tell you from experience that motherboard RAID WRITE speeds are abysmal.  A hardware coprocessor handles the intensive math calculations that go along with RAID.  There are cheap(er) FOUR PORT coprocessor boards, in the $300 range as opposed to the $500 range for the 8 port boards.

    If you are going to "do it yourself" I would suggest setting up MB raid and see how it goes.  Do some timings and see if you are happy.  The problem might be trying to get the content onto the box.  If you have a terabyte of data and are trying to get it onto the box at 3 mbytes / sec... well do the math.  We are talking weeks of nonstop data transfer.

    But set up MB raid and look at what you get. 

    Whatever you do, do NOT MOVE your data initially.  Keep a copy on your original location!!!  WHS may work great from the getgo, or it might not!


    2) Is RAID 1 likely to be either more secure or easier to recover from than Power Pack 1 (not to mention already available).


    Yes.  RAID for the system volume (in your case a pair of 1 tbyte disks? the ONLY volume) will go a long ways towards preventing drive deaths from interrupting your day.


    3) John mentioned that I could "grow" storage inside a RAID array by adding disks and rebuilding.  But he also said that this added capacity would be presented as additional volumes.  Is there no simple way in RAID to rebuild the array with additional disks and still present a single higher-capacity volume?


    Not that I am aware of.  Even if you could, you MIGHT bang up against WHS' ownership of the volume and it might complain.  I know that there are some utilities that will expand a volume, but have never tried to apply them to raid volumes, nor to a WHS volume.


    [added one]
    4) Does anyone make WHS whiteboxes with very low power consumption?
    For example the Apple Time Capsule is spec'd at 30W max consumption.  The HP EX475 w/2 disks burns 60W idle state, 73W full load.  Any way to get closer to 30W consumption with a 2-disk RAID 1 WHS solution?

    I know nothing about this one.
    Sunday, February 10, 2008 12:54 PM
  • Kevin wrote:

     

    "4) Does anyone make WHS whiteboxes with very low power consumption?"


    I don't know of any retail units, but you could probably get close to 40w if you went with a mini-ITX board with Via C7 ULV cpu and the Samsung Spinpoint's.  30W is a tall order though, especially for (2) drives.


    Ryan

    Sunday, February 10, 2008 2:26 PM
  • 30 watts is a tall order, especially if you need to get performance out of it.  The via processors are not known for horsepower, and I think you could pretty much hang up any kind of raid.  Software raid would take more horsepower than one of those low power CPUs could deliver and the RAID controller cards have a coprocessor which are not low power.

    IIWM I would look at an intel or AMD processor specificaly low power, dual core if possible, though I am no expert in that area.

    As for the drives...

    http://www.wdc.com/en/products/products.asp?driveid=336&language=en


    Sunday, February 10, 2008 5:44 PM
  •  John W. Colby wrote:

     

    Sooo....Maybe it would be a waste of money.  Unfortunately no one else has volunteered ANY transfer rate numbers to this thread so we don't have any data to go on.  I can tell you from experience that motherboard RAID WRITE speeds are abysmal.  A hardware coprocessor handles the intensive math calculations that go along with RAID.  There are cheap(er) FOUR PORT coprocessor boards, in the $300 range as opposed to the $500 range for the 8 port boards.

     

    The motherboard raid solution #'s are probably going to vary widely.  For instance the motherboard that my WHS is installed on has a RAID solution running on the PCI bus (not the PCI-E bus).  I believe this limits speeds to a maximum of 125MB/s.  With this motherboard I was getting 105MB/s averages with HDTach w/ RAID5 & ~115MB/s in RAID0 (four drives for both RAIDs).  These are read times not write times.

     

    In general, it appears that motherboard raid is as fast as software raid controller cards.  I don't see the purpose of buying a software card if your motherboard supports 4+ drives in RAID configurations.  A reference : http://www.tweaktown.com/reviews/918/1/page_1_introduction/index.html

     

     

    Sunday, February 10, 2008 10:49 PM
  • Can you give us some write times?  And some system specs as well.  MB model, processor, memory, drives used etc.  That would be very useful.

    Thanks,
    Monday, February 11, 2008 12:45 AM
  •  John W. Colby wrote:
    Can you give us some write times?  And some system specs as well.  MB model, processor, memory, drives used etc.  That would be very useful.

    Thanks,


    I didn't get any write times for RAID5 before I de-RAIDed it.  Considering the 8+ hour build time of the RAID5 I probably won't recreated it with that particular chipset.

    Specs:

    AMD 3800+ X2
    ASUS A8N-SLI Premium 939 NVIDIA nForce4 SLI

    2 gig patriot ram
    4xseagate 320gb hard drives

    The motherboard uses the Silicon Image 3114R RAID controller for RAID5.


    Monday, February 11, 2008 3:19 AM
  •  Ken Warren wrote:
    1. Tranquil PCs in the UK has some very low power systems. Take a look, but you will probably not like the specs overall, if you're looking to have a 2TB storage pool. Frankly, it's just not in the nature of Windows servers to be low-power devices, much less with a RAID array. RAID is for performance and/or data integrity. If you want power savings install the smallest possible number of large efficient drives to meet your needs. (2x WD Greenpower 1TB drives, in your case...)


    Thank you Ken and others for your replies.  I can tell that my appetite for data is a bit lower than most others on this thread.  My primary reason to have WHS + RAID1 is paranoia about losing my kids pictures/videos.  But I don't want to burn 100W leaving the thing on all the time.

    I looked into Tranquil and they sell solutions that are right up my alley.  I am in the U.S. and their prices converted to dollars are pretty steep.  So I still hope to build my own.

    So far I have been looking at a min-ITX setup along these lines:
    Via LN10000EG 1.0GHz C7 fanless w/RAID1 support     CPU < 12W max
    2 x 2.5" 250GB HDD                                                    ea drive ~ 0.6W idle
    1 x 512MB DDR2 533MHz SDRAM
    With an efficient power supply I hope to acheive ~40W.

    So far the hardest thing has been to locate a mini-ITX case that supports 2 drives.
    I had assumed, since the VIA chipset included RAID1 support, that it could handle the overhead.
    I don't think there is much required computationally (as compared to RAID 5/6).

    But I will start looking at Intel solutions.  Hope there are some out there with a very low power envelope.
    Monday, February 11, 2008 3:32 AM
  • One thing that is important to remember when discussing RAID is that software RAID always depends on the software (DUH).  If you use an add-in card then the card (and the software) can move from computer to computer.  This makes a difference if the motherboard goes south.  If you have a raid configuration driven by the motherboard, then you may be forced to go buy an exact replacement for that motherboard to get your data back, whereas if you use an addin card you can just transfer the raid card and the drives to another computer, install the software and have access to your data.

    For this reason alone, using the built-in RAID on the motherboard may be a questionable decision. 

    Again however, "it all depends".  If you are ONLY doing a raid 1 on your system disk and using non-raid DE managed drives for the actual data storage then using the MB raid is just fine.  The reason is simply that if the motherboard dies, getting the RAID 1 system disk to run correctly on a different motherboard is iffy at best (although I have heard of people doing it, though not with WHS). 

    However if you are running RAID on the data disks as well, then relying on the botherboard raid is probably not a good idea.  Motherboards do die, and you may or may not even be able to go buy another motherboard of the same make / model.  With a RAID card at least you can transfer the raid array to another machine and still get your data volumes back.
    Monday, February 11, 2008 4:35 AM
  •  Kevin Dezfulian wrote:

    I had assumed, since the VIA chipset included RAID1 support, that it could handle the overhead.
    I don't think there is much required computationally (as compared to RAID 5/6).


    You are absolutely correct there.  I was talking about Raid 5 not Raid 1.  Raid 1 is a simple mirror where whatever is written to drive A is also written to drive B.  Speeds should be just fine for that.

    My apologies for the confusion on my part.
    Monday, February 11, 2008 4:39 AM
  •  John W. Colby wrote:
    One thing that is important to remember when discussing RAID is that software RAID always depends on the software (DUH).  If you use an add-in card then the card (and the software) can move from computer to computer.  This makes a difference if the motherboard goes south.  If you have a raid configuration driven by the motherboard, then you may be forced to go buy an exact replacement for that motherboard to get your data back, whereas if you use an addin card you can just transfer the raid card and the drives to another computer, install the software and have access to your data.

    For this reason alone, using the built-in RAID on the motherboard may be a questionable decision. 

    This is something of an urban legend; it used to be (sort of) true, but hasn't been true for years. The major players in the motherboard RAID arena offer pretty good backward compatibility these days. You can, for example, take an array built on an Intel ICH7R controller and move it to an Intel ICH9R controller with no problem. (I've actually done this.) Someone (could have been Tom's Hardware) did a review of motherboard RAID portability a few months ago, and the chipset makers they looked at were about the same. Of course, that would limit you to buying a new motherboard with a chipset from the same vendor.
    Monday, February 11, 2008 12:52 PM
    Moderator
  •  Ken Warren wrote:
     John W. Colby wrote:
    One thing that is important to remember when discussing RAID is that software RAID always depends on the software (DUH).  If you use an add-in card then the card (and the software) can move from computer to computer.  This makes a difference if the motherboard goes south.  If you have a raid configuration driven by the motherboard, then you may be forced to go buy an exact replacement for that motherboard to get your data back, whereas if you use an addin card you can just transfer the raid card and the drives to another computer, install the software and have access to your data.

    For this reason alone, using the built-in RAID on the motherboard may be a questionable decision. 

    This is something of an urban legend; it used to be (sort of) true, but hasn't been true for years. The major players in the motherboard RAID arena offer pretty good backward compatibility these days. You can, for example, take an array built on an Intel ICH7R controller and move it to an Intel ICH9R controller with no problem. (I've actually done this.) Someone (could have been Tom's Hardware) did a review of motherboard RAID portability a few months ago, and the chipset makers they looked at were about the same. Of course, that would limit you to buying a new motherboard with a chipset from the same vendor.


    So it is not exactly an urban legend then?  It is still true that you would have to figure out what chipset your motherboard used and go buy a replacement MB that uses a chipset from the same manufacturer.

    In fact I just lost a motherboard.  I went out and figured out the motherboard I wanted, not the motherboard I needed to get my raid back.  I installed the MB in the same case, popped the controller in, installed the driver and was up and running.  This was for a SQL server system NOT a WHS system.  In this case my data was SQL Server databases and the whole WHS OS reinstall thig did not come into play, but still, a raid controller card does to a large extent divorce you from requirement to pick a specific motherboard.
    Monday, February 11, 2008 2:33 PM
  •  Kevin Dezfulian wrote:


    So far the hardest thing has been to locate a mini-ITX case that supports 2 drives.
    I had assumed, since the VIA chipset included RAID1 support, that it could handle the overhead.
    I don't think there is much required computationally (as compared to RAID 5/6).

    But I will start looking at Intel solutions.  Hope there are some out there with a very low power envelope.


    Kevin,

    You could use the Chenbro ES34069 case (and have room for 4 drives).


    It is referenced in this post:

    http://forums.microsoft.com/WindowsHomeServer/ShowPost.aspx?PostID=2786791&SiteID=50


    Monday, February 11, 2008 2:47 PM
  • OK... Here goes....

     

    I ended up here with the exciting prospect of building a new server 03 box for home... Looks to me like I'm loving the WHS setup, especially for the wife.

     

    I'm proficent in windows server technologies and hardware, and the prospect of building a home server system to protect all of our data without using raid is crazy. The whole purpose of the system (which originally would have been server 2k3) is for data redundancy... Don't get me wrong, I'm a little paranoid with data protection, but I've also had to deal with actionfront data recovery labs (now seagate) on a dozen or so occasions... not fun, or cheap.

     

    I'm lucky enought to have access to some pretty nice hardware leftovers.. like redundant psu's and adaptec raid controllers and such, and plan to build a no holds bar system.

     

    I digress.

     

    I am however unfamiliar with WHS... I've read the whitepapers... (although they seem more like marketing material than the whitepapers I'm used to... comeon MS.. ) on DE and fileshare, and read dozens of posts on both in the forums here.

     

    Obviously fileshare is not of a concern to me as I won't be using it with raid, but DE is an issue with >2tb array's

     

    THIS IS WHERE THE QUESTION STARTS IF YOUR SCANNING

    tia for any advice.

     

    Plan to use a Raid 6 with (5) 1tb drives for now and 3 more later (8 port adaptec card, supports OCE).

     

    From what I read, I definatley do not want to use DE... and please don't respond with any of the *** about trying to get GUID Part. Tables working.

     

    So here's what I have planned...

     

    First Volume 2tb (I understand it will be split with 20gb system and 1.98tb data)

     

    as for the second volume of 1tb and the additional volumes I'll add later on, can you add them outside of the DE system pool, and use them with normal shared folders.

     

    I understand for sure I'll lose the functionality of accessing this information through the WHS interface, but I think 2tb is enough for my wife's pictures and music, and I don't mind accessing the rest of my information through regular w2k3 style shares....

     

    Does WHS support this. Will I have a problem mapping to these shares. I probably couldn't use these extended drives through the connector interface, or any xbox systems.. is this correct. Could I stream Videos from these non DE extended drives to an xbox without installing 3'rd party xbox streaming software that is?

     

    What functionality would I be losing with the information stored on these parts of the drives....

     

    If I'd be losing to much functionality, I'm thinking of using these extended portions for less used information, and lets say for instance I store DVD images on these extended non DE drives, I could create a web interface to access these portions and copy them to the original 2tb volume when I want to use them...

     

    e.g.  want to watch a movie that is older and stored on non WHS drive... bring up interface, want to watch this movie... makes a copy on the original 1.98tb volume... watch movie streaming from this original volume (assuming I can't watch it from the non DE volume)... and then after so many days.. e.g. 30 days or something.. all of the files in the currently watching directory on 1.98 get deleted, because the file was copied from the non DE portion where it still resides...

     

    any help greatly appreciated... I kind of want to jump in feet first when my eval copy arrives... just ordered it... although more and more it looks like I might have to set up a virtual environment with a few vpc's and virtual hd's first to F around with it to see how this thing works....

     

    Dave

    wishing WHS was on MSDN in vhd format (or any format for that matter)

     

     

    Sidenote:

    if this is essentially a win2k3 server platform, does anyone know if the disk management snap-in supports virtualization of drives so that the non DE drives could be spanned to create a single 4tb drive to the user (not logically of course.. I understand this) and of course there is no way to implement virtualization on the main drive as this would have to be supported during install of the WHS which I'm sure is far beyond the scope of its use. It would just be nice to have my 2tb drive for WHS and the wife, and the 4tb left over for me, and backups etc....

    Saturday, February 16, 2008 4:38 PM
  •  mmb wrote:

    Plan to use a Raid 6 with (5) 1tb drives for now and 3 more later (8 port adaptec card, supports OCE).





     

    First Volume 2tb (I understand it will be split with 20gb system and 1.98tb data)

     

    as for the second volume of 1tb and the additional volumes I'll add later on, can you add them outside of the DE system pool, and use them with normal shared folders.



    AFAICT the underpinnings are plain old vanilla Win2003 Standard edition.  You can tell DE to ignore the additional VOLUMES, and then create normal windows shares, complete with the normal Windows 2003 security for those shares.


     

    I understand for sure I'll lose the functionality of accessing this information through the WHS interface, but I think 2tb is enough for my wife's pictures and music, and I don't mind accessing the rest of my information through regular w2k3 style shares....

     

    Does WHS support this. Will I have a problem mapping to these shares. I probably couldn't use these extended drives through the connector interface, or any xbox systems.. is this correct. Could I stream Videos from these non DE extended drives to an xbox without installing 3'rd party xbox streaming software that is?

     

    What functionality would I be losing with the information stored on these parts of the drives....


     

    Whatever the WHS interface brings to the table.  The ability to create the shares through that, see the amount of room left etc.  Not much I am thinking.



     

    any help greatly appreciated... I kind of want to jump in feet first when my eval copy arrives... just ordered it... although more and more it looks like I might have to set up a virtual environment with a few vpc's and virtual hd's first to F around with it to see how this thing works....

     

    Dave

    wishing WHS was on MSDN in vhd format (or any format for that matter)

     

     

    Sidenote:

    if this is essentially a win2k3 server platform, does anyone know if the disk management snap-in supports virtualization of drives so that the non DE drives could be spanned to create a single 4tb drive to the user (not logically of course.. I understand this) and of course there is no way to implement virtualization on the main drive as this would have to be supported during install of the WHS which I'm sure is far beyond the scope of its use. It would just be nice to have my 2tb drive for WHS and the wife, and the 4tb left over for me, and backups etc....



    I am thinking that whatever you can do with Windows 2003 SE you can do with the Windows 2003 under WHS.  Please come back and share your experience though.
    Sunday, February 17, 2008 12:49 AM
  • I've just read all 7 pages with absolute delight (as to how your contrust you posts) and absolute dismay (as the realisation of what im trying to achieve with WHS seems impossible)

     

    I shall explain that i've been directed to your thread from Wegotserved and have attached the link to my thread on that forum as reference;

     

    http://www.wegotserved.co.uk/forums/index.php?showtopic=631

     

    This is my setup so far;

     

    1x Lian Li PC-A16B Aluminium case Full tower in Black
    3x Icy Dock MB455 5 x Hotswap SATA bays
    1x Samsung 320GB SATA II 300 7200rpm (system disk)
    1x Silverstone Strider ST75F 750W PSU
    1x AMD AM2 Athlon 64 Dual Core 6400
    1x Asus M2N32 WS Professional mobo
    1x Corsair 4096MB TwinX XMS2 6400
    2x Apaptec 6 port SATA controller PCI-X
    1x ATi HD 2400 PRO

     

    No im no fan of software raid and as such have invested a couple of pounds into a couple of Adaptec 6 port raid cards, now these may not be the dogs they are still adequate cards for my needs and i shall use the controller built onto the Asus motherboard as well, i don't need massive data through-put, these are after all only for storage and retrieval but i wanted to purchase a brand that i trusted would stand the test.

     

    Ok so my plan was to use each of the controllers (2x adaptec and onboard over 3 raid 5 arrays using 5 disks, this would have given me a little over 11Tb of space if i were to use 1Tb disks, now although i don't plan on using all of this data at the moment, i do plan on being able to in the future, this being the reason for this build.  It would have been easy to buy a ready-made NAS or HP media server running WHS, but i want flexibility and expandability within the system so the only hardware i would need to buy would ne hdds as and when i need them.

     

    I have at the moment, WHS running on a single 320Gb disk and so far am enjoying the WHS experience with regard to RDP, simple 1 click backup (although no 64bit support as yet, unless im missing something) and add-ons that are available like whiist and web-guide but has come across a major problem with how WHS wants to add drives to a pool, then move data around this pool as it sees fit.  This is where i (and from reading this and many of other threads is where i come unstuck.  I want the functionality of being able to add drives to the pool so all of my data can be accessed from a remote connection but i can't see any possible way of doing this.  I have as with many other readers of this forum setup and maintained raid arrays for many years as i value my data and put my trust firmly in the hands of the hardware vendors rather than software for my data protection and backup and this is how it will continue for some time yet.  But how can i keep adding volumes (in my case another raid 5 over 5 disks) accessible through the web interface as well keeping tabs on what data is stored on what array?

     

    My plans for this build were as such;

     

    2x 320Gb - system disk - raid 1

    5x500Gb - storage - raid 5

     

    the in the future as 1Tb disks fall in price

     

    5x1000Gb - raid 5 (copy data from storage)

    remove 5x500Gb raid 5 and add

    2x 5x1000Gb - storage - raid 5

     

    Thus giving me a total of over 11Tb of storage space over 3 raid controllers

     

    Now how can i get WHS to see all 3 arrays and allow me to gain access to them through the simple and easy to use web interface so i have access to my data at any location?

     

    I am not interested in using the duplication feature and having the overheads that will be associated with DE shuffling data around as it sees fit seems quite unnecessary as once the data is in place it doesn't need to be moved for any reason other than me deciding to do so (i.e deleting unwanted data)

     

    This data will need be accessible from multiple computers around the home and simple file sharing and permissions would allow me to gain control over the availability of the data to the users i see fit but i do like the way you navigate with easy and quick point and click setup that WHS brings to the table.  I also like that WHS would see all the drives as one big pool but i would like control as to what data is stored on what volumes so setting up the shares on each folder myself through WHS would have been nice.  Seeing all 11Tb as one volume would also make it more user friendly but if i were to add 3 raid 5 arrays as well as a raid 1 would WHS have a fit if any of these hdds were to go down, and would this even be possible with WHS or 2003?

     

    Ok, so if this isn't possible then, what about me having 3 volumes, each of these being a raid 5 array and me being able to add these outside of WHS to the web interface and setup permissions for users?

     

    Also, would it be possible for the data to be recovered off any of the raid 5 arrays from another system using the original controller should anything with the computer main computer hardware itself, or the 2 system disks not allow me to do so on a WHS OS?

     

    I am at a loss as to what to do, and my only saving grace is that i am using WHS under a 30-day trial so haven't committed myself 100% financially although in my mind i kinda had!

     

    Hope someone can help outline exactly what is and isn't possible with WHS with my hardware setup.

     

    Regards

    Jacko

     

    Tuesday, February 19, 2008 11:28 PM

  • Not to complicate things, but this is an interesting post:

    http://blogs.zdnet.com/storage/?p=293


     Jackosme wrote:

    I've just read all 7 pages with absolute delight (as to how your contrust you posts) and absolute dismay (as the realisation of what im trying to achieve with WHS seems impossible)

     

    I shall explain that i've been directed to your thread from Wegotserved and have attached the link to my thread on that forum as reference;

     

    http://www.wegotserved.co.uk/forums/index.php?showtopic=631

     

    This is my setup so far;

     

    1x Lian Li PC-A16B Aluminium case Full tower in Black
    3x Icy Dock MB455 5 x Hotswap SATA bays
    1x Samsung 320GB SATA II 300 7200rpm (system disk)
    1x Silverstone Strider ST75F 750W PSU
    1x AMD AM2 Athlon 64 Dual Core 6400
    1x Asus M2N32 WS Professional mobo
    1x Corsair 4096MB TwinX XMS2 6400
    2x Apaptec 6 port SATA controller PCI-X
    1x ATi HD 2400 PRO

     

    No im no fan of software raid and as such have invested a couple of pounds into a couple of Adaptec 6 port raid cards, now these may not be the dogs they are still adequate cards for my needs and i shall use the controller built onto the Asus motherboard as well, i don't need massive data through-put, these are after all only for storage and retrieval but i wanted to purchase a brand that i trusted would stand the test.

     

    Ok so my plan was to use each of the controllers (2x adaptec and onboard over 3 raid 5 arrays using 5 disks, this would have given me a little over 11Tb of space if i were to use 1Tb disks, now although i don't plan on using all of this data at the moment, i do plan on being able to in the future, this being the reason for this build.  It would have been easy to buy a ready-made NAS or HP media server running WHS, but i want flexibility and expandability within the system so the only hardware i would need to buy would ne hdds as and when i need them.

     

    I have at the moment, WHS running on a single 320Gb disk and so far am enjoying the WHS experience with regard to RDP, simple 1 click backup (although no 64bit support as yet, unless im missing something) and add-ons that are available like whiist and web-guide but has come across a major problem with how WHS wants to add drives to a pool, then move data around this pool as it sees fit.  This is where i (and from reading this and many of other threads is where i come unstuck.  I want the functionality of being able to add drives to the pool so all of my data can be accessed from a remote connection but i can't see any possible way of doing this.  I have as with many other readers of this forum setup and maintained raid arrays for many years as i value my data and put my trust firmly in the hands of the hardware vendors rather than software for my data protection and backup and this is how it will continue for some time yet.  But how can i keep adding volumes (in my case another raid 5 over 5 disks) accessible through the web interface as well keeping tabs on what data is stored on what array?

     

    My plans for this build were as such;

     

    2x 320Gb - system disk - raid 1

    5x500Gb - storage - raid 5

     

    the in the future as 1Tb disks fall in price

     

    5x1000Gb - raid 5 (copy data from storage)

    remove 5x500Gb raid 5 and add

    2x 5x1000Gb - storage - raid 5

     

    Thus giving me a total of over 11Tb of storage space over 3 raid controllers

     

    Now how can i get WHS to see all 3 arrays and allow me to gain access to them through the simple and easy to use web interface so i have access to my data at any location?

     

    I am not interested in using the duplication feature and having the overheads that will be associated with DE shuffling data around as it sees fit seems quite unnecessary as once the data is in place it doesn't need to be moved for any reason other than me deciding to do so (i.e deleting unwanted data)

     

    This data will need be accessible from multiple computers around the home and simple file sharing and permissions would allow me to gain control over the availability of the data to the users i see fit but i do like the way you navigate with easy and quick point and click setup that WHS brings to the table.  I also like that WHS would see all the drives as one big pool but i would like control as to what data is stored on what volumes so setting up the shares on each folder myself through WHS would have been nice.  Seeing all 11Tb as one volume would also make it more user friendly but if i were to add 3 raid 5 arrays as well as a raid 1 would WHS have a fit if any of these hdds were to go down, and would this even be possible with WHS or 2003?

     

    Ok, so if this isn't possible then, what about me having 3 volumes, each of these being a raid 5 array and me being able to add these outside of WHS to the web interface and setup permissions for users?

     

    Also, would it be possible for the data to be recovered off any of the raid 5 arrays from another system using the original controller should anything with the computer main computer hardware itself, or the 2 system disks not allow me to do so on a WHS OS?

     

    I am at a loss as to what to do, and my only saving grace is that i am using WHS under a 30-day trial so haven't committed myself 100% financially although in my mind i kinda had!

     

    Hope someone can help outline exactly what is and isn't possible with WHS with my hardware setup.

     

    Regards

    Jacko

     

    Wednesday, February 20, 2008 12:46 AM
  • Unfortunately I have been "visited" by none other than Joel Burt himself and ordered to stop discussing RAID.

    I have put together a group on YAHOO if you wish to take this discussion there.

    WHSOnRaid

    Other than that I cannot help you.

    My apologies.

    Wednesday, February 20, 2008 1:07 AM
  • John,

     

    You have brought some great advice to this forum.  It is too bad this thread cannot continue.  I signed up for your group. Thank you for starting it.

     

    Todd

    Wednesday, February 20, 2008 1:17 AM
  • mmb,

     

    What are you going to do about hardware compatibility for Windows Server 2003?  Try as I might, I cannot find a reasonably priced motherboard that has 2k3 chipset drivers, 2k3 LAN driver, and I sure as heck wasn't able to find a graphics card with 2k3 drivers.  I got told that only OEMs provide drivers for gfx.  I know other folks have some existing hardware that they can play with, but I'm planning on building this from scratch and I don't have that luxury.

     

    Thanks,

     

    -Matt

    Wednesday, February 20, 2008 2:26 AM
  •  Matt Greer wrote:

    mmb,

     

    What are you going to do about hardware compatibility for Windows Server 2003?  Try as I might, I cannot find a reasonably priced motherboard that has 2k3 chipset drivers, 2k3 LAN driver, and I sure as heck wasn't able to find a graphics card with 2k3 drivers.  I got told that only OEMs provide drivers for gfx.  I know other folks have some existing hardware that they can play with, but I'm planning on building this from scratch and I don't have that luxury.

     

    Thanks,

     

    -Matt



    I'm not sure anyone can provide you with the answer you want.

    WHS will probably install just fine on any x86 motherboard.  However I understand that you don't want to risk the 'probably' in that statement.  When I've run into a driver problem I've been able to install the XP 32 version of the driver with no problems (i.e. graphics cards & NIC).  WHS installs the default VGA driver to begin with, which to be honest is all you need.

    Wednesday, February 20, 2008 2:47 AM
  •  Lliam wrote:
    WHS installs the default VGA driver to begin with, which to be honest is all you need.

     

    Yeah, I think you're right on that one.  When you remote desktop in to the server, can you apply video modes that are supported by the card on the local computer?

     

    -Matt

    Wednesday, February 20, 2008 3:23 AM
  • I'm building my system on the Asus DSEB-DG Board

    http://ca.asus.com/products.aspx?l1=9&l2=39&l3=299&l4=0&model=1988&modelmenu=1

     

    If your looking for a more value product.. minuse the dual slots etc....  check out the P5BV-C/4L

    http://ca.asus.com/products.aspx?l1=9&l2=39&l3=352&l4=0&model=1922&modelmenu=1

    I thinks it's probably a 3xxx series xeon chip for this board... they're nice and cheap i'm sure.

     

    you can get the latter for like 140 bucks oem if you don't mind downloading the manuals... ewwww.. Smile

     

    Anyways.. go here to check out all the asus server products

    http://ca.asus.com/products.aspx?l1=9&l2=39

     

    I see a lot of people asking about w 2k3 drivers for their products. I wouldn't expect most manufacturers of home based, or enthousiast based motherboard products to support 2k3 server etc...

     

    Most of them would say, just buy a server based board... that makes the most sense.

     

    Plus, these boards are more optimised for server style envrionments / cases / they run much lower power requirements etc...

     

    Any Intel server boards, or asus will have proper drivers... plus no need to install a cheap graphics card and wonder whether it has drivers... all of these boards have on boards gfx, and 2k3 drivers for them... doesn't steel one of your slots either... and if your not concerned about throughput of raid systems, they all have onboard raid and at least 6 connectors... (Note: I'm not recommending the on board raid, but it is there, and any of the onboard raid products with the intel chipsets will do okay for many people)

     

    thats all...

    have fun.

    dave

    Friday, February 22, 2008 11:38 AM
  • I did a sempron system with a GP drive, still got 55-60 watts.  Pulling the DVD power only gave me a couple watts.  I don't think anything under 50 would really be usable.
    Saturday, February 23, 2008 2:47 AM
  • The raid discussion was interesting. Too bad it doesn't continue.

    I say big deal if you want to use it and MS doesn't want to support it. You come up with a way to fix the MBR problem and you'll have a dandy product to sell to rich folks - well, except for expandibility. Anecdotally, I started with two 500 GB data drives that I thought would last quite a while. Then I started ripping dvd's I had lying around and had to buy another 500 within a few months. Now I'm out of room again and have to buy another. This is just storing the stuff I had around, not taking into account any future purchases of dvd's and cd's. Your raid method doesn't let you expand your array piecemeal like that, does it, ad infinitum?

    I thought your argument would have been stronger if you'd have delivered some real world examples of how the extra speed helped you but your posts didn't address that. You yourself wondered why some guy wanted to stream HD content to five computers. Would your raid setup do that? If so, you ought to contact him. He's got a problem, you've got a solution - well, except for that 2 TB limit. I think he said he wanted 15 TB.

    So, except for expandibility, 2 TB limit, cost ($500?!?!?!), and no showing that the extra speed actually does you any good, I'd say this raid method is pretty cool. Oh, yeah, and it's not supported.
    Monday, April 7, 2008 11:36 PM
  • John --

    I, for one, am very appreciative of your articulate, demonstrative posts re: RAID implementation under Windows Home Server. I have been running RAID 1 (OK, mirroring is not "true" RAID) on a kluged XP machine which I have been using as a home server. I am using a relatively inexpensive RAID card -- the Promise FASTRAK TX 4310. (Link: http://www.promise.com/product/product_detail_eng.asp?product_id=165).

    Prior to reading your posts, I had intended to reconfigure the card to run the disks simply as JBOD, with WHS "managing" them. However, I am now inclined to stick with RAID.

    My question is: Which RAID level would you recommend? My card will not do RAID 6, so I am left with the single drive failure tolerance of RAID 5 (3 drives), or I stick with RAID 1 (2 drives) or, finally, I use 4 drives in a "RAID 10" array.

    Any guidance will be greatly appreciated. And scr_w the folks who don't like to hear about it. :)

    Thank you.

    Mark Shneour
    Thursday, October 2, 2008 10:57 PM
  • I would be astounded if in two years you don't see HP and the other biggies shipping RAID WHS systems.  It is pretty easy to do a "flashing red light on the drive means that disk needs to be replaced" thing.  Nothing complex about that.  The drivers take over and integrate the newly installed drive into the system.  Really pretty darned "grandma proof".
    I came across this post by accident today - given that it's now 2 years since this thread took place, how do you feel the RAID vs DE debate has played out? AFAIK, HP etc aren't using RAID for WHS solutions, presumably because of the need to educate users about matching drives when upgrading/replacing, whereas with DE you can just slam in (gently of course!) whatever you have sitting around,
    Thursday, February 4, 2010 2:17 AM
  • Interresting.. :)

    Anyway my contribution to the thread:
    I have been using RAID-5 on my WHS for at least 14 month now without any problem at all.
    All the shares are set to non-duplicated. And all the discs acts like one big volyme for the server

    Well, I needed to replace one of the disc due to disc failure, when the new disc went in, the raid controller fixed it and stopped complaining
    No lost data, no lost backup, so I'm happy with RAID.
    Tuesday, February 9, 2010 6:25 PM
  • in my experience raid is only useful for installations where risk of disc failure is greater than risk of power failure. almost everone i know thats installed raid in a home / easily-accessible environment rapidly finds their pw wont boot and their entire raid array useless after the first power failure / reset.
    Wednesday, November 30, 2011 1:27 PM