locked
HELP with RAID Suggestions - WHS 2011 and Backup!? Use Dynamic Disks RAID? RRS feed

  • Question

  • I have been using WHS v1 for years now. Over that time I have accumulated 9 x 2TB WD Green drives variations EADS and EARS (4k turned off with jumper). I would like to move on to WHS 2011, and have removed all but one 2TB drive from my v1 system and am using a hodge podge of 1TB, 640GB, and many 320GB drives to take up the slack (thank you DE!) while I try to get my WHS 2011 up and running.

    Obviously without DE in WHS 2011, RAID is the best possible solution for data redundancy, but I find RAID 1 is simply a waste of drive space especially considering I have a regular backup, so RAID 5 seems the best solution. Considering these are all WD Green drives, I am reluctant to run them in hardware RAID 5 mode though from all the nightmares I've heard due to lack of TLER. Initially I have it set up in RAID 5 with six 2TB drives for 10TB storage, which is the amount of space I would prefer, using my onboard RAID controller and went without a hitch and performance seems pretty good. However I have heard that using Windows Software dynamic disk RAID 5 would actually be the safest route since Windows is aware and controlling the RAID? I know performance won't be stellar, but I use my WHS for data, video, music storage and streaming, so don't really need 100MB+ transfers (probably wouldn't get that with Green anyhow).

    In addition, I was going to buy one more 2TB WD Green drive (EARX) for a total of four 2TB drives for backup. Again, RAID 5 seems the most sensible solution. But the consumer level RAID cards I guess have horrible performance. That being said, can I set up the external storage using Windows dynamic disks in a RAID 5 configuration? I am really trying to limit costs, but prefer 10TB storage with 6TB backup without having to manually manage backup disks if I were to just backup to individual drives.

    This has become quite overwhelming. Thanks for any help.

    BTW, my home server is Core 2 Duo E6850 3GHz, 4GB DDR2 800, on a Gigabyte G33M motherboard and ICH9R controller.

    I guess the way I see it WHS DE was basically a software based JBOD controller with redundancy option (duplicate folders) and Dynamic Disk RAID should perform as well as if not better? Maybe I'm not looking at it right though.

    edit: GAH! Just did some more reading and you can't backup more than 2TB !? Has this been fixed?

    Thanks.



    • Edited by MarkBx Monday, March 5, 2012 1:58 PM
    Monday, March 5, 2012 12:38 PM

All replies

  • Backups: no, not fixed. A realistic answer would be that it probably won't be fixed in Windows Home Server 2011, because the backup tool, and therefore the limitation, is inherited from Windows Server 2008 R2, and I don't see Microsoft investing much in the way of resources into this issue (Windows 8, including Windows Server 8, is probably no more than a year from release at this point). There is a workaround, which requires you to configure multiple backup jobs manually via command line tools and scheduled tasks (read up on wbadmin for more details on that), but it's not particularly convenient. Note that there is no consumer grade backup tool which deals gracefully with extremely large volumes of data; backing up multiple terabytes of data regularly is a very hard problem to solve for a price a consumer will be willing to live with.

    As for RAID vs JBOD, and hardware vs. software RAID, there are lots of tradeoffs no matter which direction you go in and this forum isn't a good place to learn about them (everyone here has some sort of axe to grind, including me :) ). Starting on wikipedia isn't the worst idea in the world, to be honest. If you absolutely must have a single large pool of storage, and you have a backup strategy already in place to handle it (so you won't care about server backup), I would recommend sticking with Windows JBOD, and make sure backups were working and you can restore the entire server.

    Finally: Practice your disaster recovery plan! A plan that hasn't been tested may be flawed in a way that results in complete data loss!


    I'm not on the WHS team, I just post a lot. :)

    Monday, March 5, 2012 2:59 PM
    Moderator
  • Thanks. How about Windows RAID 5? It'd be nice to have some sort of redundancy. But is TLER really an issue, and is Windows dynamic disks RAID a better option over an onboard RAID controller?

    And backups seem to be just as problematic then, requiring individual drives for backup. I am considering an enclosure that manages JBOD for backups, but that doesn't seem like the most robust solution either.


    edit: I'm finding that drives that large will result in a constant "resynching". Is there not a good overall solution for this other than using the drives independently?
    • Edited by MarkBx Monday, March 5, 2012 4:00 PM
    Monday, March 5, 2012 3:11 PM
  • As I said above, this forum is not the place to gain an education in the tradeoffs inherent in different storage technologies, data redundancy, etc. I will say that I don't recommend Windows RAID to anyone because of performance and recovery issues (recovering a degraded array can be much more difficult than Microsoft would have you believe). I don't recommend consumer level "hardware" RAID solutions for much the same reason, with the exception of JBOD assuming appropriate backup solution. And I don't recommend other hardware solutions (which are usually accelerated and thus true "hardware" RAID) with anything other than enterprise drives.

    As for backups, I'm completely serious when I say that it's a very hard problem to solve for consumers. Consumers want to have huge amounts of data at their fingertips (media = huge data volume pretty much by definition), but the costs to preserve that data are likely to be enormous. If you have data in the tens of terabytes, backing it up, protecting those backups against most disasters, and restoring that volume of data, are all likely to be more expensive and harder to manage than consumers can honestly deal with. 


    I'm not on the WHS team, I just post a lot. :)

    Monday, March 5, 2012 6:12 PM
    Moderator
  • Thanks. I currently have about 3TB worth of "DATA" and about 1TB of used hard drive space in total on client PC's to backup regularly. I guess I'll just do a Windows dynamic disk JBOD and some external enclosure that supports its own JBOD, or some other auto backup system that will span across disks. My most important stuff (primarily personal photos/videos) are in quadruplet backup at least, but that's only like 50GB or so. Just a lot of the other stuff would be difficult to recreate or spend an enormous amount of time doing so. If I had 4TB drives, I could just use two of those and mirror that, with dual 4TB for backup. Six 4TB drives and I'm done! Right now that would cost me $380 x 6 = $2280!!! Or Use the 9 2TB drives I have and mirror 4 x 4 for 8TB and buy three more 2TB drives for backup.

    Hard drive prices can't come back down soon enough. I bought all my 2TB drives for less than $100, many at $79 iirc.

    I did start to look at actual RAID cards, and they start at $450. Not to mention my green drives are still really the culprit. I loved Windows Drive Extender. Maybe I will just make it work for me a little bit longer.

    Out of curiosity, do you know what the difference is with the HighPoint cards with the "SGL" vs. non "SGL" versions? The non "SGL" versions are like twice the price, but nothing at highpoint's site indicates the difference.

    • Edited by MarkBx Tuesday, March 6, 2012 6:37 PM
    Tuesday, March 6, 2012 6:22 PM
  • I recommend avoiding RAID, particularly RAID 5. The main reason to use RAID in an enterprise context is for high-availability and redundancy; it is not a data backup solution.  In consumer products, RAID injects an added level of complexity and introduces additional points of failure into a system.  In most cases, consumers can handle a little down time if a drive fails, and recoving from a backup is usually a straightforward process. There is also no risk of data loss if your motherboard or RAID controller fails.

    I have large media collections that greatly exceed the 2TB limits in WHS 2011.  My strategy is to keep my critical shared files (work related items, family photo collections, etc.) on a single 2TB server volume that participates in the scheduled server backup (twice daily).  I also keep copies of this data on a separate external drive that I sync from time to time, or before I need to do any server maintenance.

    I have a separate 4Tb spanned volume to handle my client PC backups. There is no redundancy for this data, other than the client PCs themselves.

    All other server data (Blu-Ray and DVD collections, backup of backups, etc.) I keep on a 6Tb spanned volume.  This data is synced automatically to a separate NAS (I use my old WHS v1 box with folder duplication ON) on the network via a nighly scheduled task (I happen to use SyncToy, but there are other methods).  Since this type of "backup" does not allow for recovering older versions of files, I also make sure to enable shadow copies on the spanned volume.  I use Lights Out on both servers to manage power usage rather than have both machines spinning 24 hours a day.

    Of course, the use of spanned volumes adds risk.  It is safer to spread your shares out onto multiple 2Tb volumes, but as I have a "live" backup of the data, I feel the risk is acceptable for the added convenience.  I have to use a spanned volume for my client backup disk, due to the amount of client data that I have.

    Regardless, using a second NAS for backup and initially copying multiple terrabytes of data over even a gigabit LAN can take hours or even days, so be prepared for this when migrating your server.



    • Edited by Gary Voth Tuesday, March 6, 2012 6:49 PM
    Tuesday, March 6, 2012 6:48 PM
  • Thanks for sharing your strategy. This is a lot more complicated than I thought it should be.

    I'd like to prevent running two servers for many reasons, although I had considered keeping the v1 box going as well, but just not practical for me. But yes, my biggest issue will be my media because it is by far the largest amount of data I have, although it currently doesn't exceed 2TB, but getting close.

    I have 7 2TB EADS, 2 2TB EARS, 2 1TB EADS drives that I can use. I'd prefer to buy as little equipment as possible. I guess I'll rethink a spanned volume strategy with backups somehow.

    And PC backups, I guess that's true. No need for redundancy or even backup. I don't even need six months worth of backups. If there was ever an issue, 99.999% of the time I will take the most recent backup and restore. I just want a simple strategy for keeping my data safe and easy to recreate should there be trouble.

    Perhaps I can do a RAID 1 2x2TB = 2TB for "data" or shared storage (documents, photos, etc), 2x2TB = 4TB spanned for "media", 1x2TB for client PC backups (shouldn't exceed 1TB for now) = 5 drives

    Then 4-bay external NAS or USB backup using the remaining 2TB drives to backup to, although it still doesn't fix the 4TB spanned media with 2TB backup issue unless I use a third party sync app (like your SyncToy), which really shouldn't be an problem. Maybe throw in a 1TB for the Public shared drive (that I let me friends and family use) This should last me a while...

    I am in the process of backing up my data to 2x2TB externals (using 2x2TB EARS) and yes taking a LONG time. I just want to get this thing built and done with! ;)

    edit: Hmm, the more I think about it, maybe I will keep my WHS v1 running to use for a nightly backup using "Lights Out" so it's only running a couple hours a day at most, and use my two 1TB drives in my two bay external USB for backing up my "data" on a regular basis.

    • Edited by MarkBx Tuesday, March 6, 2012 7:31 PM
    Tuesday, March 6, 2012 7:21 PM
  • <MarkBx> wrote:
    Re:  edit: Hmm, the more I think about it, maybe I will keep my WHS v1 running to use for a nightly backup using "Lights Out" so it's only running a couple hours a day at most, and use my two 1TB drives in my two bay external USB for backing up my "data" on a regular basis.

    Why not stay with WHS v1?  WHS v1 allows server data backup of greater than 2 TB's with drive spanning that is easily set up in the server backup wizard.  I have two 4 bay Direct Attached Storage (DAS) eSata enclosures that I rotate.  Choose which shared folders backup to which drive and tick the box for "Save settings for future backups" and you're done.  Granted, you have to run the backups manually and the first backup to each set of drives can take a long time, but future backups are incremental and run quite quickly.  DE allows data duplication on the server, so no need for any RAID solution.  What does WHS 2011 give you that you don't have in WHS v1? If there is something in WHS 2011 that you need then go with it, but if WHS v1 gives you everything you need, then I would stay with it.  I run a WHS 2011 test box, but my real world data and backups are still entrusted to WHS v1.


    ____________

    BullDawg
    In God We Trust
    ____________


    BullDawg
    Wednesday, March 7, 2012 2:25 AM
  • I considered sticking with v1, but for whatever reason transfer speeds start great at first but greatly diminish over time, like on the order of 10-12MB/s or less. I find DE to be great for easy expansion, but it seems to also constantly tax my machine (DEMigrator.exe primarily), cause occasional hangs and a bit finicky. I get occasional errors (asks to sent to MS), most of it related to disk I/O, despite all drives having clean SMART health and clean chkdsk surface scans, although never had any affect on my data integrity. I even upgraded my v1 system a while back from the older setup I had, and the niggling little issues and errors kept coming back. Even tried a PCI-e SATA card to eliminate possible issues with the onboard SATA, but no luck. WHS v1 has worked for me, just not ideal. Since I use it primarily as a data backup, with occasional streaming, it's worked, at least within tolerable levels. So far my test WHS 2011 box has run great, and even now transferring data to my now production WHS 2011 box, it is performing quite well.

    I think as a backup NAS more or less as suggested, WHS v1 should work great since I will never be accessing the data there directly for streaming or data transfers (well except data backup and recovery). I'll do another fresh install of WHS v1 on that box and hopefully it will run reasonably well as a backup server.

    Wednesday, March 7, 2012 3:39 PM
  • Fair enough.  Whatever meets your needs and satisfies your desires.


    ____________

    BullDawg
    In God We Trust
    ____________
    <MarkBx> wrote in message news:e2e8cb5c-a196-4429-9225-520109056e03@communitybridge.codeplex.com...

    I considered sticking with v1, but for whatever reason transfer speeds start great at first but greatly diminish over time, like on the order of 10-12MB/s or less. I find DE to be great for easy expansion, but it seems to also constantly tax my machine (DEMigrator.exe primarily), cause occasional hangs and a bit finicky. I get occasional errors (asks to sent to MS), most of it related to disk I/O, despite all drives having clean SMART health and clean chkdsk surface scans, although never had any affect on my data integrity. I even upgraded my v1 system a while back from the older setup I had, and the niggling little issues and errors kept coming back. Even tried a PCI-e SATA card to eliminate possible issues with the onboard SATA, but no luck. WHS v1 has worked for me, just not ideal. Since I use it primarily as a data backup, with occasional streaming, it's worked, at least within tolerable levels. So far my test WHS 2011 box has run great, and even now transferring data to my now production WHS 2011 box, it is performing quite well.

    I think as a backup NAS more or less as suggested, WHS v1 should work great since I will never be accessing the data there directly for streaming or data transfers (well except data backup and recovery). I'll do another fresh install of WHS v1 on that box and hopefully it will run reasonably well as a backup server.


    BullDawg
    Wednesday, March 7, 2012 4:06 PM
  • Well, getting things just about going with my WHS 2011 with v1 as backup. So far so good. Although I'm looking into the Stablebit DrivePool add-in which looks like a good Drive Extender replacement. I'll put it through its paces in VM's, but I realize until you actually run anything with your production large volume drives, you don't get the full picture. But worth a shot anyhow. Anyone recommend a good external hard drive enclosure? I'm looking at the Rosewill RSV-S8, although the included card seems to have a bit to be desired, but if DrivePool is managing the drives instead of any kind of JBOD or RAID, then it should be ok I think.
    Thursday, March 8, 2012 11:24 PM
  • I use the Mediasonic ProBox, mine is the USB 2 model as the USB 3 model was not in production when I bought mine a few years ago.

    http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&DEPA=0&Order=BESTMATCH&N=-1&isNodeId=1&Description=mediasonic

    Keep in mind, if you use the eSata port, the eSata port must be port multiplier aware.  Mine is not, so I bought a Silicon Image 3124 card ($30-40) that has 2 internal and 2 external ports.  Or if you prefer, go with the USB 3 model.  The performance with USB 3 is not quite as fast as eSata, but much better than USB 2; assuming your computer has a USB 3 port.

    Mediasonic also has an 8 bay unit.

    http://www.newegg.com/Product/Product.aspx?Item=N82E16817576012

    I have no experience with the Rosewill model you mentioned, but I do have a few Rosewill add-in cards that work well.  The Rosewill model does have some pretty good specs.


    ____________

    BullDawg
    In God We Trust
    ____________
    <MarkBx> wrote in message news:8fdf4861-b77b-407f-b621-1106841ec50c@communitybridge.codeplex.com...

    Well, getting things just about going with my WHS 2011 with v1 as backup. So far so good. Although I'm looking into the Stablebit DrivePool add-in which looks like a good Drive Extender replacement. I'll put it through its paces in VM's, but I realize until you actually run anything with your production large volume drives, you don't get the full picture. But worth a shot anyhow. Anyone recommend a good external hard drive enclosure? I'm looking at the Rosewill RSV-S8, although the included card seems to have a bit to be desired, but if DrivePool is managing the drives instead of any kind of JBOD or RAID, then it should be ok I think.


    BullDawg
    Friday, March 9, 2012 12:33 AM