Sign in
Microsoft.com
 
Microsoft
 
 
 

locked
My Story about Migration from WHS v1 to WHS 2011 (long read but hopefully helpful) RRS feed

 > 

  • General discussion

  • Discussion
    Sign in to vote
    3
    Sign in to vote

    I thought I'd share my experience upgrading from WHS v1 to WHS 2011, hopefully help others determining how to set theirs up as well, and learn from my mistakes or just information I gathered along the way. I'm an experienced PC user, but not so much in networking or server storage. I've used WHS v1 since not much after its inception, probably sometime in early 2008. It's had its share of issues, but usually minor, and used it for PC backups, data storage, and media streaming. My biggest issue was the performance. Having a gigabit ethernet, it was just a shame that most file transfers were limited to less than 25 MB/s, part of that I attributed to the WD Green drives. Well since I have started using WHS 2011, using these same 2TB WD Green drives, I get consistent 80-100 MB/s read and write. Getting the proper setup required more planning and headache than I expected, but in the end I think it was a good choice.

    My overall goal was to spend as little money or no money using components I already had. Thankfully I had enough hardware to configure a second server. Not a huge upgrade over my WHS v1 but still I think a better choice. The final configuration I opted for uses WHS 2011 as my main server and backup to WHS v1 with folder duplication on for most important files. Why use WHS v1? Well because it was available and I didn't have to fork over more money for a four bay external NAS or general USB enclosure.

    I will explain in more detail my reasons for what I did later, but first, here is what my setup ultimately ended up as:

    My Hardware:
    WHS v1: 2GHz Athlon64 X2 AM2, 2GB DDR, ASRock motherboard with six SATA II connectors
    WHS 2011: 3GHz Core 2 Duo, 4GB DDR2 800, Gigabyte motherboard with six SATA II connectors through ICH9R + SiI 3114 4-port PCI controller
    Hard Drives: 2x1TB WD Caviar Black 7200RPM for boot drives, 7x2TB WD Caviar Green EADS, 2x2TB WD Caviar Green EARS, 2x1TB WD Caviar Green EADS = 20TB overall for storage and backup

    Current Needs:
    - PC backups: currently 700GB, could be up to 1.5TB with current storage in clients
    - Media Files (video, music, games, etc): currently about 1.5TB, grows slowly
    - Data / Important files: About 500GB, about 100GB I consider priceless like family photos and such

    My Configuration:
    WHS 2011:
    - All EADS hard drives, determined clean with a full format and chkdsk surface scan
    - Boot Drive: 1TB 7200RPM WD Caviar Black
    - Data: 2x2TB (2TB storage) WHS Dynamic Disks RAID 1 (Documents, Photos, personal files, etc)
    - Media: 2x2TB (4TB Storage) WHS Dynamic Disks SPAN (videos, music, game files, software iso's/install files, etc)
    - PC Backups: 1x2TB (2TB Storage) Single MBR Disk
    - Script created to use robocopy /mir command to backup to WHS v1 middle of night (2:30am) and create LOG file date/time stamped
    - Add-ins: NONE
    - Windows Apps: robocopy, task scheduler
    - Third party software: WOLCMD (http://www.depicus.com/wake-on-lan/wake-on-lan-cmd.aspx) for WHS v1 Wake on Lan

    WHS v1:
    - Boot Drive: 1TB 7200RPM WD Caviar Black
    - Storage Pool: 2x2TB EADS + 2x2TB EARS, determined clean with a full format and chkdsk surface scan
    - Created file structure same as WHS 2011 for simple mirror data backup
    - Backup DATA and MEDIA folders to WHS v1, PC Backups remain on WHS 2011 only with no backups. Use Task Scheduler to wake WHS v1 at 12 midnight and sleep at 6am to give time for folder duplication and cleanup.
    - Folder duplication for "DATA", and no duplication for "MEDIA"
    - Add-ins: None
    - Windows Apps: Task Scheduler, Windows Commands cmd.exe for wake up a rundll32.exe for sleep (more on that later)
    - Third party software: NONE

    Backup frequency:
    - Once Daily with easy option for on demand.

    STORAGE STRATEGY:
    First of all, I quickly realized that I know a lot less about RAID than I thought I did. I just figured, hey, let's run a RAID 5 array, no problem. Well despite what I had read I did not want to believe it, I learned the hard way that WD Green drives are not designed to be used in any RAID configuration other than RAID 1 (Mirror). Enterprise class drives are required for proper RAID configuration and use, which are cost prohibitive and since I already had 9 2TB green drives, and 2 1TB Green drives, I wasn't about to run out and invest $4000+ in enterprise class drives for proper RAID. While RAID 1 offers full redundancy without any fancy parity calculations, RAID 1 is also expensive for large amounts of data with a backup, since you need three times the number of hard drives as data you're storing, two for the RAID mirror, and one for the backup.

    While real-time redundancy (i.e. RAID 5 or 6) would be great for all my data, after reading lots of posts from experienced users and experts at various forums had convinced me that isn't really necessary for home use. Their reasoning is simple. Most home users don't have enough data to require real-time redundancy, and the data that they do is usually small and can be done with a small RAID 1 setup, not to mention most likely a backup on their local PC if a server drive fails.

    I spent quite a bit of time investigating three RAID strategies: use of a hardware RAID controller (one with onboard processing, RAM, battery), using Windows RAID, or a soft RAID controller (controls data, but uses system CPU and RAM for RAID). None of them were pretty solutions. A true hardware controller was quite expensive, starting at $450 for an 8 port option, not to mention possible timeout issues with lack of proper TLER in the WD Green drives. Windows RAID would always be in "Resynching" mode with the massive drive volumes I'd use, so data would never truly be 100% duplicated or safe, so kind of defeats the purpose. And soft RAID controllers weren't much better than Windows, probably a bit faster in building and rebuilding the array but still would result in TLER issues with the crappy Green drives I own.

    So my strategy resulted in using Windows to manage all drive aspects. Dynamic Drives are a wonderful feature. On the WHS 2011 server, I have RAID 1 mirror for full redundancy of my DATA folders, i.e. files that are of most importance to me. Since my storage needs here are small, currently not exceeding 500GB, two 2TB drives in RAID 1 should satisfy my needs for some time to come. My most important data, like personal family photos, videos, tax documents, etc comprise less than 100GB and have that backed up on two additional USB slim external hard drives that I store at my parents house with occasional backups alternating the drives back and forth when I visit (2-3 times a month).

    My Media volume is comprised of two 2TB drives using WHS Spanned Volumes feature, this allows the system to see the two drives as a single large drive. For media, data like videos, music, game files, iso's etc, aren't as critical and the data isn't updated all that frequently. But it gets backed up to the WHS v1 daily anyhow.

    For PC Backups, notice I stated no backup for my client PC backups? Well my backups ARE my local PC's with System Restore enabled. Chances of my server and client PC's getting destroyed simultaneously would basically amount to my house blowing up, in which case I have bigger issues.

    This in total is five 2TB drives. I installed a PCI SATA controller card to run my boot drive from so that I could leave one onboard SATA controller free for future expansion.

    On the WHS v1 "backup" server, I have a 1TB 7200RPM WD Caviar Black drive as the boot drive, along with the four remaining 2TB WD Green drives (2 EADS, 2 EARS with jumper to turn off 4k advanced format) plus a 1TB WD Caviar Green for 9TB in total storage. I turn folder duplication on for my important "DATA" folders, no duplication for my "MEDIA" folders, and no backup of client PC backups staged on the WHS 2011. So effectively, I should have adequate backup space for everything on my WHS 2011 even with duplication of my important DATA volume folders.


    SLEEP AND WAKE UP!
    Setting up the link between WHS 2011 and WHS v1 wasn't as simple as I hoped. I didn't want WHS v1 running 24/7 since it was required for backup purposes only. Although, the nice thing about having WHS v1 as a backup, is that it can act as an actual server in the event that my WHS 2011 goes belly up or requires extended maintenance. Well, my first step was determing how to get WHS v1 to wake up and sleep consistently. That nifty add-in "lights out" is supposed to do such things. Seemed like a simple and quick solution. Well, Lights Out didn't always sleep my server, and couldn't find out why. It was getting frustrating so I sought an additional avenue. Thankfully there are simple command line utilites that can be used to achieve such goals using Task Scheduler (or "scheduled tasks" as it's called in XP / Server 2003 / WHS v1).

    Simply put, you can achieve the basics of Lights Out using built-in windows commands. Here's what I use. Task Scheduler is a great utility to get familiar with. I won't go into detail how to use it because there's tons of examples out on the web.

    To sleep the PC:
    Command line: C:\WINDOWS\system32\rundll32.exe powrprof.dll,SetSuspendState 0,1,0
    Just set this command in "scheduled tasks" for the time you want your PC to sleep. Be sure to also make sure the option is NOT selected for the user to be logged on in order for the task to happen. Otherwise if you aren't logged in, it won't run.

    To wake the PC:
    Command line: cmd.exe /c
    That command basically just does nothing, but setting this in a scheduled task with the option to "wake the computer to run this task" is enough to wake it up whenever needed.

    Schedule:
    WHS 2011 is on 24/7, since I need access to it at any given time, and at night when I sleep, it's busy doing a backup to my WHS v1 once every night.

    WHS v1 is being used primarily for backups so I don't need it on 24/7. But since it needs time to migrate files with Drive Extender and do an occasional cleanup and other tasks, I have it turned on from 12am (midnight) to 6am using the above task scheduler commands.


    BACKING UP DATA:
    This is obviously the important part. I started with Microsoft's SyncToy but it frequently would hang the system for extended periods on a specific file or folder for no apparent reason. Thankfully a command line utility 'robocopy' is included in Windows and is a very robust file backup and sync program with logging features. After SyncToy failed me I chose to give Robocopy a chance, and glad I did. If you're comfortable with command line scripting, it's a great tool, and actually pretty straightforward and easy to use.

    First here is my backup script and will try to dissect it from top to bottom:

    --------------------------

    REM Send Wake on Lan command (third party app) in case backup server is asleep
    wolcmd.exe 00ABCDEFG012 255.255.255.255 255.255.255.255

    REM Give WHS v1 time to wake up
    timeout 300

    REM Save time hour to determine if one or two digits
    set hournum=%time:~0,2%

    REM Set up filename depending on one or two digits
    if %hournum% gtr 9 (set timelog=whsbackuplog-%date:~10,4%-%date:~4,2%-%date:~7,2%_%time:~0,2%h%time:~3,2%m%time:~6,2%s) else (set timelog=whsbackuplog-%date:~10,4%-%date:~4,2%-%date:~7,2%_0%time:~1,1%h%time:~3,2%m%time:~6,2%s)

    REM Use robocopy to backup folder files
    robocopy "\\whs\archive" "\\whsbackup\archive" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20
    robocopy "\\whs\documents" "\\whsbackup\documents" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20
    robocopy "\\whs\game files" "\\whsbackup\game files" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20
    robocopy "\\whs\music" "\\whsbackup\music" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20
    robocopy "\\whs\pictures" "\\whsbackup\photos" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20
    robocopy "\\whs\public" "\\whsbackup\public" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20
    robocopy "\\whs\recorded tv" "\\whsbackup\recorded tv" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20
    robocopy "\\whs\software" "\\whsbackup\software" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20
    robocopy "\\whs\users" "\\whsbackup\users" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20
    robocopy "\\whs\videos" "\\whsbackup\videos" /mir /r:2 /w:20 /log+:%timelog%.txt /nfl /tee /ndl
    timeout 20

    --------------------------

    My WHS 2011 server name is 'whs' and WHS v1 name is 'whsbackup' (how original right?).

    First I found this command line utility WOLCMD.exe that allows you wake a PC on your LAN or remotely. On the same LAN it's as simple as setting the MAC address followed by the LAN broadcast address 255.255.255.255 twice. Just leave it at that without further explanation, it works. Just make sure the network card of the MAC address you're trying to wake is set to "magic packet" only and not "pattern match & magic packet".

    Second I send the timeout command (timeout 300) of five minutes to give the server time to wake up and grind the hard drives for a bit.

    Next is setting up the log file name with a time stamp. This ended up being a lot more complicated than it should be. But in the end there is a way. At first I just had it create a log file and then in the script renamed the log file when everything was done. For some reason, when running this logged out, it would never rename the file. That one threw me for a loop, so instead of banging my head against the wall, I went forward and just used a variable to generate the log file name based on date and time and set the log file name from the beginning.

    The trick here is understanding the syntax of date and time. Type 'date' (without quotes) in your command prompt and you'll get 'Sun 03/11/2012' for example. The %date:~10,4% you see just notes the starting point of the date and how many charcters after to include. Count from first character as 0, 10th character would be 2 from 2012 in the date, followed by the next four charaters which would be the full four year date 2012. Time uses the same type of procedure, however it's a bit more tricky because it does not use a 0 in front of hours less than 10 which a space can cause some issues. So I use a bit a trickery using if then with greater than (gtr) operation. First I determine the hour number and assigning it to the variable hournum (=%time:~0,2%). Then I use the 'if %hournum% gtr 9' operator which in plain english says 'if hournum is greater than 9 then we will use two hour charcters, otherwise we will use one character and artificially add the 0 ourselves. Note the _0%time:~1,1% after the 'else' statement, that 0 character is added there.

    The filename is now stored in the 'timelog' variable.

    Now that the log file name has been established we can use it now while using robocopy to backup / sync our server files.

    Robocopy command has an automatic Mirror option: robocopy.exe <source> <destination> /mir

    Simple enough. However what they don't tell you is that if a file fails to copy, it defaults to one MILLION retries (yes you can put your pinky at your mouth if you wish). One million!? Really who thought of that. So using the /r:x where x is the number of retries (I use 2 in case that file is actually captured by windows or another user for whatever reason) will save you there. It defaults to a 30 second wait time which isn't horrible, but I set mine to 20 by using the /w:20 flag.

    Finally logging. Robocopy can log every little thing it does, or you can restrict it to show only summary along with any failures or deletes it made. The command line flags /nfl /ndl will basically not show you files or folders it inspects for change unless there is an error. /tee just brings up the command window so you can view progress.

    To actually log the command, you can use the /log:<filename> or /log+:<filename> with the latter used to append to the log. Since I run a script to robocopy multiple directories, I use the log+ feature.

    If you notice I use timeout 20 after every robocopy command in case I happen to want to watch progress and view each main folder result.

    I created a folder on my WHS 2011 server called 'backup' on the c: drive that houses the above script in a 'whsbackup.bat' file along with the wolcmd.exe utility. All log files are stored in this directory.

    I'm not saying this is the best strategy for everyone, and not even saying it's a good one, but it's one that works for me. Hopefully this will help other users in deciding how to best set up their home server needs. Of course it goes without saying, test your setup and procedures thoroughly first and make sure you have a solid backup of your files before attempting any upgrade or file transfer to a new server.


    • Edited by MarkBx Sunday, March 11, 2012 11:29 PM
    • Changed type Ken Warren Monday, March 12, 2012 2:58 AM Not a question...
    Sunday, March 11, 2012 11:16 PM