locked
WHS Restore -> SLOW! Estimated 3 hrs., now 8 hrs. and counting... RRS feed

  • Question

  • Like the title says, the restore is unbelievably slow!  I checked my Disk Management Plugin, and only one disk out of two data drives is being hit and only at about 1700 KB/s.  I am running gigabit ethernet, and checked that to confirm no bottle neck there.  And finally, the WHS processor is mostly idle, hitting 10% at most.  I am restoring to a Dell M4400 with a drive that is about 300GB, with only about 150GB used.

    That is going on?  This is almost laughable performance; I don't mean to complain, but do restores really take this long?
    Friday, January 15, 2010 12:07 PM

Answers

  • You did not tell us too much about your network connection, but even with GigaBit Ethernet and your client server connection the only active connection for the server your "only" 150 GByte will take considerable time (although I would assume a 100 MBit connection from your given data - is each part of the chain GigaBit compatible and is the server really connecting with GigaBit speed, since Windows Server 2003 checks the quality and will degrade the speed, if the physical connection is not good enough for high speed). And we all know, that GigaBit Ethernet often is not the same, depending from local situation, switch, network hardware, network traffic from other PCs drivers etc.

    Also the backup database is stored in large chunk files, often on one of the server disks. So it is not strange, that only one disk is touched. Since the information needs to be rebuilt from the backup database (which only stores clusters and the references which clusters are stored on which PCs and belonging to which data to make the single instance storage of backups for multiple PCs possible), this will involve the performance of the disk, the amount of memory, the power of the CPU on the server as well (one of the few scenarios, in which these factors really might matter). Again, with the lack of information hard to tell, what could be the bottleneck in your case.

    Best greetings from Germany
    Olaf
    Friday, January 15, 2010 1:32 PM
    Moderator

All replies

  • You did not tell us too much about your network connection, but even with GigaBit Ethernet and your client server connection the only active connection for the server your "only" 150 GByte will take considerable time (although I would assume a 100 MBit connection from your given data - is each part of the chain GigaBit compatible and is the server really connecting with GigaBit speed, since Windows Server 2003 checks the quality and will degrade the speed, if the physical connection is not good enough for high speed). And we all know, that GigaBit Ethernet often is not the same, depending from local situation, switch, network hardware, network traffic from other PCs drivers etc.

    Also the backup database is stored in large chunk files, often on one of the server disks. So it is not strange, that only one disk is touched. Since the information needs to be rebuilt from the backup database (which only stores clusters and the references which clusters are stored on which PCs and belonging to which data to make the single instance storage of backups for multiple PCs possible), this will involve the performance of the disk, the amount of memory, the power of the CPU on the server as well (one of the few scenarios, in which these factors really might matter). Again, with the lack of information hard to tell, what could be the bottleneck in your case.

    Best greetings from Germany
    Olaf
    Friday, January 15, 2010 1:32 PM
    Moderator
  • To clarify my gigabit - it's a 25ft run on Cat 5e with Intel Pro NIC in the server and the onboard on the laptop.  I can easily transfer 8GB files between the two in a matter of minutes.

    Also, as I mentioned - neither the hard disk, the network, or CPU is being taxed during the restore process; I can easily push all three well beyond their rates and tested this during the restore process itself by copying large files to/from the server on another PC, running a virus scan, and running Prime95 (CPU).

    So your explanation simply leaves me to conclude WHS is sitting around doing nothing most of the time during the restore?  Maybe Microsoft forget to remove the "Sleep(10000)" statement somewhere in the restore code.
    Friday, January 15, 2010 1:40 PM
  • Yes, unfortunately it can take that long. I did a restore of a Win7 Ultimate a couple months ago that did take several hours. I basically had to walk away once it started and left it fore most of the day. I think it ended up taking 4 hours and there was maybe 30 Gigs of data restored. So WHS is obviously not the quickest restore method.
    http://www.wwiivehicles.com
    Friday, January 15, 2010 5:47 PM
  • Wow, okay, glad to hear some data from someone else.  So the time was not unexpected then.

    Well, seems I cannot do much about it - with nothing being highly utilized, there is nothing really to upgrade on my server.  I may look at an alternative backup solution, but WHS works okay, this was just a time pressure issue for me.  Although, I am more concerned about my other post where restoring caused my computer to be removed from the domain.  I am planning on filing a trouble ticket with Microsoft on that one.
    Saturday, January 16, 2010 2:48 AM
  • No, it's not just you. There have been hundreds, if not thousands,  of complaints about slow restore speeds and sometimes restores fail in various ways too. Personally, I believe they need to set the CPU priority above normal on the restore cd. I've seen GHOST do very slow restores if you didn't select a high enough CPU priority. Unfortunately, I've not figured out how to change it either.


    "He who dares not offend cannot be honest." -- Thomas Paine
    Saturday, January 16, 2010 4:58 AM
  • Good to know S_M_E - that makes perfect sense, although most of my CPU was idle, so I'm not sure why Windows would prefer Idle over anything else.

    I am currently looking at Acronis Server to do backups now.  I've used it without a hitch before and I'm starting to get very worried about loosing data due to WHS issues.  I see far too many threads on here about data being gone.  I am also thinking of doing a hardware RAID 1 setup to increase speed and not have to worry about all of this data loss stuff (of course I would still do a backup, likely weekly or monthly just in case of corruption).  I found a RAID card provider that says the disks can be read outside of the mirror on the card, which would be great if any disk fails.

    Anyway - thanks for the input, not sure what I am going to do yet.
    Monday, January 18, 2010 2:50 AM
  • Your post was a while ago now and I came across it while trying to find a way to speed up my home server's restoring speeds.  I had the same problem where the estimated restore time climbed up from hours to days and it looked like it really would take that long.  The solution I discovered was to use the home server pc restore's disk manager to delete and recreate and fast format the disk being restored. This is at the step where it is asking which restore partition to place on what physical partition.  Then the restore time was only a couple of hours for a 500GB disk about 50% full.
    • Proposed as answer by matthew.h Wednesday, June 15, 2011 4:59 PM
    Tuesday, March 8, 2011 7:07 PM
  • magarity,

    I definitely endorse this approach.

    Using the usual WHS client dashboard to do a restore after a disc died, I was 2 DAYS and apparently only halfway through a restore of about half a TB of data when I pulled the plug on the process. I followed your instructions booting from the restore CD and was able to do a full disc restore in about 4-5 hours.

    Another bit of advice -- anyone relying on WHS to do system backups should definitely do a test run of booting from their restore CD to ensure they can connect to their WHS server. I fumbled around a bit getting my network drivers to install -- copy them from c:\drivers for full system restore folder on the client pc's backup and put them on a USB stick -- and then load them if/when the restore CD prompts you to do so. I had the luxury of a few other computers around to gather what I needed; in a disaster situation, your options may be limited....

    Thanks again!

    -Ken

     

     

     

     

    Wednesday, March 16, 2011 11:31 PM
  • Using the PC restore disk manager to recreate the partition and format them in advance made a big difference for me as well. Thanks for the guidance. I killed the first pass after ~18 hours overnight (with a message that there were 10 days to go to finish). Afterward the restore took a few hours once I made sure that the connection between server and PC was operating at 1Gbit settings.
    Wednesday, June 15, 2011 5:02 PM
  • Well, I am now also into this bad experience. My Win7 PC crashed, it was caused by the OCZ Synapse SSD cache driver. Have removed that SSD, and now I need - for the first time in  3 years - to ecover my PC. Two drives, one with 500 GB, and the second with 1TB. I started with the smaller C drive, and it takes not ages, it takes centuries :-( The progress bar tells me 19,5 hours just for the 500 GB drive. OMG ! I am praying that it does not stop somewhere in the middle, like last night, where WHS shut down for installing patch files :-( And then about 2 days restore time for the 1 TB drive. Thats unusable slow.

    My PC is an i750, 8GB RAM, Gigabit network into the WHS server (incl Gigabit switch), and I know that I can copy large files very fast to the shared folders on my WHS. But this restore-feature is awful...

    I can really imagine where this slow speed comes from: the drive extender stores all the data packages spread across the whole disk space, and the database is highly fragmented with trillions of data records. It just take too long time to search for the right bits and restore them...

     I will look now for Acronis backup, which is a hell lot faster for disk image restore. For single file and folder recovery the WHS restore is ok. But not at all for disk paritions :-(((

    I would like to know if WHS 2011 is faster?

    Monday, May 28, 2012 10:06 AM
  • No WHS 2011 is not faster! 2 days into a 54Gb Restore, no other traffic on the network as we have been away for the weekend... it makes sense to just dump the restore and start from a different direction but I am fascinated about how long it can take so will leave it running and do something else until it's completed! I think the bottle neck is the RAM, just a guess but there appears to be a virtual disk created to extract the files to, which I think is in ram, maybe paged out to HDD as needed... but I am not about to install 54Gb Ram to prove the theory!

    My Story:- Old setup: WHS 1 (2003) I did a full PC restore onto new hardware having upgraded a PC for my wife, it was just over 500Gb, took about 2 hours, then install the right drivers for the new motherboard etc. took about 6 hours from box of components to full working restored PC.

    Then I used my Wife’s Old PC as the base for a server upgrade (dual core to quad core processor) so bought WHS2011, disaster it doesn't do the JBOD with directory mirror that was so handy on the old system...... dumped it and put WHS 1 on instead, played with it for a while..... then thought what the hell lets go with 2011 and see what it does do well. so rebuilt server now with 500Gb mirror for OS and some data (Music shared folder) and 2 x 2TB Mirrors for the data. bit of juggling of folders to fit them into appropriate volumes and it's a working system which despite the fact I will need to do more management on the disk usage than on the 2003 system we are sort of in business... except the backup!

    Now in the same car wreck fascination that means I have to let that restore finish even if it takes a week (which it looks like it may!) I watched a great concept disintegrate before my eyes, the backup in WHS is useless! 2tb partitions OK I can see that once upon a time someone said lets build to this ridiculous level nobody will ever want to backup 2Tb of data! But hey in my early days I was amazed when running a 40Mb storage (CPM server) shared between 40 odd users Server when one day my dual floppy disk machine was upgraded with a 20Mb hdd!!! 2Tb even or probably especially for home users is no longer massive photo’s and video’s killed that limit…

    Let’s keep in mind the Operative word here is HOME server so backup has to be within home budgets commercial level systems are just no use to most WHS users. So the obvious answer is backup to hard disks somehow…

    I think we are a fairly typical household in that we have a LOT of data but it doesn’t really change much or often, I don’t think Microsoft would have needed much market research to know that, now falling out of the end of my upgrade cycle is 5 1Tb hdd’s which would make an obvious backup medium so idea one was put them all in USB enclosures weirdly enough I do actually have 5 knocking around some were old ones of mine, others I rescued data from failing external disks for people and they let me keep the failed unit… but simple answer put 5 1Tb disks into the enclosures and backup to them 3 onsite being one for the bare metal, which will never go off site 2 in the workshop at the end of the garden should give me fireproof protection….

    WRONG!!! Backup schedule won’t work like that only allows one backup to be configured and it has to run daily!

    OK so I have to make an array of my disks so it can work with one backup schedule! Expensive and restrictive… BUT I do have a PC running which has space for the disks… so put them in there and raid 5 them not as good on the fire proof but it is at least in another room…….. even considered putting them in the old server case as a WHS server and running it in the workshop to give fire protection and backup all in one hit… WRONG! Backup to a shared path can only do full backup which takes about 30 hours…. And it has to run daily!

    So a bit of research later I am back to the USB enclosures and scheduling backups via command line in scheduler, not particularly difficult for me but then I have been running servers since DR DOS and CPM were the only multi-user OS’s around…

    OK so my solution:-

    Split server into 3 backups over 6 USB external drives, Bare Metal and Music, (will fit onto 500Gb easily so can press a 500Gb disk I found into service)

    Weekly backups doing Bare Metal on Mondays, Volume 1 on Tuesdays and Volume 2 on Wednesdays, not a twice a day regime but will finish easily overnight and  if a total disaster happens we will be mostly happy to have lost less than a week’s data it isn’t a commercial server after all! May swear a bit on a minor disaster but we can remember what we did last week!

    On the server open Task Scheduler and create in my case 3 new tasks something along the lines of:-

    WBADMIN START BACKUP -backupTarget:g: -include:e: -quiet

    WBADMIN START BACKUP /? Will help you find the command lines you need, set them to run as appropriate

    And you have some functional backup systems…

    My next task is to take 3 of the backup disks out of service, put them into the old server chassis and attempt a restore… an untested backup system is a pointless waste of effort.

    NOW HERE IS THE REAL PROBLEM!

    How can I sell WHS to non-technical people who regularly bring me there dead laptops and PC’s with that sad look in their eyes as they know years’ worth of family trivia is lost, these people have old hardware, and thanks to internet being supplied via routers with switches built in they have all they need for a home network so a simple server offering easy backup would have saved them so much hassle…. But they are not going to pay 100’s or 1000’s on a server or backup system, most have a USB drive or something that is supposed to be a manual backup when they remember…. Most never use it.

    As a closing thought I know what Microsoft did wrong with WHS.

    The got the Commercial team to make an interface that was friendly for WHS

    They should have used the Xbox team to make the server based backend work within the  home network.


    • Edited by UK Pete Monday, July 23, 2012 2:51 PM
    Monday, July 23, 2012 10:43 AM
  • Yes, unfortunately it can take that long. I did a restore of a Win7 Ultimate a couple months ago that did take several hours. I basically had to walk away once it started and left it fore most of the day. I think it ended up taking 4 hours and there was maybe 30 Gigs of data restored. So WHS is obviously not the quickest restore method.

    What's odd is I first really tested out a full machine restore from WHS 2011 with the Windows 8 preview - I needed to change around the hard drives in my test machine.  When I decided Windows 8 wasn't for me and decided to buy and load Windows 7 and turn the machine in to my new gaming computer, I set it up fresh, loaded my games and went on.  Then something else came up and I needed to change my hard drive layout, so based on positive experience with WHS v1 and my Windows 8 experience with 2011 I thought "no problem" - bzzt!  I should have checked the forums first.  Live and learn.  I find it odd that it restored Windows 8 with no problem - just as fast as a backup, but Windows 8 wasn't running - obviously it's the restore boot USB that WHS 2011 creates.  Strange.  I dont' know what would be different about Windows 8 "bits" vs. Windows 7 bits, but there it is - perhaps the backups are stored differently in the backup databse?

    And the only thing that changed in all of this was the hard drives - SAS to SATA and back to SAS - everything else - CPU, memory, network, etc. - was exactly the same.  The disks are idle most of the time, and the network is far from saturated (as others have pointed out).  Very, very weird....

    Tuesday, July 24, 2012 11:11 PM
  • Update:-

    The 64Gb restore did eventually finish 28 hours and 17 minutes later.... an full restore from the WHS boot disk is considerably faster. also full restore from WHS 1 is a lot faster than identical restore from WHS 2011, though that may well be down to the disk configuration (2011 is on quad core processor with software mirrored HDD, WHS 1 is on dual core with JBOD, both have 4Gb RAM)

    Server Backup to USB drives:-

    It's functional.... BUT interestingly there is absolutely no compresion achieved on the backup 704 GB of data on the server disk takes 704Gb on the backup disk, a lot of the data is JPEG so would not expect much from that, but there are an equal number of RAW picture files, and many Word, and Adobe Indesign documents where I would expect any form of compression to achieve results (winzip will compress a RAW file to approx 10% of its original size) consequently I can't backup >1Tb on the server to a 1Tb external drive.

    Consequently I have had to use 4 x 1TB drives to backup the server. I go for a once a week backup of the server with a daily backup of the OS because the WHS client wants to do some form of daily backup no matter what.

    Backup of 700Gb of data to USB 3 connected external drives takes approxomatly 5 hours, so from 2am to 7am on average, I have achieved a restore from this method, it's obviously not as simple as having the entire backup in one but it is usable as a disaster recovery system, for all except fire, as  I don't have enough spare hardware to sort an off site regime, so at some point I will be looking to improve it. I do have a network cable running down to a workshop at the bottom of the garden idle thoughts of running a monthly restore to create a clone server and have a regular test of the backup system are brewing.... though it would be simpler to just buy some more external HDDs and swap them out regularly.

    Pete

     

    Wednesday, August 1, 2012 9:09 AM
  • Consequently I have had to use 4 x 1TB drives to backup the server. I go for a once a week backup of the server with a daily backup of the OS because the WHS client wants to do some form of daily backup no matter what.

    Pete, Don't break the bank trying to back up your server disks to yet more disks.  There are plenty of online backup services.  I've been using Carbonite and it works pretty well with WHS2011.  Well, mostly OK with it. You have to tell it to disregard the backups of the PCs on the network or it will crash the backup service.  So I have them back up media and the server OS. The server has a little USB drive plugged in that it thinks it is backing its OS up to so I don't have to see the alerts. As for the PC backups, it seems to me that backups of backups is a bit excessive since the server has all the pictures and documents on it and they go off to the cloud backup. In a real crisis like a fire I'm not worried about having to re-install individual PCs.  Pick your favorite cloud backup if you don't like Carbonite.


    • Edited by magarity Thursday, August 29, 2013 2:25 AM
    Thursday, August 29, 2013 2:22 AM