locked
Backup takes a long time RRS feed

  • Question

  • My WHS takes quite a long time every night to backup my client pc. Below are the times of the latest 8 backups (hh:mm):

    4:48, 10:51, 5:25, 12:04, 7:07, 4:00, 6:50, 4:34.

    The server is a low end Intel C2D 2140, 9 disks with 8 TB in total, 1,34 TB free. The client is a more powerful desktop with Windows 7, four 1 TB disks, 3 TB of data in them. The machines are connected with gigabit ethernet. There's no other traffic in lan during backup.

    WHS backed up the client last night, as usual. Today I logged in and started a manual backup and waited for the system to finish it. The 4:48 listed above was how long it took. I know lots of MS updates were installed after the automatic backup last night, but I still feel the backup takes too much time.

    There isn't any sceduled defragmentation going in the client on that I'd be aware of. A while ago I had trouble with WHS though, the backup failing constantly. I chkdsk'd the server disks but they were ok. I repaired the backup database, losing some of the backups of this specific client (and all the backups of another client). The backup took long time alreadybefore that incident, though. The times were around 2-4 hours, but there was less to backup, too (perhaps around 2 TB).

    I wonder if these backup figures of several hours are simply something to accept, as there's several terabytes of data to maintain. As most of the data is quite static (like personal photo & video library that I seldomly touch), I expected WHS backup process to be able to handle it with minimal copying/organizing in server side.

    Any points of view on this?

    ivideur

    Thursday, October 14, 2010 1:47 PM

All replies

  • You have a huge amount of data on that client; it's going to take a while for the Windows Home Server backup engine to evaluate what's changed. Even 10-12 hours may not be excessive; how much of it changes frequently? Does it all need to live there, or can some/all of it move to your server, where other clients will also have access to it? In the latter case, you would probably be able to reduce the amount of storage in that client, while adding storage to your server. When adding storage to your server, plan for duplication (high availability, immediate protection from disk failure) and off-site backups (disaster recovery).


    I'm not on the WHS team, I just post a lot. :)
    Thursday, October 14, 2010 2:56 PM
    Moderator
  • It's more typical to add new files than to modify existing ones, so I'd say 99,9% of content is static/day (meaning that there shouldn't be more than few gigs of new or modified files in normal case when the nightly backup starts -- most often there isn't just about any). There are some virtual machine image files (typically 10-20 GB each), which WHS needs to backup as files (and not as clients from inside running virtual machine), which may not be the suggested way of taking care of the backups of them, but at most one or two of them is opened during a normal day. Of course even a single opened virtualpc image would then mean 20 GB to backup, I guess.

    I've been wondering how to change the layout, where to store all the stuff. I see three problems with moving data to WHS server. First, having them within a single hardware sounds more prone to failure, even considering duplication (about off-site backups below). Second, I would lose the file history, which I have now in backup history. Third, access to those files (editable video files, large raw photo files, virtualpc files) would be much slower because of Ethernet (vs. Serial ATA) and slowish processor and disks in WHS. 

    The last one of the reasons is really the critical one. If it would be possible to have WHS share data in an external eSATA/USB3 drive, which I could move between WHS and client without any of them to have a hiccup, it would be great. WHS should then sync the updated data from reattached drive to the duplication drive. Or as a more straightforward solution, if the network connection had better performance, that would solve the whole problem of course.

    I haven't really considered implementing off-site backups through internet, because of the volume of the data. Taking snapshots of client content occationally (too seldom really) off-site in usb drives etc. works as the last resort as long as the data resides in two distinct pc's onsite, but being only in WHS, hmm.

    Anyway, I appreciate your attention, and I think your questions & suggestions point me to rethink  where the increasing data should be placed, or if I should use something else but WHS to back it up. I had a hope that WHS would have a quicker way to find out the updated/added content in client, but it seems like that's not the case.

     

    Thursday, October 14, 2010 4:52 PM
  • You should have a look in which phase of the backup the time is spend. At my WHS the second phase (reorgansiation of Backup Database) last very long due to size of it. And unfortunately if there are more partitions to backup thos reorganisation will bes done multiple

    Regards from Germany
    Nobby1805

    Thursday, October 14, 2010 5:04 PM
  • Thanks for the hint, that's interesting! Where should I check the backup phase times? Is the phase shown just by watching the status text at the bottom line of the console? Or are phase changed stored in WHS event log or something?

    ivideur

    Thursday, October 14, 2010 5:11 PM
  • Best to look at the client backup-logs (called BacklupEngine...)

    This two lines shows start and end of the server reorganisation phase

    [1]101013.203232.1151: Status: Determine changed: scanned 24091 of 24096 total with 4 fixups (bytes sent=477740 received=0)
    [1]101013.204012.4474: Status: Server phase Reorganize1 complete

    Thursday, October 14, 2010 5:21 PM
  • I'm not quite sure what to make of this, but by looking for the backup log (in C:\Users\All Users\Microsoft\Windows Home Server\logs, not in Event log where I checked first) I found two things:

    First, this Windows 7 client actually has scheduled weekly defragmentation for all disks, each wed at 1 am (in contrast to what I said in my first post).

    This time, it occurred 13.10.2010 01:00. Most recent event log entries related to defrag:

     

    13.10.2010 10:57:13 The disk defragmenter successfully completed defragmentation on (C:)
    14.10.2010 10:20:27 The disk defragmenter successfully completed defragmentation on 201009 (H:)
    14.10.2010 10:34:41 The disk defragmenter successfully completed defragmentation on (C:)
    14.10.2010 10:39:44 The disk defragmenter successfully completed defragmentation on Samsung F3 1TB (D:)
    14.10.2010 10:52:08 The disk defragmenter successfully completed defragmentation on Samsung F3 1TB-2 (E:)
    14.10.2010 13:45:14 The disk defragmenter successfully completed boot optimization on (C:)
    14.10.2010 13:47:17 The disk defragmenter successfully completed defragmentation on Samsung F1 (F:)
    
    So, it took around 37 hours for the client to get the defragmentation done. Part of the time the client was in sleep state, but anyway it processed the defrag and whs backup simultaneously during the last nightly backup and in the morning as I started the manual backup.

    Second, these whs backup log items:

    [1]101014.090820.0593: Status: Start
    
    [1]101014.092444.0556: Status: Determine changed: scanned 6426731 of 13519568 total with 4 fixups (bytes sent=134 313 867 received=0)
    [1]101014.102522.9767: Status: Server phase Reorganize1 complete
    [1]101014.102805.0480: Status: Server phase Reorganize2 complete
    
    [1]101014.102805.0500: Status: Volume 2 of 5 is D: size 1000202043392 used 797374955520
    [1]101014.103503.7589: Status: Determine changed: scanned 168747 of 194671809 total with 4 fixups (bytes sent=3 564 580 received=0)
    [1]101014.110245.4400: Status: Server phase Reorganize1 complete
    [1]101014.110253.6244: Status: Server phase Reorganize2 complete
    
    [1]101014.110253.6254: Status: Volume 3 of 5 is E: size 1000202043392 used 880322891776
    [1]101014.112223.0093: Status: Determine changed: scanned 4652309 of 214723600 total with 4 fixups (bytes sent=98 715 009 received=0)
    [1]101014.114644.3299: Status: Server phase Reorganize1 complete
    [1]101014.114705.8681: Status: Server phase Reorganize2 complete
    
    [1]101014.114705.8691: Status: Volume 4 of 5 is F: size 1000202043392 used 756663058432
    [1]101014.123247.2699: Status: Determine changed: scanned 50380035 of 184730289 total with 4 fixups (bytes sent=1 072 230 039 received=0)
    [1]101014.130512.0012: Status: Server phase Reorganize1 complete
    [1]101014.130529.7482: Status: Server phase Reorganize2 complete
    
    [1]101014.130529.7492: Status: Volume 5 of 5 is H: size 1000202043392 used 711433306112
    [1]101014.132245.7564: Status: Determine changed: scanned 17 096 147 of 173 688 138 total with 4 fixups (bytes sent=371 127 674 received=0)
    [1]101014.135618.8756: Status: Server phase Reorganize1 complete
    [1]101014.135643.8480: Status: Server phase Reorganize2 complete
    [1]101014.135659.9619: Status: Completed 5 volumes
    
    [1]101014.135700.0409: Status: Bytes sent=3 167 943 371, bytes received=19 332 460, 0 reconnects
    
    I'm not quite sure how to read the byte figures and times taken for different phases, but if I understand correctly, the server is very quick in reorganizing the backup database (phase Reorganize2), and time is spent in equal amounts to actually send the data to the server and to do the phase "Reorganize1", whatever its meaning.

    ivideur

    Thursday, October 14, 2010 7:57 PM
  • No, thats wrong. The reorganization is done in phase1 (no idea what is done in phase2) and your Vol5 takes about 32 minutes

    you chan check this if you launch the backup status from the connector icon where you can see the percentage of the backup and the phases and look to taskman at the WHS where you found heavy activity but no network traffic during this server reorganization phase

    Friday, October 15, 2010 11:09 AM