locked
Incremental Backups of 3.5tb taking 16 hours. Will increasing the cluster size shorten this? RRS feed

  • Question

  • One of my client machines has two large volumes; one is 2TB the other 1.2TB.
    The first backup took 26 hours - no problems, I expected that.

    However, the subsequent daily backups are still taking 14-18 hours even though the source data hadn't changed.


    Do you think that this is how long it takes because there a lot of clusters to check? If so would changing the cluster size to 16K help?
    Sunday, June 28, 2009 10:02 PM

Answers

  • No files at all were added. I turned off my scheduled defrag some time ago because I read that this would 'fool' VSS into thinking that the cluster had changed, even though WHS would find the hash in its database and not resend the cluster.

    What do you think about changing the client cluster size to 16k?

    Thanks.

    I'm with Ken.  I don't think it will help much.  You might try running chkdsk /r on the hard drives in both server and client.
    • Marked as answer by Pearly King Tuesday, June 30, 2009 11:20 AM
    Tuesday, June 30, 2009 3:22 AM
    Moderator

All replies

  • Increasing the cluster size will probably not help. It's taking an unusually long time to back up that client, though; you should check to see what sort of network speed you have for that particular machine.
    I'm not on the WHS team, I just post a lot. :)
    Monday, June 29, 2009 2:35 AM
    Moderator
  • Thanks,
    It is  a gigabit network, jumbo frames no flow control, with WHS and the client the only devices on the switch. I will do some timed transfers to check the throughput.

    The client volumes are on a striped 8x500gig RAID 5 array; the WHS storage (10 drives) is mainly SATA I devices with a couple of IDE drives

    Is there a write up anywhere that describes how VSS detrmines if a cluster has changed? 3.5 TB is over 900 million clusters to check so I sort of thought that, by increasing the cluster size to 16k, I could cut that 'checking' time to a quarter of what it was - simplistically speaking. Or am I way off base?
    Monday, June 29, 2009 8:12 PM
  • Copying a couple of 6-9 gigabyte files gives an average disc to disc throughput of 250 mbps. Is that reasonable?
    Tuesday, June 30, 2009 12:18 AM
  • Copying a couple of 6-9 gigabyte files gives an average disc to disc throughput of 250 mbps. Is that reasonable?

    I would say so.  Did you add a lot of files to your client between backups?  Did you defrag your client between backups?
    Tuesday, June 30, 2009 1:54 AM
    Moderator
  • No files at all were added. I turned off my scheduled defrag some time ago because I read that this would 'fool' VSS into thinking that the cluster had changed, even though WHS would find the hash in its database and not resend the cluster.

    What do you think about changing the client cluster size to 16k?

    Thanks.
    Tuesday, June 30, 2009 2:04 AM
  • No files at all were added. I turned off my scheduled defrag some time ago because I read that this would 'fool' VSS into thinking that the cluster had changed, even though WHS would find the hash in its database and not resend the cluster.

    What do you think about changing the client cluster size to 16k?

    Thanks.

    I'm with Ken.  I don't think it will help much.  You might try running chkdsk /r on the hard drives in both server and client.
    • Marked as answer by Pearly King Tuesday, June 30, 2009 11:20 AM
    Tuesday, June 30, 2009 3:22 AM
    Moderator
  • OK thanks. I'll play around with various options and configs and see what happens.
    • Marked as answer by Pearly King Tuesday, June 30, 2009 11:19 AM
    • Unmarked as answer by Pearly King Tuesday, June 30, 2009 11:20 AM
    Tuesday, June 30, 2009 11:19 AM
  • one thing i have noticed when i rebuilt my server with new disks is that backup and file transfers times decreased by at least 50%

    i used to have a 8 pata disks as my storage disks which i have now replaced with 4 1TB sata II disks and speed is much better

    one of the factors you are seeing is possibly the speed of your storage drives

    if you check the network traffic in Task manager is it the checking of the cluster thats taking the time? if so what sort of speed are you getting?

    In one of my client PC's i have approx 500meg of data to backup (over two disks) and the checking for changed clusters takes approx 5-10 minutes. Client PC is Vista64 (sp2)

    I used to get slow performance before SP1 for vista - what client OS has your data on?

    Also try turning your jumbo frames off to see if you get any different performance as you can get problems if your NIC's do not support it properly. You did configure your switch and NIC all to the same setting for jumbo frames :)
    2 pata, 4 sata, 4.4tb storage, 2gig ram - home bake system
    Tuesday, June 30, 2009 3:47 PM
  • .............i used to have a 8 pata disks as my storage disks which i have now replaced with 4 1TB sata II disks and speed is much better one of the factors you are seeing is possibly the speed of your storage drives

    if you check the network traffic in Task manager is it the checking of the cluster thats taking the time? if so what sort of speed are you getting?
    Also try turning your jumbo frames off to see if you get any different performance as you can get problems if your NIC's do not support it properly. You did configure your switch and NIC all to the same setting for jumbo frames :)
    2 pata, 4 sata, 4.4tb storage, 2gig ram - home bake system

    I am sure that the slower PATA drives have some effect, but they are only 3 out of 11 drives, so I would thought that the overall impact was relatively small - I am guessing.
    I don't know how to answer your question about is it the cluster checking that is taking the time because I don't know how to measure that seperately. Given that hardly any data has chnaged on teh client machines, the answer must be that i is the checking that is taking the time. However, I didn't think that it referenced back to the WHS to check that.
    The "Technical Brief for Home Computer Backup and Restore" says:
    The home computer backup occurs as follows:
    • When a home computer is backed up to the home server, Windows Home Server software figures out what clusters have changed since the last backup.
    • The home computer software then calculates a hash for each of these clusters and sends the hashes to the home server. A hash is a number that uniquely identifies a cluster based on its contents.
    • The home server looks into its database of clusters to see if they are already stored on the home server

    Does anyone know how "it figures out"

    I'll turn off jumbo frames and retime the transfers. Both NICS are set to 9014, the switch is unmanaged so there are no settings for that.
    Tuesday, June 30, 2009 7:14 PM