locked
Server Shared Folder Backup Overhead? RRS feed

  • Question

  • I've been backing up my server shared folder data to an external 1 TB drive for a couple months now.  I also use the WHS BDBB add-in to save my back-up database to the same external drive. 

    From the server storage tab:
     - Total reported size of my external drive: 931.51 GB
     - Shared folder size: 464 GB
     - PC Backups: 398 GB

    Based on the above numbers I would expect to have no problem storing both sets data on the external drive however recently when I went to back up the shared folders again I noticed that it was indicating very little free space on the external drive (less than 5 GB).  When I viewed the available server backups list, it showed them taking up not quite 530 GB of space for approximately 20-25 different backup dates.  While I guess I would expect some overhead from having multiple backups on the disk (and the corresponding links from files to various backup dates) 66 GB seemed rather excessive. 

    I deleted my backup database backup because I was going to replace it anyways and then I decided to start deleting some of the shared folder backups to see if I could free up space as well.  To my surprise, as I deleted various backups, the space used actually increased !  Initially I was not just deleting the oldest backups, but rather cleaning out intermediate daily ones so that I would still have some weekly ones going back as far as I had space for.  As an experiment, I then also deleted the oldest remaining backup and the space used decreased as I would've expected but when I deleted the next oldest one the space used increased again.

    Does anyone have any thoughts on what is going on here?  How much overhead is there in creating multiple backups?  Why should the space used on an external backup increase when backup points are deleted rather than decrease?

    Thanks,

    Mark

    Monday, April 27, 2009 2:33 AM

Answers

  • OK, I think I figured out what was going on and to paraphrase Pogo (I think), "I have met the enemy and it is me".  What was happening was that I had set up system restore on my server sometime back on the C: drive (and disabled it on D: as well as all the mount points for the storage pool drives).  It was not disabled on the the external drive however and what I finally observed was that each time I deleted a backup point from that drive, system restore created a restore point on the drive and used up some of the remaining free space.  After disabling system restore on that drive, all the expected free space was restored and deletions of backup points resulted in consistently increasing or neutral free space rather than sometimes a decrease.

    Ken and Karyia21, thank you for your time and input.

    Mark
    • Marked as answer by MYERSM Thursday, May 7, 2009 3:38 AM
    Thursday, May 7, 2009 3:38 AM

All replies

  • When you create multiple backups of your shares, a file which hasn't changed will not be backed up again, but one which has changed (or one that's new) will. So over time your backups will grow, and growing an average of slightly less than 3 GB per backup isn't all that unreasonable.

    As for the disk usage changes as you delete old backups, I think I know what's going on, but will have to wait until later in the week to confirm it.
    I'm not on the WHS team, I just post a lot. :)
    Monday, April 27, 2009 12:40 PM
    Moderator
  • Thanks Ken. 

    Relative to your first point, in general I would've said that I felt I was mostly just adding new files to my shared folders rather than modifying existing ones which is why I was a little surprised by the delta between the amount of space taken up on the external drive vs the size of the shared folders on the server.  But that's clearly just a 'feeling' on my part and and I could very well be wrong especially given the fact that I'm talking about a couple months worth of backups.

    As for increasing rather than decreasing disk usage as I delete backup points, I will wait with interest to hear what you find out.  That is something I would certainly like to understand and get corrected whether it is a problem with what I am doing or one with WHS itself in some fashion

    Mark
    Monday, April 27, 2009 5:51 PM
  • Well, a new file has to go somewhere when you back it up, now doesn't it? Why would you be surprised that the total space consumed by the server backup would increase over time, even if it does use hard links where it can?

    And here's a question for you until I can do a few tests: How are you cleaning up backups? In the Windows Home Server console, or by deleting them while the disk is connected to a computer (could be your server, could be another computer)?
    I'm not on the WHS team, I just post a lot. :)
    Monday, April 27, 2009 8:45 PM
    Moderator

  • I wasn't surprised that the total size of the backup grew as I added new files to the server and subsequently backed them up, what surprised me was how different the total space used on the backup drive for the shared folders was from the space used for the same files on the server... i.e. 530GB used on the external drive to backup 464GB of data on the server?  Multiple versions of files and/or some kind of overhead could certainly account for some of the difference but I just wasn't sure I had that many files with multiple versions or if that much overhead seemed reasonable.

    In answer to your question, I have not manipulated the data on the backup disk through anything other than the WHS console.  I have deleted the backups via the console by right clicking on the server on the Computers and Backup tab, selecting View Backups and then selecting a backup from the list and clicking on the Delete button and letting it process. 

    Thanks,

    Mark
    Monday, April 27, 2009 11:43 PM
  • Below is some additional data from some experimentation I did with the backups I had, I really don't understand what is going on with the free space in terms of how WHS handling the creation and deletion of server backups:

    Starting point: 931 GB external disk containing backups of both my shared folders and the backup DB. 
     - The backup DB occupied 398 GB of space and the shared folders size on WHS was 464 GB. 
     - The total reported usage on the external disk at the point I started taking this data was 912 GB.
     - I sequentially started deleting backups via the WHS console and recorded the reported disk usage after each deletion.  My expectation would be that usage would decrease each time as a backup was deleted but that did not happen consistently as you can see that the usage actually increased after some deletions. 
     - After I finally deleted the last backup and the backup database backup, there were 61 GB occupied by something even though there were no files or folders that I could see on the disk.  After deleting the partition and recreating it, I was able to get that space back.  \
     - I then backed up the shared folders once and then again immediately to see if there was any overhead associated with the creation if hard links to the later backup and there did not seem to be any of significance. 
     - Finally I deleted the second backup I had just made and again the usage INCREASED so there seems to me to perhaps be an issue with how WHS handles the deletion of shared folder backups that will slowly but surely eat up space as those backups are managed over time?


                                            Used                      Free
    Action                              Space    Delta        Space
    Start                               912.00                    19.00
    Del 3/5 Backup                916.00     4.00         15.00
    Del 3/8 Backup                920.00     4.00         11.00
    Del 3/17 Backup              925.00     5.00           6.00
    Del 3/22 Backup              928.00     3.00           3.00
    Del 3/31 Backup              927.00    -1.00           4.00
    Del 4/4 Backup                927.00     0.00           4.00
    Del 4/5 Backup                923.00    -4.00           8.00
    Del 4/13 Backup              919.00    -4.00         12.00
    Del 4/19 Backup              916.00    -3.00         15.00
    Del 4/22 Backup              918.00     2.00         13.00
    Del 4/27 Backup              459.00    -459.00    472.00
    Del Backup DB Backup      61.00    -398.00    870.00
    Del partition and reformat    0.00    931.00     931.00
    Backup 5/1 12:05            472.00     472.00    459.00
    Backup 5/1 2:56              472.00     0.00       459.00
    Del 5/1 2:56 Backup        477.00     5.00       454.00
    Monday, May 4, 2009 2:04 AM
  • Below is some additional data from some experimentation I did with the backups I had, I really don't understand what is going on with the free space in terms of how WHS handling the creation and deletion of server backups:

    Starting point: 931 GB external disk containing backups of both my shared folders and the backup DB. 
     - The backup DB occupied 398 GB of space and the shared folders size on WHS was 464 GB. 
     - The total reported usage on the external disk at the point I started taking this data was 912 GB.
     - I sequentially started deleting backups via the WHS console and recorded the reported disk usage after each deletion.  My expectation would be that usage would decrease each time as a backup was deleted but that did not happen consistently as you can see that the usage actually increased after some deletions. 
     - After I finally deleted the last backup and the backup database backup, there were 61 GB occupied by something even though there were no files or folders that I could see on the disk.  After deleting the partition and recreating it, I was able to get that space back.  \
     - I then backed up the shared folders once and then again immediately to see if there was any overhead associated with the creation if hard links to the later backup and there did not seem to be any of significance. 
     - Finally I deleted the second backup I had just made and again the usage INCREASED so there seems to me to perhaps be an issue with how WHS handles the deletion of shared folder backups that will slowly but surely eat up space as those backups are managed over time?


                                            Used                      Free
    Action                              Space    Delta        Space
    Start                               912.00                    19.00
    Del 3/5 Backup                916.00     4.00         15.00
    Del 3/8 Backup                920.00     4.00         11.00
    Del 3/17 Backup              925.00     5.00           6.00
    Del 3/22 Backup              928.00     3.00           3.00
    Del 3/31 Backup              927.00    -1.00           4.00
    Del 4/4 Backup                927.00     0.00           4.00
    Del 4/5 Backup                923.00    -4.00           8.00
    Del 4/13 Backup              919.00    -4.00         12.00
    Del 4/19 Backup              916.00    -3.00         15.00
    Del 4/22 Backup              918.00     2.00         13.00
    Del 4/27 Backup              459.00    -459.00    472.00
    Del Backup DB Backup      61.00    -398.00    870.00
    Del partition and reformat    0.00    931.00     931.00
    Backup 5/1 12:05            472.00     472.00    459.00
    Backup 5/1 2:56              472.00     0.00       459.00
    Del 5/1 2:56 Backup        477.00     5.00       454.00

    It's probably VSS related.  If you enable the ability to view system folders, you will probably see one called System Volume Information.  If you check its Properties, it will say it's empty, but it's not.  The reason it says it's empty is because you don't have permissions to read what's in that folder.  If you add your user account and grant yourself read permissions, you will no doubt see the extra GBs in there.
    Monday, May 4, 2009 5:06 AM
    Moderator
  • Ok, thanks for the input.  I'll have to take a look at this further tonight when I get home.  I've got a second backup drive that I'll be bringing home from work so I can look at what kind of data it has on it in terms of System Volume Information (I rotate backups between two drives week to week storing one at my office).

    So I'm familiar at a high level with VSS as a method of creating a snapshot of files at a point in time that can then be backed up and restored later but I guess I don't understand yet exactly how that fits into what's going on here?  I also can't quite tell from the posts I've seen on VSS if it is really working or not on WHS?  Ref: http://social.microsoft.com/forums/en-US/whssoftware/thread/ccecb859-7b85-443a-92ad-171b8de7a260/ and http://social.microsoft.com/forums/en-US/whssoftware/thread/ccecb859-7b85-443a-92ad-171b8de7a260/


    A few questions to further my education:
     - Does VSS somehow explain the odd behaviour of free space sometimes going down after I delete a backup point instead of up?
     - Are snapshots being created on the external drive when I backup the shared folders and/or later delete some of those backups or is their creation on the drive independent of the backup process? 
     - Do I somehow need to manually manage the number of and space taken up by those snapshots or is it self-regulating somehow via a system setting I can change?  The problem I was running into was that the space taken up by the snapshots was getting to the point that I did not have room on the disk for my data.

    Thanks for your time!

    Mark


     
    Monday, May 4, 2009 2:34 PM
  • VSS should not be affecting the space on the server backup drive. As for the backups, yes, you will need to manage those manually though the console. If you haven't noticed, you will only be able to do this with a backup drive connected.
    I'm not on the WHS team, I just post a lot. :)
    Tuesday, May 5, 2009 3:53 PM
    Moderator
  • OK, I think I figured out what was going on and to paraphrase Pogo (I think), "I have met the enemy and it is me".  What was happening was that I had set up system restore on my server sometime back on the C: drive (and disabled it on D: as well as all the mount points for the storage pool drives).  It was not disabled on the the external drive however and what I finally observed was that each time I deleted a backup point from that drive, system restore created a restore point on the drive and used up some of the remaining free space.  After disabling system restore on that drive, all the expected free space was restored and deletions of backup points resulted in consistently increasing or neutral free space rather than sometimes a decrease.

    Ken and Karyia21, thank you for your time and input.

    Mark
    • Marked as answer by MYERSM Thursday, May 7, 2009 3:38 AM
    Thursday, May 7, 2009 3:38 AM
  • ... and to paraphrase Pogo (I think), "I have met the enemy and it is me". ...
    "We have met the enemy and he is us."

    Glad I could help. 

    I'm not on the WHS team, I just post a lot. :)
    Thursday, May 7, 2009 5:55 PM
    Moderator