locked
If 2 drives are a pool where are the backups "shadow copies" stored? RRS feed

  • Question

  •  

    If 2 drives are a pool where are the backups "shadow copies" stored?
    Monday, January 28, 2008 9:24 PM

All replies

  • Maybe a better way to say this is where are the Duplicated Folders stored? Say one drive dies. What happens? I have the other drive left, where can I retrieve the files from? Will they still be there?

     

    Monday, January 28, 2008 9:36 PM
  • As I understand it the master shadow file will be stored on the drive with the less free space, and the alternate shadow will be stored on the other drive, (in theory). That would suggest that in a two pool drive system (three total or more) both pool drives will have almost the same level of use. Now the sys drive also has a pool feature and in this case I believe both the master shadow and the alternate shadow are move to the external pool drive. At least my experience is in a system with just two physical drives WHS attempts to keep all files on the pool drive.

     

    Monday, January 28, 2008 9:48 PM
  • If you're trying to determine if home PC backups participate in the Drive Extender reliability mechanism, then no they don't. Backups are "duplicated" by the PCs that are backed up. If your backup database needs to be reset, you just back up your home PCs again.

    If you're asking what goes on which drive, in a 2 disk WHS system, the system drive will typically contain:
    • The operating system on the C: partition.
    • Tombstones on the D: partition.
    • One shadow copy of any files in shares flagged for duplication.
    • Maybe some components of the backup database if the secondary disk is full.
    • Very unlikely, but possible if the secondary disk is full, is that the primary shadow of some files will be on the system disk. If this is the case, you're probably getting duplication warnings, because the secondary disk is full and there's no place to put additional shadows of duplicated files (both shadows can't reside on the same disk).
    On the secondary disk in the storage pool, you'll have most/all of the backup database components, the primary shadow of files in shares marked for duplication, and the only shadow of files in shares not marked for duplication.
    Tuesday, January 29, 2008 2:38 AM
    Moderator
  • Maybe I don't understand this whole pool thing.... I have two disks. According to what I read shares will only be stored on the second disk? If thats the case and the second drive fails I'll lose all my shared files because they wont be duplicated?

     

    Tuesday, January 29, 2008 2:17 PM
  •  Ken Warren wrote:
    If you're trying to determine if home PC backups participate in the Drive Extender reliability mechanism, then no they don't. Backups are "duplicated" by the PCs that are backed up. If your backup database needs to be reset, you just back up your home PCs again.<snip>


    And of course, the carefully avoided subject... what about the snapshots?  I need to roll back to last week but ooops.... I just caught a virus and my WHS lost all of my backups.  But don't worry, your PC is your backup.  I lost a file that was on my PC last month.  WHS lost all of my snapshots.  But don't worry, your PC is your backup.

    Thanks WHS.

    Do I sound less than impressed?
    Tuesday, January 29, 2008 2:26 PM
  • Thats scary.... Can someone answer my question ? I dont care about PC backups I just want to know where and what happens to the shared files

     

    Tuesday, January 29, 2008 2:38 PM
  •  jshoemaker21 wrote:
    Thats scary.... Can someone answer my question ? I dont care about PC backups I just want to know where and what happens to the shared files

     


    From my readings it is a little technical but the gist of it is this:

    FOR your backups...

    WHS uses a database with little pointer records for each 4 k block to say what disk that block is stored on.  Each database record has a pair of pointers, each pointer points to the block on a different drive IF file duplication is turned on.  So in fact individual files aren't even stored per se.  The BLOCKS inside of the files are stored, and a database is maintained that says where the blocks are stored, and what "files" the blocks belong to.

    Let's say that you have a text document on your computer.  That text document just happens to need FIVE 4K chunks of space to store it.  WHS creates FIVE pointers (in a database from my readings) for that one text document.  It breaks the file down into the FIVE blocks and starts saving the blocks somewhere:

    Block 1 goes to physical drive C
    Block 2 goes to physical drive C
    Block 3 goes to physical drive C
    Block 4 goes to physical drive C
    Block 5 goes to physical drive C

    The five "database block pointers" are updated so that one of the two pointers each points to the right place on Drive C:.

    The fascinating (and cool) part of this is that let's say that you have the exact same file out on a completely different computer.  You tell that computer to store that same file out on WHS.  WHS looks at the file name and a checksum of the five blocks of data that make up the file.  It discovers that it has already stored the blocks of that file and that they are out on drive C.  WHS does not store the file all over again.  My interpretation of this thing is that it creates ANOTHER of those database block pointers for the document, but for the new computer.  It then fills out the database pointer records with the locations of the blocks of the file.  Remember that the 4k blocks are already in storage, so the database pointer records just point to where they already are.  Voila, you have "stored" the same file again, but you have used almost no additional space because you did not actually store the file, just a little pointer to where the blocks are.

    Pretty cool huh?

    Apparently the file duplication does not use that stuff at all.  If you turn on duplication, then the the system simply copies the entire file off to another drive, apparently keeping pointers to the two copies somewhere?  Tombstones?.

    The scarier part is that if you happen to have different sized disks things get hairy for the file duplication part.  Let's say that you throw a 500 gig drive in the machine and a 300g drive.  You start using file duplication.  Suddenly, somewhere around 300 gigs of duplicated files, WHS says you cannot use duplication anymore because your are "out of space".  The reason is that while you do indeed have 200 gigs left, those 200 gigs are all one one disk, and there is no "matching" 200 gigs anywhere else.

    Lovely. 

    So now you go buy another 500 gig drive.  Hmmmm.... what will happen?  Will WHS magically redistribute all the stuff onto the two 500 gig drives and use them for duplicated files, leaving the 300 gig drive empty?  I doubt it.  My guess is that you will start adding more shared file and now you will get the same error message again after you add an additional 200 gigs because you have a permanent imbalance in your drive sizes.

    This whole "you can throw any drives you want on there and all is good" is obviously not exactly the full story.
    Tuesday, January 29, 2008 3:31 PM
  • This has NOT been my experience.

     

    Take a HP EX475 with two 500 GB disk.

     

    This appears as Disk 0 a Drive C and Drive D, and Disk 1 a pool drive.

     

    Now take one notebook computer with a lot of storage connected to it and start copying files to the EX475 what happens is the Disk 1 fills up and will quickly reach 90% full, on the other hand Disk 0 partition D, part of the pool drive remains at less than 40% full. The WHS prefers to keep the Disk 0 partition D as empty as possible. See what you can recover if you pull and replace that Disk 1 with another disk.

     

    Another example pull the top disk from an EX475, format the sys disk using factory image so that it is wipped and knows nothing about the second disk. Backup your pc, the smaller the better. Now add the second disk, and as soon as it is added shut down. Pull the disk, (this is what would happen if the disk failed) reboot the EX475, and it will complain about the missing disk, that has nothing stored on it at this point. Tell the EX475 that the disk is lost and all backups are gone! I had many 100s of GB of backup on my EX475, and tried this, there is no way it had time to move many if indeed any of my backups off the D partition and onto the second disk, yet it happily told me they where all gone.

    Tuesday, January 29, 2008 4:11 PM
  • In a WHS PC with two disks, two copies of any files in shares flagged for duplication will exist. One will be on the secondary disk, one will be on the primary disk. Should you have shares that are not flagged for duplication, the files in those shares will only be on the secondary disk.

    The files are just files in the file system, not (as John has incorrectly stated) stored as blocks in a database. Should you suffer a disaster which renders your entire server unbootable, but which leaves your drives still functional, at worst you can copy the files off the individual disks.
    Tuesday, January 29, 2008 4:13 PM
    Moderator
  • Darkone3, when you add a second disk to a WHS PC that has only one, practically the first thing that happens is that that the server starts moving the backup database off of the system disk to the newly added secondary disk. Because the backup database is made up of a number of files, and because those files are all very tightly coupled, the loss of any file in the database effectively destroys the database. (Yes, it would be nice if it were otherwise, but it isn't. There are suggestions on Connect to change this for future versions of the product; I suggest you go vote on one or more.) In your scenario, you add a second disk to the storage pool, and WHS starts moving the database. You then shut down and physically remove the disk. Any files that were moved are now inaccessible. When you tell WHS (through the console) to remove the missing disk, components of the backup database are missing, so WHS does the only thing it can: delete the database.
    Tuesday, January 29, 2008 4:21 PM
    Moderator
  •  Ken Warren wrote:
    <snip>The files are just files in the file system, not (as John has incorrectly stated) stored as blocks in a database.
    Ken, please read my post more carefully.  I never said the BLOCKS are stored in the database.  I said that the blocks of a file are only stored once (with no duplication) and that POINTERS to the blocks are stored in the database. 

    And if you go read "this looks like magic" post:

    http://forums.microsoft.com/WindowsHomeServer/ShowPost.aspx?PostID=1214404&SiteID=50

    that is where I found this.  Is it true? 

    All of which simply has to do with the "cool" part of not storing every file a million times.

    Tuesday, January 29, 2008 4:46 PM
  • Files in the shares on a WHS PC are stored as files in a file system. There is no database.

    Only the backup database uses the SIS-like block level storage.
    Tuesday, January 29, 2008 4:51 PM
    Moderator
  •  Ken Warren wrote:
    Files in the shares on a WHS PC are stored as files in a file system. There is no database.

    Only the backup database uses the SIS-like block level storage.


    Ahhh.  Thanks for that clarification Ken.  I will go delete my post entirely then.
    Tuesday, January 29, 2008 4:54 PM
  • So all in all, my 2 250gb hard drives, will each hold a copy of shared folders? One with a primary and one with a backup?

     

    Tuesday, January 29, 2008 4:56 PM
  • Yes, if all shares are flagged for duplication. As I said, if a share is not flagged for duplication, then files in that share will exist on only one drive (normally the secondary storage drive rather than the system drive).
    Tuesday, January 29, 2008 5:03 PM
    Moderator
  • Are all shares flagged for duplication by default?

     

     

     

    Tuesday, January 29, 2008 5:07 PM
  • If that where indeed the case one would expect that the two disk filled at the same rate. Indeed with the OS on the 0 disk it should fill faster due to less space because of the OS. On a HP MediaSmart EX475 my experience is they do not. That the second disk fills much faster than the internal disk does. Now that maybe limited to the EX475, or to just my EX475, but that is what I saw, one disk at 90+% filled and the other at actually less than 40% filled. With duplication on it is unlikely that both disk contain copies of all files, and where that different in size. IMHO.

     

    Tuesday, January 29, 2008 5:28 PM
  • What you're seeing is normal operation, and consistent with what I've said above. With two disks, all your backups are normally on one disk, the secondary disk. Windows Home Server does this to protect against losing the backup database to a failure of any disk; if the entire database is on a single disk, then only loss of that disk causes problems with the backup database. All your files (assuming you've got duplication turned on for all your shares) are on both disks. Add your files and your backups together, and you'll probably find out that your total is the space actually used on your secondary disk. If it's not there's something wrong with your server.
    Tuesday, January 29, 2008 5:44 PM
    Moderator
  • Ok ken, So how do I enable duplication? Also, say disk 0 fails, what would I do to recover the files? Would they be readable in an XP machine on Disk 1?

     

    Tuesday, January 29, 2008 7:00 PM
  • You start by installing Windows Home Server, which I don't think you've done yet. You can get a time limited evaluation version from Microsoft for the cost of shipping if you need to. When you install, if you have two disks in the PC, duplication will be turned on automatically for all the default shares. You can also turn it on/off using the Windows Home Server console, the management interface for WHS. You can't turn duplication on for a WHS PC with only a single disk.
    Tuesday, January 29, 2008 7:23 PM
    Moderator
  • You're good,. Now, Also, say disk 0 fails, what would I do to recover the files? Would they be readable in an XP machine on Disk 1?

     

    Tuesday, January 29, 2008 8:20 PM
  • You replace your system disk and perform a "server reinstallation," which is an option during the installation of WHS if it detects a complete set of secondary drives. Or you remove your secondary drive, connect it to another PC on your network, and copy the files (which are just files in a filesystem as stated previously) to wherever you like. Or perhaps you perform a search in the forums here; "recover files" seems to produce many good answers to your question.

    I highly recommend you install and experiment with Windows Home Server.
    Tuesday, January 29, 2008 9:05 PM
    Moderator
  • I bought it! I better install it!   So if either drive fails i'm still ok. Even if it's not the system disk? Need to know before I go putting my lifes data on a server! Thanks I do appreciate your help.

    Tuesday, January 29, 2008 9:08 PM
  • Again, I highly recommend you install and experiment with it. You can install it without providing a license key and activating it, which will give you 30 days to decide if you like the configuration, performance, etc. as initially set up.

    You'll find that most of your questions will answer themselves once you start experimenting with the product.
    Tuesday, January 29, 2008 9:27 PM
    Moderator
  • So if either drive fails i'm still ok. Even if it's not the system disk?

    Thats my last question. Are the duplicated files still going to be accessable, say either disk dies?

    Wednesday, January 30, 2008 2:44 PM
  • No matter which drive dies, if:
    • you have duplication turned on for all shares, and
    • you aren't getting "duplication failing" network health warnings (usually due to having one drive fill up before the other, as is common with only two drives because of how backups are stored)
    you will be able to retrieve the files in the shares. You may have to remove a drive from the server ahd retrieve the files on another system, but that's not much of a challenge.
    Wednesday, January 30, 2008 6:27 PM
    Moderator
  • Do you have a link on how to retrieve files on another system>?

     

    Wednesday, January 30, 2008 8:32 PM
  • Last question? Smile

    You remove the drive from system A and connect it to system B. The files that you have in your shares will be in a hidden folder (\DE\Shares) in the root of the drive, in folders organized like your shares.
    Wednesday, January 30, 2008 10:55 PM
    Moderator
  • Im sure these are password protected?? And do you mean an XP machine also?

     

    Thursday, January 31, 2008 5:48 PM
  • No. Yes.
    Thursday, January 31, 2008 6:03 PM
    Moderator
  • Wow so anyone taking the hard drive out has access!

     

    Thursday, January 31, 2008 6:06 PM
  • Ken I reviewed the info in the white papers I could find. There is no answer as to how many days the shadow copies last. Is there any more specific answers than a few days. Im'm sure the programmers didn't  program "a few days" in the software. Thanks for any help.

     

    Wednesday, February 6, 2008 11:44 PM
  • It depends!

    In general, Shadow Copies are set to utilise a selected amount of disk space which will fill at a rate dependant on the schedule which is set.

    In WHS, you can see how this is worked out by RDPing into the server, then go to Computer Management and then 'right-click' on the Shared Folders entry and select 'All Tasks'.  This will give you the option to configure the Shadow Copies, both the amount of space used, and the schedule you want to use.

     

    HTH,

     

    Colin

    Thursday, February 7, 2008 5:53 PM
  • As Colin says, it depends. While you can view some information about shadow copies in the location he points out, you should make no changes here, as Windows Home Server controls the taking of shadow copies of files itself.

    For more information (in general) about the previous versions functionality, see this Technet article.
    Thursday, February 7, 2008 7:10 PM
    Moderator
  • Cool that response helps a lot. Now are they sotred in a certain directoty by WHS?

     

    Thursday, February 7, 2008 11:18 PM
  • There is a great deal of useful information on VSS on Technet and in the Microsoft knowledgebase. I highly recommend you do some of your own research if you want to know how it works.
    Friday, February 8, 2008 2:45 AM
    Moderator
  • Dont know what vss is just want to know where they are sotred. Thanks anyways

     

    Ken if you dont want to answer thats fine. I've been through all the white papers. Thanks
    Friday, February 8, 2008 4:08 PM
  • Checked into it all.... Cant seem to find where WHS stores the Shadow Copies?

     

    Thursday, February 14, 2008 5:11 PM