locked
Can someone explain what kind of deduplication we have in Vail RRS feed

  • Question

  • Hi all,

    I'm trying to find out how deduplication is hanlded in Vail - Is there any documentation on this?

     

    Thanks


    Björn Axell MVP - Advisec AB http://blog.advisec.com
    Thursday, August 19, 2010 6:25 PM

Answers

  • Are you talking about backups? If so, at the level of files on individual clients, it looks like it's handled about the same way it is in V1. So data is stored by clusters, rather than files. Vail examines the client computer and compares clusters to data already in the database, and only new or changed clusters are actually sent across the wire.

    If you're talking about detecting duplicate files in the shares, nothing has changed there, either, so there is no attempt to "deduplicate" that data. And likewise, there's no cross comparison between the shares and the backup database.


    I'm not on the WHS team, I just post a lot. :)
    Thursday, August 19, 2010 6:39 PM
    Moderator

All replies

  • Hi all,

    I'm trying to find out how deduplication is hanlded in Vail - Is there any documentation on this?

    Thanks


    Björn Axell MVP - Advisec AB http://blog.advisec.com
    MS hasn't released any details regarding Drive Extender v2 (and I doubt that they will before it reaches RTM status, if they do at all).  Having said that, what exactly do you mean by "deduplication"?
    Thursday, August 19, 2010 6:29 PM
    Moderator
  • Are you talking about backups? If so, at the level of files on individual clients, it looks like it's handled about the same way it is in V1. So data is stored by clusters, rather than files. Vail examines the client computer and compares clusters to data already in the database, and only new or changed clusters are actually sent across the wire.

    If you're talking about detecting duplicate files in the shares, nothing has changed there, either, so there is no attempt to "deduplicate" that data. And likewise, there's no cross comparison between the shares and the backup database.


    I'm not on the WHS team, I just post a lot. :)
    Thursday, August 19, 2010 6:39 PM
    Moderator
  • Deduplication as done in ZFS is not present in WHS Vail, unfortunately. 
    • Proposed as answer by Michael Coelho Thursday, August 19, 2010 7:20 PM
    Thursday, August 19, 2010 7:20 PM
  • Hi,

    Yes, I'm talking about backup. I have heard that files are only backup ones For example if I have two clean Windows 7 and make full backup of them  - it should only take diskspace as it was one.

     

    Is this correct? I haven't played with V1 so I haven't seen any documents about this for V1.

     

    Thanks


    Björn Axell MVP - Advisec AB http://blog.advisec.com
    Thursday, August 19, 2010 8:45 PM
  • Bjorn,

    You might want to check out the V1 technical brief on Backup and Restore.

    http://www.microsoft.com/downloads/details.aspx?FamilyID=196fe38c-df20-4e19-92ca-6bda7bec3ecb&DisplayLang=en

     


    This post is "AS IS" and confers no rights. Jonas Svensson [MSFT] Windows Home Server Community Program Manager
    Windows Home Server Team Blog
    Connect Windows Home Server
    Windows Home Server
    Thursday, August 19, 2010 11:53 PM
  • Drive Extender Duplication. Only thing that comes to mind for me, and probably wrong.


    Nancy Ward
    Windows 8 BetaFerret

    "kariya21 [MVP]" wrote in message news:5ea526f2-2bfa-4359-837d-92328dd1bc66@communitybridge.codeplex.com...

    Hi all,

    I'm trying to find out how deduplication is hanlded in Vail - Is there any documentation on this?

    Thanks


    Björn Axell MVP - Advisec AB http://blog.advisec.com

    MS hasn't released any details regarding Drive Extender v2 (and I doubt that they will before it reaches RTM status, if they do at all).  Having said that, what exactly do you mean by "deduplication"?


    Nancy Ward
    Friday, August 20, 2010 1:06 AM
  • Drive Extender Duplication. Only thing that comes to mind for me, and probably wrong.


    Nancy Ward

    Actually, I looked on the internet (Google is everyone's friend  :)  ) after answering and found data deduplication.  I've just never heard it called that before (I've always used the term Single Instance Storage).
    Friday, August 20, 2010 3:30 AM
    Moderator
  • Fancy schmancy way of saying removing duplicate stuff. :) But, but, but, I thought we wanted to duplicate files. At least, that's what I learned in WHS 1. That way if one hard drive fails, there's one left.


    Nancy Ward
    Windows 8 BetaFerret

    "kariya21 [MVP]" wrote in message news:287065fe-8dd6-4d99-8ce7-a6b8e2721452@communitybridge.codeplex.com...

    Drive Extender Duplication. Only thing that comes to mind for me, and probably wrong.





    --


    Nancy Ward

    Actually, I looked on the internet (Google is everyone's friend  :)  ) after answering and founddata deduplication <http://en.wikipedia.org/wiki/Data_deduplication>.  I've just never heard it called that before (I've always used the term Single Instance Storage).


    Nancy Ward
    Friday, August 20, 2010 4:29 AM
  • Fancy schmancy way of saying removing duplicate stuff. :) But, but, but, I thought we wanted to duplicate files. At least, that's what I learned in WHS 1. That way if one hard drive fails, there's one left.


    Nancy Ward

    Folder Duplication is not the same thing as deduplication.  Deduplication means having only one copy of data, even though multiple sources rely on that data.  For example, you have 5 computers at home, all with Win7.  You back all 5 of them up to the server.  The OS files are only stored on the server once (because the files are identical for all 5 computers), thereby saving space (no point in backing up the exact same set of files 5 times, right?)
    Friday, August 20, 2010 4:39 AM
    Moderator
  • CLICK! The light just came on.

    Thanks for the information, Kariya. I learn something new every day, and this is good and much appreciated.

    Saving space is good, even with 1 or 2 TB drives in the server.


    Nancy Ward
    Windows 8 BetaFerret

    "kariya21 [MVP]" wrote in message news:7215fc11-db24-4985-a703-c64e4f4ed96d@communitybridge.codeplex.com...

    Fancy schmancy way of saying removing duplicate stuff. :) But, but, but, I thought we wanted to duplicate files. At least, that's what I learned in WHS 1. That way if one hard drive fails, there's one left.


    Nancy Ward

    Folder Duplication is not the same thing as deduplication.  Deduplication means having only one copy of data, even though multiple sources rely on that data.  For example, you have 5 computers at home, all with Win7.  You back all 5 of them up to the server.  The OS files are only stored on the server once (because the files are identical for all 5 computers), thereby saving space (no point in backing up the exact same set of files 5 times, right?)


    Nancy Ward
    Friday, August 20, 2010 3:19 PM