none
Is there a replica size threshold ? RRS feed

  • Question

  • We use the sync framework (version 1) to sync files between a local and network location (which is a DFS folder).  Network bandwidth, storage size, etc is not an issue.  Our experience so far has shown that when you have a large folder pairing, it will take a while for synchronization to complete. 

    Is there an upper limit for a replica we should keep in mind to effectively sync the replicas (i.e. replicas should be less then 2GB)?  Not looking for anything other then a guideline going forward. 

    Monday, November 1, 2010 3:24 PM

Answers

  • I got an answer for this if anyone is interested ...

    It is the number of files ( and not the size of the files itself) that affects how efficiently we can detect changes – anything upto half a million we can handle pretty efficiently. After that – you might want to look into splitting this out into more folder pairs.

     

     

    • Marked as answer by f1ctc95 Tuesday, November 16, 2010 11:14 PM
    Tuesday, November 16, 2010 11:14 PM

All replies

  • Hi,

    File Sync Provider (FSP) does not impose a hard upper limit on the number of files or file size as long as the file system underlying the folder supports it. Can you share some numbers like how many files, how long it takes to sync, what's the percentage of items being synced ?

    Also, when syncing folders with large number of files or large underlying hierarchy, following aspects can influence sync time (you might be already aware of these):

    1) Are the folders being synced for the first time ? If so, FSP is looking at each item and building up it's database. This initial sync could be longer than subsequent syncs, given there aren't too many changes to sync subsequently.

    2) Is FSP looking at file contents to determine if the file changed by calculating file hash? If so, this could be slower than the option when you configure FSP to not look at file contents. Can you confirm if this option is being used in your setup or not ?

    Thanks,
    Sameer

    Tuesday, November 2, 2010 1:02 AM
  • To answer your questions ... I do see subsequent sync's take much less time then the first and the options are set to ExplicitDetectChanges.  

    However, I should have been more clear when I used the term, "effective."  To give some background on my question, I've seen a 7GB folder sync produce a 32MB .metadata file.  Subsequent sync's have taken over 10 minutes for a single delete.  Users won't always want to wait for sync to finish, so they stop it (one way or another) and their metadata file becomes corrupted.

    I think this really comes down to user education.  Most of the files in this example are not actively used.  What I am looking for is if there are any benchmarks or recommendations for sile threasholds we could follow and educate our users as a best practice.

    Thanks again !

    Tuesday, November 2, 2010 5:33 PM
  • I got an answer for this if anyone is interested ...

    It is the number of files ( and not the size of the files itself) that affects how efficiently we can detect changes – anything upto half a million we can handle pretty efficiently. After that – you might want to look into splitting this out into more folder pairs.

     

     

    • Marked as answer by f1ctc95 Tuesday, November 16, 2010 11:14 PM
    Tuesday, November 16, 2010 11:14 PM