locked
Efficiently enumerating changes in a custom provider's GetChangeBatch method RRS feed

  • Question

  • Greetings,

    I'm developing a custom provider for syncing an object model and am now implementing GetChangeBatch on the provider. Everywhere I look in the written documentation, the description of what goes on in here is to enumerate _all_ of the item metadata and check if the mapped destination knowledge contains that item metadata or not.

    This seems inefficient, and there must be some way to store the last time that the destination asked for changes and just send the incremental changes. It appears that this is what the ADO.Net Sync provider does, but it's not clear to me how it does it. What would I need to store to determine the incremental changes? I assume I'd also need to check any item exceptions - is this true?

    Thanks...
    • Moved by Max Wang_1983 Thursday, April 21, 2011 1:28 AM forum consolidation (From:SyncFx - Technical Discussion [ReadOnly])
    Thursday, June 18, 2009 8:42 PM

Answers

  • The efficiency with which you can do this depends on the store you have.  At some point, you'll have to go through all of the items to determine what has changed in order to update the metadata efficiently.  Many providers do this at the beginning of a session or in GetChangeBatch, then change detection (updating metadata) and change enumeration (putting the metadata in the change batch) are basically the same step.  Some stores, like databases with change tracking, can avoid the change detection step because it is done when changes are made.

    Assuming you don't need to go through every item to detect changes, you then are limited by the efficiency of your metadata store.  If you store the metadata in a store that has efficient query, there are some tricks you can do to optimize the change enumeration.  One approach is to store an extra field for every item that includes the local tick count the last time you wrote the item, regardless of whether it's a save change or an update (you would increment it when you detect updates).  If you set this as your local tick count in knowledge before a sync with a store, the next time you interact with the store, you can use FindMinTickCountForReplica to determine the last tick count it had for the source the last time you did a sync, and you can efficiently query to items with a greater tick count.  You still want to go through each of these items and check if they are contained in the destination knowledge, because if you had another replica sync to the source replica and the destination replica, you could send unneccessary changes.  So this method mainly helps to reduce the items for which you do the knowledge check, but it doesn't get rid of it entirely.

    At the end of the day, the knowledge check is what helps you determine incremental changes to send while allowing you to have varied topologies.  You can optimize the way you do this, but you probably still want to do it.

    Aaron


    SDE, Microsoft Sync Framework
    Thursday, June 18, 2009 10:31 PM