none
Initial Data SnapShot with Full Enumeration Provider Sync RRS feed

  • Question

  • I have two Custom Full Enumeration Providers that seem to be running fairly slow and I wanted to at least speed up the initial sync by created a snapshot outside of MS sync of the remote provider for the local to use. Unfortunately after doing this, when I do the initial sync Microsoft sync tries to insert records that are already there.  I assume this is because the separate SQLMetaData store is unaware that the records are already there.  Is this the case or is there something else that is wrong?  I would think that during the full enumeration of both sides, if the meta data matched it would still not do the insert.
    Monday, March 22, 2010 9:41 PM

All replies

  • dosborn,

    How did you create the snapshot? I assume you copied the data only. Internally each item has a Global Id, which is unique across the whole sync ecosystem, and in your case the Sync Engine thinks it deals with 2 DISTINCT items which happen to have the same data. In order to achieve the result you wanted to, you'll need to also propagate the metadata, so that the engine is aware that these items are in fact the same. However, as a rule of thumb, "touching" the metadata is NOT the best thing someone can do, as you can end up losing data.

    Unless this is not a top priority, I'd recommend waiting for the initial sync to complete on its own. Initial sync always takes more time since it populates all the data. The following syncs (incremental) are much faster.

     

    Hope this helps.

    Adrian

    Monday, March 22, 2010 9:57 PM
  • Good to know.  This is a bit of a priority as it is taking several hours to do the initial sync.  I may need to try to implementing KnowledgeSyncProvider instead.  I'm getting killed on performance because the FullEnumeration provider handles each item individually.  It appears that the KnowlegeSyncProvider has the concept of batches so I could possibly process quicker that way.  Otherwise I just may need to roll my own.  I was hoping I could implement this snapshot idea as a quick fix.
    Monday, March 22, 2010 10:25 PM
  • Simple providers are slower than KnowledgeSyncProviders - that's the price for being simple. You say performance is an issue for you - is it an issue during initial sync only, or is it an issue after the initial sync as well?

    Are you committing the knowledge after each item? The InsertItem() method has an OUT BOOL parameter - commitKnowledgeAfterThisItem. If this is set to true, the knowledge will be saved after each item, which affects the performance, since knowledge operations are expensive.

    Adrian

    Monday, March 22, 2010 10:46 PM
  • Performance is an issue overall and I planned to switch to a KnowledgeSyncProvider, but I mainly wanted to get the initial sync addressed to buy some time to convert things to a KnowlegeSyncProvider.  I am committing knowledge on the insert, though turning that off doesn't increase performance that much.
    Monday, March 22, 2010 10:53 PM