none
A not so simple hierarchical sync issue - how to correctly defer an update? RRS feed

  • Question

  • I am facing an issue with hierarchical data, and I have found a "hack" that I am not sure it is OK.

    I include a toy project. Repo A has 2 records. Repo B has none. The two records form a parent-child relationship. The global ids for the records are such, that during the sync process the child item is transferred before the parent. I cannot change the global ids to invert this order. For the sake of the example, my data is immutable, it will never get deleted, I will only sync once, etc. In short: I focus on the first sync A -> B, where those two fields will be copied to B.

    Please refer to the sourcecode below: it can be copied-and-pasted to a Console application just as is if you want to execute it.

    The logic in SaveItemChange for SaveChangeAction.Create is simple: when I receive the data, if its parent is already in the store (or if it has no parent), I add it. If the parent is not in the store, I invoke "context.RecordConstraintConflictForItem (ConstraintConflictReason.NoParent)" and move on.

    Now, the idea in my head is to retry those records later on (during the sync process, but not now), when - hopefully - the parent record will be already there. To get this done, I have created a MemoryConflictLog and implemented INotifyingChangeApplierTarget2.

    My findings:

    * if I do not listen to the event DestinationCallbacks.ItemConstraint, the conflict does not get saved into the log (the method SaveConstraintConflict is not called), and the item does not get retried later. The same if I listen to the event using ConstraintConflictResolutionAction.SkipChange.

    * if I do listen to the event DestinationCallbacks.ItemConstraint and set ConstraintConflictResolutionAction.SaveConflict, the item gets saved, but as a "permanent" conflict. This is not what I want, because it is not retried later, and that would mean that I have to persist this conflict and take care of it later - not during the current sync session.

    * The hack: if I don't listen to the event (or use SkipChange), *and* I call by hand SaveConstraintConflict, then everything works as expected: during the current session, Sync Framework retries the item later, and the sync finishes with the two items in B.

    As far as I know, the call to SaveConstraintConflict should be done by the framework, not by me. But I cannot see any other way to get the conflict automatically solved than this one.

    Can anybody tell me if I am doing right, or if there is any other way to solve these hierarchical conflicts by deferring the item action?

    The second question is: provided that this hack is not a hack but a feature, is this call correct? Sepcially the second and the SyncKnowledge parameters.

    SaveConstraintConflict (change, change.ItemId, ConstraintConflictReason.NoParent, data, new SyncKnowledge (IdFormats, m_metadata.ReplicaId, m_metadata.GetNextTickCount ()), true);

    Thanks a lot in advance.

    Code:

    namespace FailingHierarchicalSync
    {
      using System;
      using System.Linq;
      using System.Collections.Generic;
      using System.Text;
      using Microsoft.Synchronization;
      using Microsoft.Synchronization.MetadataStorage;
      using System.Runtime.InteropServices;
      using System.IO;
    
      class Program
      {
        public class Data { public Guid Id { get; set; } public Guid ParentId { get; set; } }
    
        static void Main (string[] args)
        {
          List<Data> store1 = new List<Data> ();
          Guid guidProvider1 = Guid.NewGuid ();
          Console.WriteLine ("Provider1: " + guidProvider1.ToString ());
          File.Delete (guidProvider1.ToString ());
          Provider provider1 = new Provider (guidProvider1, store1);
    
          List<Data> store2 = new List<Data> ();
          Guid guidProvider2 = Guid.NewGuid ();
          Console.WriteLine ("Provider2: " + guidProvider2.ToString ());
          File.Delete (guidProvider2.ToString ());
          Provider provider2 = new Provider (guidProvider2, store2);
    
          // parent = 222...
          // child = 111...
          store1.Add (new Data () { Id = new Guid ("11111111-1111-4111-1111-111111111111"), ParentId = new Guid ("22222222-2222-4222-2222-222222222222") });
          store1.Add (new Data () { Id = new Guid ("22222222-2222-4222-2222-222222222222"), ParentId = Guid.Empty });
    
          SyncOrchestrator agent = new SyncOrchestrator ();
          agent.Direction = SyncDirectionOrder.Upload; // from 1 to 2
          agent.LocalProvider = provider1;
          agent.RemoteProvider = provider2;
          agent.Synchronize ();
    
          if (store2.Count != 2) { throw new Exception ("Not all items were transferred"); }
          Console.WriteLine ("Ok!");
          Console.ReadLine ();
        }
    
        internal class Provider : KnowledgeSyncProvider, IChangeDataRetriever, INotifyingChangeApplierTarget, INotifyingChangeApplierTarget2
        {
          internal Provider (Guid replicaId, List<Data> store)
          {
            m_store = store;
            m_replicaId = replicaId;
            m_idFormats = new SyncIdFormatGroup ();
            m_idFormats.ItemIdFormat.IsVariableLength = false;
            m_idFormats.ItemIdFormat.Length = (ushort) (Marshal.SizeOf (typeof (Guid)));
            m_idFormats.ReplicaIdFormat.IsVariableLength = false;
            m_idFormats.ReplicaIdFormat.Length = (ushort) (Marshal.SizeOf (typeof (Guid)));
    
            this.DestinationCallbacks.ItemConstraint += DestinationCallbacks_ItemConstraint;
          }
    
          private void DestinationCallbacks_ItemConstraint (object sender, ItemConstraintEventArgs e)
          {
            // TODO! If I set .SaveConflict, the SaveConstraintConflict is called, but with not-temporary flag, and it fails
            // If I set .SkipChange (or I don't use this event at all), the SaveConstraintConflict is not called... 
            // but if then I use the hack, everything *seems* to work... !!!!!
            // the hack is below, in SaveItemChange
            //e.SetResolutionAction (ConstraintConflictResolutionAction.SaveConflict);
            e.SetResolutionAction (ConstraintConflictResolutionAction.SkipChange);
          }
    
          private Guid m_replicaId;
          private SqlMetadataStore m_metadataStore;
          private ReplicaMetadata m_metadata;
          private SyncIdFormatGroup m_idFormats;
          private SyncSessionContext m_currentSessionContext;
          private List<Data> m_store;
          private MemoryConflictLog m_memConflictLog;
    
          #region KnowledgeSyncProvider
    
          public override void BeginSession (SyncProviderPosition position, SyncSessionContext syncSessionContext)
          {
            if (!File.Exists (m_replicaId.ToString ()))
            {
              m_metadataStore = SqlMetadataStore.CreateStore (m_replicaId.ToString ());
              m_metadata = m_metadataStore.InitializeReplicaMetadata (m_idFormats, new SyncId (m_replicaId), null, null);
            }
            else
            {
              m_metadataStore = SqlMetadataStore.OpenStore (m_replicaId.ToString ());
              m_metadata = m_metadataStore.GetSingleReplicaMetadata ();
            }
            m_metadataStore.BeginTransaction ();
            UpdateMetadataStoreWithLocalChanges ();
            m_metadataStore.CommitTransaction ();
            m_currentSessionContext = syncSessionContext;
            m_memConflictLog = new MemoryConflictLog (IdFormats);
          }
    
          private void UpdateMetadataStoreWithLocalChanges ()
          {
            SyncVersion newVersion = new SyncVersion (0, m_metadata.GetNextTickCount ());
            m_metadata.DeleteDetector.MarkAllItemsUnreported ();
            foreach (Data data in m_store)
            {
              ItemMetadata item = m_metadata.FindItemMetadataById (new SyncId (data.Id));
              if (null == item)
              {
                item = m_metadata.CreateItemMetadata (new SyncId (data.Id), newVersion);
                item.ChangeVersion = newVersion;
                m_metadata.SaveItemMetadata (item);
              }
              else
              {
                // assume data never changes
                m_metadata.DeleteDetector.ReportLiveItemById (new SyncId (data.Id));
              }
            }
            foreach (ItemMetadata item in m_metadata.DeleteDetector.FindUnreportedItems ())
            {
              item.MarkAsDeleted (newVersion);
              m_metadata.SaveItemMetadata (item);
            }
          }
    
          public override void EndSession (SyncSessionContext syncSessionContext)
          {
            m_metadataStore.Dispose ();
            m_metadataStore = null;
          }
    
          public override ChangeBatch GetChangeBatch (uint batchSize, SyncKnowledge destinationKnowledge, out object changeDataRetriever)
          {
            changeDataRetriever = this;
            ChangeBatch result = m_metadata.GetChangeBatch (batchSize, destinationKnowledge);
            ;
            return result;
          }
    
          public override FullEnumerationChangeBatch GetFullEnumerationChangeBatch (uint batchSize, SyncId lowerEnumerationBound, SyncKnowledge knowledgeForDataRetrieval, out object changeDataRetriever)
          {
            throw new NotImplementedException ();
          }
    
          public override void GetSyncBatchParameters (out uint batchSize, out SyncKnowledge knowledge)
          {
            batchSize = 10;
            knowledge = m_metadata.GetKnowledge ();
          }
    
          public override SyncIdFormatGroup IdFormats
          {
            get { return m_idFormats; }
          }
    
          public override void ProcessChangeBatch (ConflictResolutionPolicy resolutionPolicy, ChangeBatch sourceChanges, object changeDataRetriever, SyncCallbacks syncCallbacks, SyncSessionStatistics sessionStatistics)
          {
            m_metadataStore.BeginTransaction ();
            IEnumerable<ItemChange> localChanges = m_metadata.GetLocalVersions (sourceChanges);
            NotifyingChangeApplier changeApplier = new NotifyingChangeApplier (m_idFormats);
            changeApplier.ApplyChanges (resolutionPolicy, CollisionConflictResolutionPolicy.ApplicationDefined,
              sourceChanges, changeDataRetriever as IChangeDataRetriever, localChanges, m_metadata.GetKnowledge (),
              m_metadata.GetForgottenKnowledge (), this, m_memConflictLog, m_currentSessionContext, syncCallbacks);
            m_metadataStore.CommitTransaction ();
          }
    
          public override void ProcessFullEnumerationChangeBatch (ConflictResolutionPolicy resolutionPolicy, FullEnumerationChangeBatch sourceChanges, object changeDataRetriever, SyncCallbacks syncCallbacks, SyncSessionStatistics sessionStatistics)
          {
            throw new NotImplementedException ();
          }
    
          #endregion KnowledgeSyncProvider
    
          #region IChangeDataRetriever Members
    
          public object LoadChangeData (LoadChangeContext loadChangeContext)
          {
            return m_store.Single (x => x.Id == loadChangeContext.ItemChange.ItemId.GetGuidId ());
          }
    
          #endregion
    
          #region INotifyingChangeApplierTarget Members
    
          public IChangeDataRetriever GetDataRetriever ()
          {
            return this;
          }
    
          public ulong GetNextTickCount ()
          {
            return m_metadata.GetNextTickCount ();
          }
    
          public void SaveChangeWithChangeUnits (ItemChange change, SaveChangeWithChangeUnitsContext context)
          {
            throw new NotImplementedException ();
          }
    
          public void SaveConflict (ItemChange conflictingChange, object conflictingChangeData, SyncKnowledge conflictingChangeKnowledge)
          {
            throw new NotImplementedException ();
          }
    
          public void SaveItemChange (SaveChangeAction saveChangeAction, ItemChange change, SaveChangeContext context)
          {
            switch (saveChangeAction)
            {
              case SaveChangeAction.Create:
                {
                  Data data = (Data) context.ChangeData;
                  if (data.ParentId == Guid.Empty)
                  {
                    // this is a parent record - add it, no probs
                    m_store.Add (new Data () { Id = data.Id, ParentId = data.ParentId });
                    ItemMetadata item = m_metadata.CreateItemMetadata (change.ItemId, change.CreationVersion);
                    item.ChangeVersion = change.ChangeVersion;
                    m_metadata.SaveItemMetadata (item);
                  }
                  else
                  {
                    // this is a child record - add only if parent exists
                    if (m_store.Any (x => x.Id == data.ParentId))
                    {
                      // ok, can be added
                      m_store.Add (new Data () { Id = data.Id, ParentId = data.ParentId });
                      ItemMetadata item = m_metadata.CreateItemMetadata (change.ItemId, change.CreationVersion);
                      item.ChangeVersion = change.ChangeVersion;
                      m_metadata.SaveItemMetadata (item);
                    }
                    else
                    {
                      // no, it cannot be added
                      context.RecordConstraintConflictForItem (ConstraintConflictReason.NoParent);
                      // HACK: With this call, and the event DestinationCallbacks_ItemConstraint to SkipChange, I manage
                      // to get what I need: Sync Framework will queue this item and will try again later
                      // however, as far as I know, I am not supposed to call SaveConstraintConflict by hand...
                      // What should I do, then?
                      SaveConstraintConflict (change, change.ItemId, ConstraintConflictReason.NoParent, data,
                        new SyncKnowledge (IdFormats, m_metadata.ReplicaId, m_metadata.GetNextTickCount ()), true);
                    }
                  }
                  break;
                }
              default:
                throw new NotImplementedException ();
            }
          }
    
          public void StoreKnowledgeForScope (SyncKnowledge knowledge, ForgottenKnowledge forgottenKnowledge)
          {
            m_metadata.SetKnowledge (knowledge);
            m_metadata.SetForgottenKnowledge (forgottenKnowledge);
            m_metadata.SaveReplicaMetadata ();
          }
    
          public bool TryGetDestinationVersion (ItemChange sourceChange, out ItemChange destinationVersion)
          {
            ItemMetadata metadata = m_metadata.FindItemMetadataById (sourceChange.ItemId);
    
            if (metadata == null)
            {
              destinationVersion = null;
              return false;
            }
            else
            {
              destinationVersion = new ItemChange (m_idFormats, m_metadata.ReplicaId, sourceChange.ItemId,
                  metadata.IsDeleted ? ChangeKind.Deleted : ChangeKind.Update,
                  metadata.CreationVersion, metadata.ChangeVersion);
              return true;
            }
          }
    
          #endregion
    
          #region INotifyingChangeApplierTarget2 Members
    
          public void SaveConstraintConflict (ItemChange conflictingChange, SyncId conflictingItemId, ConstraintConflictReason reason, object conflictingChangeData, SyncKnowledge conflictingChangeKnowledge, bool temporary)
          {
            if (!temporary)
            {
              // The in-memory conflict log is used, so if a non-temporary conflict is saved, it's
              // an error.
              throw new NotImplementedException ("SaveConstraintConflict can only save temporary conflicts.");
            }
            else
            {
              // For temporary conflicts, just pass on the data and let the conflict log handle it.
              m_memConflictLog.SaveConstraintConflict (conflictingChange, conflictingItemId, reason,
                conflictingChangeData, conflictingChangeKnowledge, temporary);
            }
          }
    
          #endregion
        }
      }
    }
    
    Tuesday, April 20, 2010 4:25 PM

Answers

  • Hi Fernando,

    What you're observing is expected behavior.  The only time the change applier will generate temporary conflicts for you is when you specify a collision conflict.  This is because we detect temporary conflicts by looking to see if the conflicting item's knowledge from the source is contained in the destination knowledge and that the source knowledge contains the creation version for the destination conflicting change -- this basically lets us know whether there may be changes for the conflicting item that haven't been sent from the source yet (the creation version check verifies that it's an item the source knows about). 

    When conflicts are reenumerated, the are two types of reenumeration...regular conflicts and temporary conflicts.  Regular conflicts will only be reenumerated if the destination knowledge contains the source knowledge for the conflicting item.  This lets us know that everything is up to date for the item.  This happens at the end of each batch and the end of a session.  Temporary conflicts are always reenumerated, but ones with the knowledge check are handled first (originally we only reenumerated changes that passed the knowledge check, but we had to fix it because we would never reenumerate circular collisions).

    So this gets us to no-parent conflicts.  They aren't temporary conflicts because we can't do the initial knowledge detections that determines that they are temporary, and they won't be reenumerated as logged conflicts because they don't have a conflicting item to check knowledge against.  Unfortunately, specifying the parent id as a conflicting id when the constraint conflict is reported won't work because the check for temporary conflicts will complain when it tries to get the version from the destination (which makes some sense...the provider reported a conflict with a change that doesn't exist).

    Fortunately, there is one workaround:  Pass the parent item id with the data (which you're already doing).  In the callback, specify the SaveConflict action.  When the change applier calls SaveConstraintConflict, the conflicting item id will be null.  Pull out the parent item id from your data and pass it as the conflicting item id when you save it to the in-memory conflict log.  It's similar to your workaround, except that it's passing the parent id as the conflicting item id, rather than using the item id.  You may be better off letting the change applier save the constraint conflict for you by using the callback, just because it introduces less risk in terms of figuring out how knowledge looks.

    I'm sorry you're hitting this issue.  Most of the complexity of constraint conflicts was around handling collisions correctly, and it looks like we overlooked some stuff for no-parent issues.  We have some ideas for fixes, but it may boil down to our priorities going forward.

    Hope this helps,

    Aaron


    SDE, Microsoft Sync Framework
    Thursday, April 22, 2010 12:28 AM
    Answerer

All replies

  • More information: after doing some more tests, it seems that Sync Framework only retries these temporary conflicts once. In the example above, that was enough. But with a more complex set of items, it is not.

    store1.Add (new Data () { Id = new Guid ("11111111-1111-4111-1111-111111111111"), ParentId = new Guid ("22222222-2222-4222-2222-222222222222") });
    store1.Add (new Data () { Id = new Guid ("22222222-2222-4222-2222-222222222222"), ParentId = new Guid ("33333333-3333-4333-3333-333333333333") });
    store1.Add (new Data () { Id = new Guid ("33333333-3333-4333-3333-333333333333"), ParentId = Guid.Empty });
    store1.Add (new Data () { Id = new Guid ("44444444-4444-4444-4444-444444444444"), ParentId = new Guid ("11111111-1111-4111-1111-111111111111") });
    store1.Add (new Data () { Id = new Guid ("55555555-5555-4555-5555-555555555555"), ParentId = new Guid ("44444444-4444-4444-4444-444444444444") });

    With this set of items, Sync Framework tries to insert them in this order:

    11111111111141111111111111111111 -> no (missing 2)
    22222222222242222222222222222222 -> no (missing 3)
    33333333333343333333333333333333 -> ok
    44444444444444444444444444444444 -> no (missing 1)
    55555555555545555555555555555555 -> no (missing 4)
    22222222222242222222222222222222 -> 2nd try, ok
    11111111111141111111111111111111 -> 2nd try, ok
    55555555555545555555555555555555 -> 2nd try, no (missing 4)
    44444444444444444444444444444444 -> 2nd try, ok

    Sync Framework does not try again element 5, and the sync finishes.

    The question remains... is there any good way to solve this problem? I am tempted to use a custom queue to hold elements that cannot be applied (reporting them with RecordRecoverableErrorForItem) and when the parent is added, execute the operation on its queued children. However, I do not know if that would have any unexpected side effect in the knowledge - first the item is marked as RecoverableError, and then, later on the same sync session, the item gets updated. Would this be ok?

    Wednesday, April 21, 2010 8:24 AM
  • Hi Fernando,

    What you're observing is expected behavior.  The only time the change applier will generate temporary conflicts for you is when you specify a collision conflict.  This is because we detect temporary conflicts by looking to see if the conflicting item's knowledge from the source is contained in the destination knowledge and that the source knowledge contains the creation version for the destination conflicting change -- this basically lets us know whether there may be changes for the conflicting item that haven't been sent from the source yet (the creation version check verifies that it's an item the source knows about). 

    When conflicts are reenumerated, the are two types of reenumeration...regular conflicts and temporary conflicts.  Regular conflicts will only be reenumerated if the destination knowledge contains the source knowledge for the conflicting item.  This lets us know that everything is up to date for the item.  This happens at the end of each batch and the end of a session.  Temporary conflicts are always reenumerated, but ones with the knowledge check are handled first (originally we only reenumerated changes that passed the knowledge check, but we had to fix it because we would never reenumerate circular collisions).

    So this gets us to no-parent conflicts.  They aren't temporary conflicts because we can't do the initial knowledge detections that determines that they are temporary, and they won't be reenumerated as logged conflicts because they don't have a conflicting item to check knowledge against.  Unfortunately, specifying the parent id as a conflicting id when the constraint conflict is reported won't work because the check for temporary conflicts will complain when it tries to get the version from the destination (which makes some sense...the provider reported a conflict with a change that doesn't exist).

    Fortunately, there is one workaround:  Pass the parent item id with the data (which you're already doing).  In the callback, specify the SaveConflict action.  When the change applier calls SaveConstraintConflict, the conflicting item id will be null.  Pull out the parent item id from your data and pass it as the conflicting item id when you save it to the in-memory conflict log.  It's similar to your workaround, except that it's passing the parent id as the conflicting item id, rather than using the item id.  You may be better off letting the change applier save the constraint conflict for you by using the callback, just because it introduces less risk in terms of figuring out how knowledge looks.

    I'm sorry you're hitting this issue.  Most of the complexity of constraint conflicts was around handling collisions correctly, and it looks like we overlooked some stuff for no-parent issues.  We have some ideas for fixes, but it may boil down to our priorities going forward.

    Hope this helps,

    Aaron


    SDE, Microsoft Sync Framework
    Thursday, April 22, 2010 12:28 AM
    Answerer
  • I should clarify that the workaround is not something we've tested at all.  I know it worked for someone else I suggested it to, but it's definitely a bit of a hack.  I can't think of why it wouldn't work, but I haven't verified that it will.

    Aaron


    SDE, Microsoft Sync Framework
    Thursday, April 22, 2010 12:40 AM
    Answerer
  • Hi Aaron,

    Thank you so much for your answer. I have tried the workaround/hack that you have suggested, and it seems to work - in my example, I got the 5 items sync'd in one shot.

    The fact that the call

    context.RecordConstraintConflictForItem (new SyncId (data.ParentId), ConstraintConflictReason.NoParent);

    provoked an ItemHasNoVersionDataException was misleading. Thanks to your explanation now I understand the reason.

    I hope next versions of the Sync Framework could improve the way this conflict is handled. I would be interested in knowing how does the File Provider manages this situation - after all, it could happen that a tree of folders have to be sync'd - basically the same I was doing in my example.

    Anyway, again, thanks a lot.

    Regards, Fernando

     

    Thursday, April 22, 2010 10:07 AM
  • Regarding the FileSyncProvider, it basically does something similar to what the change applier does, but it does it by hand, since it wasn't built into the change applier when the FileSyncProvider was written (so it has to maintain its own log and stuff like that).

     

    Aaron


    SDE, Microsoft Sync Framework
    Thursday, April 22, 2010 6:31 PM
    Answerer
  • I am facing the same problem, and would like to use the solution provided here. However after reading through the posts a few times I am sill not sure how it is done.

    The sample code manually saves the conflict with source item's id (not the parent's id) and passing true to "temporary" argument.

    I would like to understand how Arron Greene's solution works. To be more specific should the argument "temporary" be ignored?

    Monday, November 5, 2012 7:03 PM