A not so simple hierarchical sync issue - how to correctly defer an update?

• Question

• 

  

  0

  Sign in to vote

  I am facing an issue with hierarchical data, and I have found a "hack" that I am not sure it is OK.

  I include a toy project. Repo A has 2 records. Repo B has none. The two records form a parent-child relationship. The global ids for the records are such, that during the sync process the child item is transferred before the parent. I cannot change the global ids to invert this order. For the sake of the example, my data is immutable, it will never get deleted, I will only sync once, etc. In short: I focus on the first sync A -> B, where those two fields will be copied to B.

  Please refer to the sourcecode below: it can be copied-and-pasted to a Console application just as is if you want to execute it.

  The logic in SaveItemChange for SaveChangeAction.Create is simple: when I receive the data, if its parent is already in the store (or if it has no parent), I add it. If the parent is not in the store, I invoke "context.RecordConstraintConflictForItem (ConstraintConflictReason.NoParent)" and move on.

  Now, the idea in my head is to retry those records later on (during the sync process, but not now), when - hopefully - the parent record will be already there. To get this done, I have created a MemoryConflictLog and implemented INotifyingChangeApplierTarget2.

  My findings:

  * if I do not listen to the event DestinationCallbacks.ItemConstraint, the conflict does not get saved into the log (the method SaveConstraintConflict is not called), and the item does not get retried later. The same if I listen to the event using ConstraintConflictResolutionAction.SkipChange.

  * if I do listen to the event DestinationCallbacks.ItemConstraint and set ConstraintConflictResolutionAction.SaveConflict, the item gets saved, but as a "permanent" conflict. This is not what I want, because it is not retried later, and that would mean that I have to persist this conflict and take care of it later - not during the current sync session.

  * The hack: if I don't listen to the event (or use SkipChange), *and* I call by hand SaveConstraintConflict, then everything works as expected: during the current session, Sync Framework retries the item later, and the sync finishes with the two items in B.

  As far as I know, the call to SaveConstraintConflict should be done by the framework, not by me. But I cannot see any other way to get the conflict automatically solved than this one.

  Can anybody tell me if I am doing right, or if there is any other way to solve these hierarchical conflicts by deferring the item action?

  The second question is: provided that this hack is not a hack but a feature, is this call correct? Sepcially the second and the SyncKnowledge parameters.

  SaveConstraintConflict (change, change.ItemId, ConstraintConflictReason.NoParent, data, new SyncKnowledge (IdFormats, m_metadata.ReplicaId, m_metadata.GetNextTickCount ()), true);

  Thanks a lot in advance.

  Code:

  namespace FailingHierarchicalSync
  {
    using System;
    using System.Linq;
    using System.Collections.Generic;
    using System.Text;
    using Microsoft.Synchronization;
    using Microsoft.Synchronization.MetadataStorage;
    using System.Runtime.InteropServices;
    using System.IO;
  
    class Program
    {
      public class Data { public Guid Id { get; set; } public Guid ParentId { get; set; } }
  
      static void Main (string[] args)
      {
        List<Data> store1 = new List<Data> ();
        Guid guidProvider1 = Guid.NewGuid ();
        Console.WriteLine ("Provider1: " + guidProvider1.ToString ());
        File.Delete (guidProvider1.ToString ());
        Provider provider1 = new Provider (guidProvider1, store1);
  
        List<Data> store2 = new List<Data> ();
        Guid guidProvider2 = Guid.NewGuid ();
        Console.WriteLine ("Provider2: " + guidProvider2.ToString ());
        File.Delete (guidProvider2.ToString ());
        Provider provider2 = new Provider (guidProvider2, store2);
  
        // parent = 222...
        // child = 111...
        store1.Add (new Data () { Id = new Guid ("11111111-1111-4111-1111-111111111111"), ParentId = new Guid ("22222222-2222-4222-2222-222222222222") });
        store1.Add (new Data () { Id = new Guid ("22222222-2222-4222-2222-222222222222"), ParentId = Guid.Empty });
  
        SyncOrchestrator agent = new SyncOrchestrator ();
        agent.Direction = SyncDirectionOrder.Upload; // from 1 to 2
        agent.LocalProvider = provider1;
        agent.RemoteProvider = provider2;
        agent.Synchronize ();
  
        if (store2.Count != 2) { throw new Exception ("Not all items were transferred"); }
        Console.WriteLine ("Ok!");
        Console.ReadLine ();
      }
  
      internal class Provider : KnowledgeSyncProvider, IChangeDataRetriever, INotifyingChangeApplierTarget, INotifyingChangeApplierTarget2
      {
        internal Provider (Guid replicaId, List<Data> store)
        {
          m_store = store;
          m_replicaId = replicaId;
          m_idFormats = new SyncIdFormatGroup ();
          m_idFormats.ItemIdFormat.IsVariableLength = false;
          m_idFormats.ItemIdFormat.Length = (ushort) (Marshal.SizeOf (typeof (Guid)));
          m_idFormats.ReplicaIdFormat.IsVariableLength = false;
          m_idFormats.ReplicaIdFormat.Length = (ushort) (Marshal.SizeOf (typeof (Guid)));
  
          this.DestinationCallbacks.ItemConstraint += DestinationCallbacks_ItemConstraint;
        }
  
        private void DestinationCallbacks_ItemConstraint (object sender, ItemConstraintEventArgs e)
        {
          // TODO! If I set .SaveConflict, the SaveConstraintConflict is called, but with not-temporary flag, and it fails
          // If I set .SkipChange (or I don't use this event at all), the SaveConstraintConflict is not called... 
          // but if then I use the hack, everything *seems* to work... !!!!!
          // the hack is below, in SaveItemChange
          //e.SetResolutionAction (ConstraintConflictResolutionAction.SaveConflict);
          e.SetResolutionAction (ConstraintConflictResolutionAction.SkipChange);
        }
  
        private Guid m_replicaId;
        private SqlMetadataStore m_metadataStore;
        private ReplicaMetadata m_metadata;
        private SyncIdFormatGroup m_idFormats;
        private SyncSessionContext m_currentSessionContext;
        private List<Data> m_store;
        private MemoryConflictLog m_memConflictLog;
  
        #region KnowledgeSyncProvider
  
        public override void BeginSession (SyncProviderPosition position, SyncSessionContext syncSessionContext)
        {
          if (!File.Exists (m_replicaId.ToString ()))
          {
            m_metadataStore = SqlMetadataStore.CreateStore (m_replicaId.ToString ());
            m_metadata = m_metadataStore.InitializeReplicaMetadata (m_idFormats, new SyncId (m_replicaId), null, null);
          }
          else
          {
            m_metadataStore = SqlMetadataStore.OpenStore (m_replicaId.ToString ());
            m_metadata = m_metadataStore.GetSingleReplicaMetadata ();
          }
          m_metadataStore.BeginTransaction ();
          UpdateMetadataStoreWithLocalChanges ();
          m_metadataStore.CommitTransaction ();
          m_currentSessionContext = syncSessionContext;
          m_memConflictLog = new MemoryConflictLog (IdFormats);
        }
  
        private void UpdateMetadataStoreWithLocalChanges ()
        {
          SyncVersion newVersion = new SyncVersion (0, m_metadata.GetNextTickCount ());
          m_metadata.DeleteDetector.MarkAllItemsUnreported ();
          foreach (Data data in m_store)
          {
            ItemMetadata item = m_metadata.FindItemMetadataById (new SyncId (data.Id));
            if (null == item)
            {
              item = m_metadata.CreateItemMetadata (new SyncId (data.Id), newVersion);
              item.ChangeVersion = newVersion;
              m_metadata.SaveItemMetadata (item);
            }
            else
            {
              // assume data never changes
              m_metadata.DeleteDetector.ReportLiveItemById (new SyncId (data.Id));
            }
          }
          foreach (ItemMetadata item in m_metadata.DeleteDetector.FindUnreportedItems ())
          {
            item.MarkAsDeleted (newVersion);
            m_metadata.SaveItemMetadata (item);
          }
        }
  
        public override void EndSession (SyncSessionContext syncSessionContext)
        {
          m_metadataStore.Dispose ();
          m_metadataStore = null;
        }
  
        public override ChangeBatch GetChangeBatch (uint batchSize, SyncKnowledge destinationKnowledge, out object changeDataRetriever)
        {
          changeDataRetriever = this;
          ChangeBatch result = m_metadata.GetChangeBatch (batchSize, destinationKnowledge);
          ;
          return result;
        }
  
        public override FullEnumerationChangeBatch GetFullEnumerationChangeBatch (uint batchSize, SyncId lowerEnumerationBound, SyncKnowledge knowledgeForDataRetrieval, out object changeDataRetriever)
        {
          throw new NotImplementedException ();
        }
  
        public override void GetSyncBatchParameters (out uint batchSize, out SyncKnowledge knowledge)
        {
          batchSize = 10;
          knowledge = m_metadata.GetKnowledge ();
        }
  
        public override SyncIdFormatGroup IdFormats
        {
          get { return m_idFormats; }
        }
  
        public override void ProcessChangeBatch (ConflictResolutionPolicy resolutionPolicy, ChangeBatch sourceChanges, object changeDataRetriever, SyncCallbacks syncCallbacks, SyncSessionStatistics sessionStatistics)
        {
          m_metadataStore.BeginTransaction ();
          IEnumerable<ItemChange> localChanges = m_metadata.GetLocalVersions (sourceChanges);
          NotifyingChangeApplier changeApplier = new NotifyingChangeApplier (m_idFormats);
          changeApplier.ApplyChanges (resolutionPolicy, CollisionConflictResolutionPolicy.ApplicationDefined,
            sourceChanges, changeDataRetriever as IChangeDataRetriever, localChanges, m_metadata.GetKnowledge (),
            m_metadata.GetForgottenKnowledge (), this, m_memConflictLog, m_currentSessionContext, syncCallbacks);
          m_metadataStore.CommitTransaction ();
        }
  
        public override void ProcessFullEnumerationChangeBatch (ConflictResolutionPolicy resolutionPolicy, FullEnumerationChangeBatch sourceChanges, object changeDataRetriever, SyncCallbacks syncCallbacks, SyncSessionStatistics sessionStatistics)
        {
          throw new NotImplementedException ();
        }
  
        #endregion KnowledgeSyncProvider
  
        #region IChangeDataRetriever Members
  
        public object LoadChangeData (LoadChangeContext loadChangeContext)
        {
          return m_store.Single (x => x.Id == loadChangeContext.ItemChange.ItemId.GetGuidId ());
        }
  
        #endregion
  
        #region INotifyingChangeApplierTarget Members
  
        public IChangeDataRetriever GetDataRetriever ()
        {
          return this;
        }
  
        public ulong GetNextTickCount ()
        {
          return m_metadata.GetNextTickCount ();
        }
  
        public void SaveChangeWithChangeUnits (ItemChange change, SaveChangeWithChangeUnitsContext context)
        {
          throw new NotImplementedException ();
        }
  
        public void SaveConflict (ItemChange conflictingChange, object conflictingChangeData, SyncKnowledge conflictingChangeKnowledge)
        {
          throw new NotImplementedException ();
        }
  
        public void SaveItemChange (SaveChangeAction saveChangeAction, ItemChange change, SaveChangeContext context)
        {
          switch (saveChangeAction)
          {
            case SaveChangeAction.Create:
              {
                Data data = (Data) context.ChangeData;
                if (data.ParentId == Guid.Empty)
                {
                  // this is a parent record - add it, no probs
                  m_store.Add (new Data () { Id = data.Id, ParentId = data.ParentId });
                  ItemMetadata item = m_metadata.CreateItemMetadata (change.ItemId, change.CreationVersion);
                  item.ChangeVersion = change.ChangeVersion;
                  m_metadata.SaveItemMetadata (item);
                }
                else
                {
                  // this is a child record - add only if parent exists
                  if (m_store.Any (x => x.Id == data.ParentId))
                  {
                    // ok, can be added
                    m_store.Add (new Data () { Id = data.Id, ParentId = data.ParentId });
                    ItemMetadata item = m_metadata.CreateItemMetadata (change.ItemId, change.CreationVersion);
                    item.ChangeVersion = change.ChangeVersion;
                    m_metadata.SaveItemMetadata (item);
                  }
                  else
                  {
                    // no, it cannot be added
                    context.RecordConstraintConflictForItem (ConstraintConflictReason.NoParent);
                    // HACK: With this call, and the event DestinationCallbacks_ItemConstraint to SkipChange, I manage
                    // to get what I need: Sync Framework will queue this item and will try again later
                    // however, as far as I know, I am not supposed to call SaveConstraintConflict by hand...
                    // What should I do, then?
                    SaveConstraintConflict (change, change.ItemId, ConstraintConflictReason.NoParent, data,
                      new SyncKnowledge (IdFormats, m_metadata.ReplicaId, m_metadata.GetNextTickCount ()), true);
                  }
                }
                break;
              }
            default:
              throw new NotImplementedException ();
          }
        }
  
        public void StoreKnowledgeForScope (SyncKnowledge knowledge, ForgottenKnowledge forgottenKnowledge)
        {
          m_metadata.SetKnowledge (knowledge);
          m_metadata.SetForgottenKnowledge (forgottenKnowledge);
          m_metadata.SaveReplicaMetadata ();
        }
  
        public bool TryGetDestinationVersion (ItemChange sourceChange, out ItemChange destinationVersion)
        {
          ItemMetadata metadata = m_metadata.FindItemMetadataById (sourceChange.ItemId);
  
          if (metadata == null)
          {
            destinationVersion = null;
            return false;
          }
          else
          {
            destinationVersion = new ItemChange (m_idFormats, m_metadata.ReplicaId, sourceChange.ItemId,
                metadata.IsDeleted ? ChangeKind.Deleted : ChangeKind.Update,
                metadata.CreationVersion, metadata.ChangeVersion);
            return true;
          }
        }
  
        #endregion
  
        #region INotifyingChangeApplierTarget2 Members
  
        public void SaveConstraintConflict (ItemChange conflictingChange, SyncId conflictingItemId, ConstraintConflictReason reason, object conflictingChangeData, SyncKnowledge conflictingChangeKnowledge, bool temporary)
        {
          if (!temporary)
          {
            // The in-memory conflict log is used, so if a non-temporary conflict is saved, it's
            // an error.
            throw new NotImplementedException ("SaveConstraintConflict can only save temporary conflicts.");
          }
          else
          {
            // For temporary conflicts, just pass on the data and let the conflict log handle it.
            m_memConflictLog.SaveConstraintConflict (conflictingChange, conflictingItemId, reason,
              conflictingChangeData, conflictingChangeKnowledge, temporary);
          }
        }
  
        #endregion
      }
    }
  }
  

  Tuesday, April 20, 2010 4:25 PM