none
After SyncAbortedException Synchronize() takes a very long time RRS feed

  • Question

  • Using the MS Sync Framework, I have a SyncOrchestrator and the two file FileSyncProviders.  During a normal Synchronize() call, it takes about 3 minutes to finish the synchronize process.

    After a SyncAbortedException, the same process takes 45 minutes.

    How can I clean up after a SyncAbortedException?

    Why does it take so long after a SyncAbortedException?

     

    Thanks

    Jim

    Wednesday, October 27, 2010 6:15 PM

Answers

  • Hi,

    I can repro your scenario now. And it is a bug in FileSyncProvider metadata handling during change detection. It only happens when change detection fails in the middle. We will investigate this issue and consider a fix in the next SyncFx release. Thanks a lot for help us find this defect.

    With current FileSyncProvider, is it possible for you to avoid cancelling or failing a sync during provider ChangeDetection call? If not, you may want to call FileSyncProvider.ChangeDetect explicitly, and keep a copy of the .metadata file aside to replace the bad one when change detection fails in the middle. Considering there is no data lost but just performance impact, you can decide if this workaround is needed for you or not.

    Thanks,
    Dong 


    This posting is provided AS IS with no warranties, and confers no rights.
    Thursday, November 4, 2010 5:56 PM
    Moderator
  • I am currently using a work-a-round similar to the one you suggested. 

    1) I save the metadata after a successful synchronize().

    2) If any exception happens during synchronize()

       a. Delete the metadata and replace with the last know good metadata.

    This has been working quite well and I have not yet experienced any performance problems.

     

    Thank you for taking the time to reproduce the problem. 

    Will you let me know if or when this issue gets resolved in future releases?

    Thanks, Jim

    • Marked as answer by jim.software Thursday, November 4, 2010 6:20 PM
    Thursday, November 4, 2010 6:19 PM

All replies

  • That does seem a very long time. Have you tried running any profiling tools (Visual Studio or Red Gate ANTS Performance Profiler) to see exactly which code is taking so long to run? Which version of the Sync Framework are you using?
    Wednesday, October 27, 2010 7:53 PM
  • I'm using the Sync Framework v2.0.

    The function that is taking so long is SyncOrchestrator.Synchronize().  My profiler will not give me further details.

    It only takes a long time after I cancel the Synchronize operation or after I lose connection to my remote data path (ie network is unreachable).

     

    Wednesday, October 27, 2010 8:17 PM
  • What does the Sync Framework 2.0/2.1 expect the programmer to do after an aborted synchronize()?

     

    More information....


    1) First synchronize (4-5 minutes)
    2) Next synchronize with no file changes (2-3 minutes)
    3) Start another synchronize and abort it (call SyncOrchestrator.Cancel())
        a) Throws SyncAbortedExctption (expected)
       
    4) Next synchronze (10-50 minutes)
       This results in lots of uploads, downloads or both even though
       the filesystem has not changed.
      
       Immedetely After this synchronize()...
        a)  i.  Clean up tombstones (metadata.CleanupDeletedItems())
            ii. Next synchronize -> throws NotImplementedException
           
        b)  i.  DON'T Clean up tombstones
            ii. Next synchronize -> OK
           
           

    Normal Full Sync - Initial (No Metadata)
    -------------------------------------------------------------------------------
    Sync Start Time: 10/28/2010 10:37:45 AM
    Download Applied: 31078
    Download Failed: 0
    Download Total: 31078
    Upload Applied: 23432
    Upload Failed: 0
    Upload Total: 23432
    Sync End Time: 10/28/2010 10:42:02 AM


    Normal Full Sync - No Changes (Using current Metadata)
    -------------------------------------------------------------------------------
    Sync Start Time: 10/28/2010 11:37:45 AM
    Download Applied: 0
    Download Failed: 0
    Download Total: 0
    Upload Applied: 0
    Upload Failed: 0
    Upload Total: 0
    Sync End Time: 10/28/2010 11:39:22 AM



    After SyncAbortedException - No Changes in FileSystem (Using current Metadta)
    --Run A------------------------------------------------------------------------
    Sync Start Time: 10/28/2010 8:50:05 AM
    Download Applied: 3962
    Download Failed: 0
    Download Total: 3962
    Upload Applied: 0
    Upload Failed: 0
    Upload Total: 0
    Sync End Time: 10/28/2010 9:07:59 AM

    After SyncAbortedException - No Changes in FileSystem (Using current Metadta)
    --Run B-----------------------------------------------------------------------
    Sync Start Time: 10/29/2010 8:34:10 AM
    Download Applied: 1
    Download Failed: 0
    Download Total: 1
    Upload Applied: 7419
    Upload Failed: 1
    Upload Total: 7420
    Sync End Time: 10/29/2010 9:01:29 AM



    If I clean up tombstone items (ie CleanupDeletedItems()) after this step, the
    next synchronize() will throw NotImplementedException.

    Friday, October 29, 2010 4:40 PM
  • Even more info....

     

    This time it took 90 minutes (After a network failure).  And for most of that 90 minutes it was using 100% of one of my two CPUs.

    It's obvious, by the lack of responses to this thread, that no one is using MS Sync Framework (FileSyncProvider & SyncOrchestrator) for serious synchronization.  It is still a beta product that can only be used as a Sync Toy.

     

    1) First synchronize (4-5 minutes)
    2) Next synchronize with no file changes (2-3 minutes)
    3) Start another synchronize (between local and network share). Then, pull the network cable.
        a) (expected) Throws SyncException: The FileSyncProvider failed while detecting changes.
       
    4) Next synchronze (90 minutes)!!!!

    -------------------------------------------------------------------------------

    Sync Start Time: 10/29/2010 11:30:27 AM
    Download Applied: 8170
    Download Failed: 0
    Download Total: 8170
    Upload Applied: 0
    Upload Failed: 0
    Upload Total: 0
    Sync End Time: 10/29/2010 1:00:11 PM

    Friday, October 29, 2010 8:47 PM
  • Hi,

    May you provide more information for me to understand your scenario better?

    1. For your first sync, it only took 4-5 mins to sync totally 31078 + 23432 = 54510 files and folders. It is a lot of files. May I know what is the average size of your files? They should be pretty small files, right?

    2. Without sync, how fast for you to copy these files to the same remote share? Since your empty sync takes about 2-3 minutes that mainly change detection times, I assume the full copy time is less than 2 minutes for your file system. Can you verify if it is correct?

    3. After SyncAbortException, your next time took a lot time, but it is not an empty sync. They are thousands of changes synced in either download or upload syncs. May I know how many files are inserted/updated/deleted per hour on your file system? As you know, FileSyncProvider metadata also track your delete files/folders. I would lilke to guess how many deletes accumulated on your file system when you first hit a SyncAbortException.

    4. May you retry your above scenario by calling FileSyncProvider.DetectChanges for both providers before SyncOrchestrator.Sync() and measure how long the ChangeDetects() calls takes? You need to create your FileSyncProvider with FileSyncOptions.ExplicitDetectChanges in this case.

    5. Just want to confirm, after SyncAbortException, you didn't reuse the same FileSyncProvider instances to start a new sync, right?

    6. Last, because your initial sync didn't take very long, After SyncAbortException, have you tried to delete the .metadata file from both root folders before re-sync? If it can make the sync speed to be normal again, there might be some reasons to explain it.

    Thanks,
    Dong

     


    This posting is provided AS IS with no warranties, and confers no rights.
    Saturday, October 30, 2010 3:54 AM
    Moderator
  • 1) Yes, the files are small and there may actually be more directories than files.

    2) The total copy time is about 10-15 minutes.  The total sync time, when the local filesystem is empty, is about the same 10-15 minutes.

    3) After SyncAbortedException, I did not change any files on the local or remote filesystem.  It appears the metadata is corrupted or confused.

               This is a little complicated but shouldn't be hard to follow.

               a.  Using the last successful sync metadata after SyncAbortedException results in a quick normal sync.

                          i) Successful sync -> save metadata (this is a good copy of the metadata)

                          ii) Start sync, then, Abort -> delete metadata and replace with saved metadata from above (the good copy from a successful sync)

                          iii) Start sync, results in a normal 1-3 minute sync process

     

               No files were changed on the filesystem.  The only thing different between case (a.) and case (b.) is the metadata.

     

               b. Using the same metadata after SyncAbortedException results in long sync

                          i) Successful sync

                          ii) Start sync, then, Abort (do NOT delete or modify metadata)

                          iii) Start sync, results in a very long 15-90 minute sync

     

               From my testing, whenever SyncOrchestrator.Synchronize() throws an exception, the metadata is corrupted or confused.

               The following exceptions cause the next sync to take a very long time.

                          SyncAbortedExceptio, InvalidOperationException, SyncException  (I have not yet tested FileNotFoundException)

               Sometimes the metadata is so corrupted the next sync results in NotImplementedException and the metadata must be deleted.

     

    4) Looks like most of the time is spent in SyncOrchestrator.Synchronize()

    Normal Synchronization
        LocalFileSyncProvider.DetectChanges: Time in seconds:16
        RemoteFileSyncProvider.DetectChanges: Time in seconds:36
        ----------------------------------------------
        Sync Start Time: 11/1/2010 1:37:17 PM
        Download Applied: 0
        Download Failed: 0
        Download Total: 0
        Upload Applied: 0
        Upload Failed: 0
        Upload Total: 0
        Sync End Time: 11/1/2010 1:37:20 PM
        ----------------------------------------------
        SyncTime: 16+36+3 = 55 seconds
       
    After SyncAbortedException
        LocalFileSyncProvider.DetectChanges: Time in seconds:24
        RemoteFileSyncProvider.DetectChanges: Time in seconds:65
        ----------------------------------------------
        Sync Start Time: 11/1/2010 1:46:54 PM
        Download Applied: 8556
        Download Failed: 0
        Download Total: 8556
        Upload Applied: 0
        Upload Failed: 0
        Upload Total: 0
        Sync End Time: 11/1/2010 3:15:19 PM
        ----------------------------------------------
        SyncTime: 24+65+3 = 5305 seconds = 1 hr 29 min 54 sec

     

    5) I've tried it both ways, reusing the FileSyncProvider and not reusing it.  Also, I have completely stopped my application after the SyncAbortedException and restarted the application.  The results are the same for all cases, a long sync time.

     

    6) This is well explained in number 3). 

    Why would I want to delete my metadata files?  The sync framework cannot properly detect changes without the metadata.  That completely defeats the purpose of having the metadata to begin with.

     

    Thanks
    Jim

     

    Monday, November 1, 2010 4:36 PM
  • Just to let you know I updated 4).
    Monday, November 1, 2010 10:31 PM
  • Hi,

    I tried to write a sync app with FileSyncProvider to repro your scenario, but I cannot.

    I first did a bi-direction normal sync to bring bith folders in sync. After calling SyncOrchestrator.Cancel() in the first ApplyingChangeEvent of the second sync, the Sync session is stopped by SyncAbortException. When I tried to sync again immediately, I saw a normal empty sync that finished with 2-3 seconds. I have created enough items and one provider's change detection time is around 17 seconds.

    May you share the exact source code for repro?

    Thanks,
    Dong


    This posting is provided AS IS with no warranties, and confers no rights.
    Tuesday, November 2, 2010 7:06 AM
    Moderator
  • I'll see what I can drum up. 

    Thanks, Jim

    Tuesday, November 2, 2010 3:18 PM
  • Here are my test results using a test project.  This test project uses the same code for the sync providers and sync orchestrator as my original project.  Moreover, I get the same results.

     

    I can give you the entire test project if you have a place to share files.  Until you or I come up with a place to share files, I will post the entire content of the classes here.

     

    This post only contains data from my test.  The next post will contain the code.

     

    Size:           559 MB (587,043,458 bytes)
    Size on disk:   570 MB (598,253,568 bytes)
    Contains:       7,771 Files, 15,336 Folders

    1) Synchronize: Executing Synchronize() -> Complete (2 min 18 sec)
    2) Synchronize: Executing Synchronize() -> Cancel
    3) Synchronize: Executing Synchronize() -> Complete (2 min 18 sec)
    4) Synchronize: Executing Synchronize() -> Cancel
    5) Synchronize: Executing Synchronize() -> Complete (5 min 5 sec)
    6) Synchronize: Executing Synchronize() -> Cancel
    7) Synchronize: Executing Synchronize() -> Complete (22 min 10 sec)


    Synchronize: Executing Synchronize() Function
    Synchronize: Synchronize() Complete
    Synchronize Status
    Sync Start Time: 11/2/2010 9:48:46 AM
    Download Applied: 0
    Download Failed: 0
    Download Total: 0
    Upload Applied: 0
    Upload Failed: 0
    Upload Total: 0
    Sync End Time: 11/2/2010 9:51:04 AM
    The thread 0x131c has exited with code 0 (0x0).
    Synchronize: Executing Synchronize() Function
    Cancel: SyncOrchestrator Cancelling...
    A first chance exception of type 'Microsoft.Synchronization.SyncAbortedException' occurred in Microsoft.Synchronization.dll
    The thread 0x17f8 has exited with code 0 (0x0).
    Synchronize: Executing Synchronize() Function
    Synchronize: Synchronize() Complete
    Synchronize Status
    Sync Start Time: 11/2/2010 9:51:35 AM
    Download Applied: 0
    Download Failed: 0
    Download Total: 0
    Upload Applied: 0
    Upload Failed: 0
    Upload Total: 0
    Sync End Time: 11/2/2010 9:53:53 AM
    The thread 0x14d0 has exited with code 0 (0x0).
    Synchronize: Executing Synchronize() Function
    Cancel: SyncOrchestrator Cancelling...
    A first chance exception of type 'Microsoft.Synchronization.SyncAbortedException' occurred in Microsoft.Synchronization.dll
    The thread 0xa58 has exited with code 0 (0x0).
    Synchronize: Executing Synchronize() Function
    The thread 0x1740 has exited with code 0 (0x0).
    Synchronize: Synchronize() Complete
    Synchronize Status
    Sync Start Time: 11/2/2010 9:59:43 AM
    Download Applied: 1794
    Download Failed: 0
    Download Total: 1794
    Upload Applied: 0
    Upload Failed: 0
    Upload Total: 0
    Sync End Time: 11/2/2010 10:04:48 AM
    The thread 0x968 has exited with code 0 (0x0).
    Synchronize: Executing Synchronize() Function
    Cancel: SyncOrchestrator Cancelling...
    A first chance exception of type 'Microsoft.Synchronization.SyncAbortedException' occurred in Microsoft.Synchronization.dll
    The thread 0x15d4 has exited with code 0 (0x0).
    Synchronize: Executing Synchronize() Function
    The thread 0x111c has exited with code 0 (0x0).
    Synchronize: Synchronize() Complete
    Synchronize Status
    Sync Start Time: 11/2/2010 10:14:31 AM
    Download Applied: 5740
    Download Failed: 0
    Download Total: 5740
    Upload Applied: 0
    Upload Failed: 0
    Upload Total: 0
    Sync End Time: 11/2/2010 10:36:41 AM
    The thread 0xd2c has exited with code 0 (0x0).

    Tuesday, November 2, 2010 6:02 PM
  • The Form1 class is a basic form for testing

    --------------------------------------------------------------------
    |                                 |
    |  --------------------------------------------------------   |
    |  | local directory TextBox1              |   |
    |  --------------------------------------------------------   |
    |  --------------------------------------------------------   |
    |  | remote directory TextBox2             |   |
    |  --------------------------------------------------------   |
    |                                 |
    |    ---------------------   ---------------------     |
    |    | sync Button1   |   | cancel Button2  |     |
    |    ---------------------   ---------------------     |
    |                                 |
    |  --------------------------------------------------------   |
    |  | output display  TextBox3             |   |
    |  |                           |   |
    |  |                           |   |
    |  |                           |   |
    |  |                           |   |
    |  |                           |   |
    |  |                           |   |
    |  --------------------------------------------------------   |
    |                                 |
    --------------------------------------------------------------------
    

     

    Public Class Form1
    
      Private m_FileSynchronizer As FileSyncFileGroup = Nothing
    
      Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        If m_FileSynchronizer Is Nothing Then
          m_FileSynchronizer = New FileSyncFileGroup(SyncDirectionOrder.UploadAndDownload, TextBox1.Text, TextBox2.Text)
          AddHandler m_FileSynchronizer.SynchronizeComplete, AddressOf SyncProcessCompleted
    
          TextBox1.Enabled = False
          TextBox2.Enabled = False
        End If
        m_FileSynchronizer.StartSynchProcess()
      End Sub
    
      Private Delegate Sub SyncProcessCompletedDelegate(ByVal syncResult As String)
      Private Sub SyncProcessCompleted(ByVal syncResult As String)
        Dim i As System.ComponentModel.ISynchronizeInvoke = _
            CType(TextBox3, System.ComponentModel.ISynchronizeInvoke)
    
        If i.InvokeRequired Then
          Dim tempDelegate As New SyncProcessCompletedDelegate(AddressOf SyncProcessCompleted)
          Dim args() As Object = {syncResult}
          i.BeginInvoke(tempDelegate, args)
          Return
        End If
    
        TextBox3.Text = TextBox3.Text + vbCrLf + syncResult
      End Sub
    
      Private Sub Button2_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button2.Click
        If m_FileSynchronizer IsNot Nothing Then
          m_FileSynchronizer.CancelSynchProcess()
        End If
      End Sub
    End Class
    
    
    
    Public Class FileSyncFileGroup
    
      Public Event SynchronizeComplete(ByVal syncResult As String)
    
      Private m_SyncDirection As SyncDirectionOrder = SyncDirectionOrder.UploadAndDownload
      Private m_LocalDir As String = String.Empty
      Private m_RemoteDir As String = String.Empty
    
      Private m_SyncOrchestrator As SyncOrch = Nothing
    
      Private m_NeedsFullSynchronization As Boolean = False
      Private m_WorkerThread As BackgroundWorker = Nothing
      Private m_SynchronizeThread As Thread = Nothing
    
      Private Sub New()
        ' Disallow Default Constructor
      End Sub
    
      Public Sub New( _
        ByVal nDirection As SyncDirectionOrder, _
        ByVal nLocalPath As String, _
        ByVal nRemotePath As String)
    
        MyBase.New()
    
        Dim mLocalDir As DirectoryInfo = Nothing
        Dim mRemoteDir As DirectoryInfo = Nothing
    
        m_SyncDirection = nDirection
        m_LocalDir = nLocalPath
        mLocalDir = New DirectoryInfo(nLocalPath)
        If mLocalDir Is Nothing Then
          Throw New DirectoryNotFoundException("Local Path is invalid: " + nLocalPath)
        ElseIf Not mLocalDir.Exists Then
          Directory.CreateDirectory(mLocalDir.FullName)
        End If
    
        m_RemoteDir = nRemotePath
        mRemoteDir = New DirectoryInfo(nRemotePath)
        If mRemoteDir Is Nothing Then
          Throw New DirectoryNotFoundException("Remote Path is invalid: " + nLocalPath)
        ElseIf Not mRemoteDir.Exists Then
          ' Let's not do this....
          ' The network might be down or temporally off line when we start
          'Throw New DirectoryNotFoundException("Remote Path does not exist: " + nLocalPath)
        End If
    
        m_SyncOrchestrator = New SyncOrch(Me)
    
        m_WorkerThread = New BackgroundWorker()
        m_WorkerThread.WorkerSupportsCancellation = True
        AddHandler m_WorkerThread.DoWork, AddressOf MonitorFullSyncStatus
        m_WorkerThread.RunWorkerAsync()
    
      End Sub
    
      Private Function CheckLocalPathIsAvailable() As Boolean
        Return Directory.Exists(m_LocalDir)
      End Function
    
      Private Function CheckRemotePathIsAvailable() As Boolean
        Return Directory.Exists(m_RemoteDir)
      End Function
    
      Private Function BothPathsAvailable() As Boolean
        Return (CheckRemotePathIsAvailable() And CheckLocalPathIsAvailable())
      End Function
    
      Private Sub MonitorFullSyncStatus(ByVal sender As System.Object, ByVal e As System.ComponentModel.DoWorkEventArgs)
        Dim worker As BackgroundWorker = CType(sender, BackgroundWorker)
    
        m_NeedsFullSynchronization = False
    
    
        While Not worker.CancellationPending
    
          Try
            If (m_NeedsFullSynchronization AndAlso BothPathsAvailable()) Then
              If (Not SynchronizeThreadIsAlive()) Then
                StartSynchProcess()
              End If
            ElseIf (Not BothPathsAvailable()) Then
              ' Make sure the sync process is not running while the network path is not available.
              ' The sync process tends to get into an infinate loop when it loses the network path.
              CancelSynchProcess()
            End If
    
          Catch ex As Exception
            Console.WriteLine("MonitorFullSyncStatus: Exception: " + ex.ToString)
    
          End Try
    
          Thread.Sleep(10000)
    
        End While
    
      End Sub
    
      Private Sub Synchronize()
        Dim syncResult As String
    
        Try
          syncResult = m_SyncOrchestrator.Synchronize()
    
          If (syncResult.Contains("Success:")) Then
            m_NeedsFullSynchronization = False
          End If
    
        Catch ex As ThreadInterruptedException
          syncResult = "ERROR: The File Sync operation has been canceled."
    
        Catch ex As Exception
          syncResult = "ERROR: Exception: The File Sync operation has thrown an exception." + vbNewLine + ex.ToString
        End Try
    
        RaiseEvent SynchronizeComplete(syncResult)
      End Sub
    
      Private Function SynchronizeThreadIsAlive() As Boolean
        SynchronizeThreadIsAlive = False
    
        If (m_SynchronizeThread IsNot Nothing) Then
          SynchronizeThreadIsAlive = m_SynchronizeThread.IsAlive
        End If
    
      End Function
    
      Public Sub StartSynchProcess()
    
        If (SynchronizeThreadIsAlive()) Then
          Exit Sub
        End If
    
        m_SynchronizeThread = New Thread(AddressOf Synchronize)
        m_SynchronizeThread.Priority = ThreadPriority.Lowest
        m_SynchronizeThread.Start()
    
      End Sub
    
      Public Sub CancelSynchProcess()
        m_SyncOrchestrator.Cancel()
    
        ' Wait a few seconds for the thread to die
        Dim tenSecTimer As New Stopwatch()
        tenSecTimer.Start()
    
        While (SynchronizeThreadIsAlive())
          If tenSecTimer.ElapsedMilliseconds > 10000 Then
            tenSecTimer.Stop()
            tenSecTimer = Nothing
            Exit While
          End If
          Thread.Sleep(100)
        End While
    
      End Sub
    
      Public ReadOnly Property OrchestratorSyncDirection() As SyncDirectionOrder
        Get
          Return m_SyncDirection
        End Get
      End Property
    
      Public ReadOnly Property LocalDir() As String
        Get
          Return m_LocalDir
        End Get
      End Property
    
      Public ReadOnly Property RemoteDir() As String
        Get
          Return m_RemoteDir
        End Get
      End Property
    
      Public ReadOnly Property SyncOptions() As FileSyncOptions
        Get
          SyncOptions = _
            FileSyncOptions.RecycleConflictLoserFiles Or _
            FileSyncOptions.RecycleDeletedFiles Or _
            FileSyncOptions.RecyclePreviousFileOnUpdates
        End Get
      End Property
    
    
      Public ReadOnly Property SyncState() As SyncOrchestratorState
        Get
          Dim retSyncState As SyncOrchestratorState
          retSyncState = m_SyncOrchestrator.SyncState
    
          If (retSyncState = SyncOrchestratorState.Ready) Or (retSyncState = SyncOrchestratorState.Canceled) Then
            If (SynchronizeThreadIsAlive()) Then
              retSyncState = SyncOrchestratorState.UploadingAndDownloading
            End If
          End If
    
          Return retSyncState
        End Get
      End Property
    
      Public Property NeedsFullSynchronization() As Boolean
        Get
          Return m_NeedsFullSynchronization
        End Get
        Set(ByVal value As Boolean)
          m_NeedsFullSynchronization = value
        End Set
      End Property
    
    End Class
    
    
    
    
    
    Imports System.Reflection
    
    Public Class SyncOrch
    
    
      Private Shared m_MetadataStorageBasePath As String = String.Empty
    
      Private m_LocalFileSyncProvider As FileSyncProvider = Nothing
      Private m_RemoteFileSyncProvider As FileSyncProvider = Nothing
      Private m_SyncOrchestrator As SyncOrchestrator = Nothing
      Private m_LastSuccessfulSyncTime As Date = Nothing
    
      Private m_SyncGroup As FileSyncFileGroup = Nothing
      Private m_ScopeFilter As FileSyncScopeFilter = Nothing
    
      Private Sub New()
        ' Disallow Default Constructor
      End Sub
    
      Public Sub New(ByRef nSyncGroup As FileSyncFileGroup)
    
        MyBase.New()
    
        Try
          m_SyncGroup = nSyncGroup
    
          m_SyncOrchestrator = New SyncOrchestrator()
          m_SyncOrchestrator.Direction = m_SyncGroup.OrchestratorSyncDirection
    
        Catch ex As Exception
          Console.WriteLine("New: Exception: " + ex.ToString)
        End Try
    
      End Sub
    
    
      Public Function Synchronize() As String
        Dim stats As Microsoft.Synchronization.SyncOperationStatistics = Nothing
        Dim syncResult As String = "ERROR: Sync Not Started for FileGroup "
    
        Try
          If (m_SyncOrchestrator.State = SyncOrchestratorState.Ready) Or (m_SyncOrchestrator.State = SyncOrchestratorState.Canceled) Then
            Console.WriteLine("Synchronize: Executing Synchronize() Function")
    
            m_SyncOrchestrator.Direction = m_SyncGroup.OrchestratorSyncDirection
            stats = SyncOrchestrator.Synchronize()
            syncResult = "Success: "
            m_LastSuccessfulSyncTime = Now
    
            Console.WriteLine("Synchronize: Synchronize() Complete")
          Else
            Console.WriteLine("Synchronize: SyncOrchestrator is busy.... NOT Executing Synchronize() Function")
            syncResult = "In-Process: Sync Busy "
          End If
        Catch ioex As InvalidOperationException
          ' Thrown by SyncOrchestrator if State is Not Ready and Not Canceled
          syncResult = "In-Process(Operation-based): "
        Catch saex As SyncAbortedException
          ' Thrown by SyncOrchestrator on call to Synchronize when the
          ' FileSyncProvider operation has been canceled..
          syncResult = "ERROR: SyncAbortedException: The FileSyncProvider operation has been canceled."
    
        Catch fnfex As FileNotFoundException
          ' Thrown by SyncOrchestrator on call to Synchronize when the
          ' Local or Remote folder is no longer available. Probable cause is network disconnect.
          syncResult = "ERROR: FileNotFoundException: The FileSyncProvider failed while detecting changes." + _
            vbNewLine + "Probable cause is RemoteDir <" + m_SyncGroup.RemoteDir + "> is inaccessible."
    
        Catch syncex As SyncException
          ' Thrown by SyncOrchestrator on call to Synchronize when the
          ' Local or Remote folder is no longer available. Probable cause is network disconnect.
          If (syncex.Message.Contains("The FileSyncProvider failed while detecting changes.")) Then
            syncResult = "ERROR: SyncException: The FileSyncProvider failed while detecting changes." + _
              vbNewLine + "Probable cause is RemoteDir <" + m_SyncGroup.RemoteDir + "> is inaccessible due to network disconnect."
          Else
            syncResult = "ERROR: SyncException: " + syncex.ToString
          End If
    
        Catch ex As NotImplementedException
          syncResult = "ERROR: NotImplementedException: The File Sync operation has tried to execute a function that was not implemented."
    
        Catch otherex As Exception
          syncResult = "ERROR: Exception: " + otherex.ToString
    
        Finally
          If stats IsNot Nothing Then
            ' Display the SyncOperationStatistics
            Dim statusString As String = _
            vbNewLine & _
            "Sync Start Time: " & stats.SyncStartTime & _
            vbNewLine & _
            "Download Applied: " & stats.DownloadChangesApplied & _
            vbNewLine & _
            "Download Failed: " & stats.DownloadChangesFailed & _
            vbNewLine & _
            "Download Total: " & stats.DownloadChangesTotal & _
            vbNewLine & _
            "Upload Applied: " & stats.UploadChangesApplied & _
            vbNewLine & _
            "Upload Failed: " & stats.UploadChangesFailed & _
            vbNewLine & _
            "Upload Total: " & stats.UploadChangesTotal & _
            vbNewLine & _
            "Sync End Time: " & stats.SyncEndTime
    
            Console.WriteLine("Synchronize Status " + statusString)
          End If
        End Try
    
        Return syncResult
    
      End Function
    
      Public Sub Cancel()
        Try
          Console.WriteLine("Cancel: SyncOrchestrator Cancelling...")
    
          If (m_SyncOrchestrator IsNot Nothing) _
            AndAlso ((m_SyncOrchestrator.State > SyncOrchestratorState.Ready) _
            And (m_SyncOrchestrator.State < SyncOrchestratorState.Canceling)) Then
    
            m_SyncOrchestrator.Cancel()
    
          End If
    
        Catch ex As Exception
          ' NOTE: May Throw System.InvalidOperationException:
          ' This operation is not valid in any state of the object
          ' except for SyncAgentState.Uploading or
          ' SyncAgentState.Downloading. - IGNORE
          Console.WriteLine("Cancel: Threw exception (IGNORE) " + ex.ToString)
        End Try
    
      End Sub
    
    
      Private Function GetLocalMetadataFileName() As String
        Return "local.metadata"
      End Function
    
      Private Function GetRemoteMetadataFileName() As String
        Return "remote.metadata"
      End Function
    
      Private Function GetLocalTempPath() As String
        Dim result As String
    
        result = MetadataStoragePath() + "local.temp"
    
        If Not Directory.Exists(result) Then
          Directory.CreateDirectory(result)
        End If
    
        Return result
      End Function
    
      Private Function GetRemoteTempPath() As String
        Dim result As String
    
        result = MetadataStoragePath() + "remote.temp"
    
        If Not Directory.Exists(result) Then
          Directory.CreateDirectory(result)
        End If
    
        Return result
      End Function
    
      Private Function GetLocalConflictLoserPath() As String
        Dim result As String
    
        result = MetadataStoragePath() + "local.conflict"
    
        If Not Directory.Exists(result) Then
          Directory.CreateDirectory(result)
        End If
    
        Return result
      End Function
    
      Private Function GetRemoteConflictLoserPath() As String
        Dim result As String
    
        result = MetadataStoragePath() + "remote.conflict"
    
        If Not Directory.Exists(result) Then
          Directory.CreateDirectory(result)
        End If
    
        Return result
      End Function
    
      Private Function MetadataStoragePath() As String
        Dim result As String
    
        ' TBD: Should this be a config parameter?
        If (String.IsNullOrEmpty(m_MetadataStorageBasePath)) Then
          m_MetadataStorageBasePath = Path.GetDirectoryName(Assembly.GetEntryAssembly.Location)
        End If
    
        result = Path.Combine(m_MetadataStorageBasePath, "MEIFileSyncServiceMetadata\" + "\")
    
        If Not Directory.Exists(result) Then
          Directory.CreateDirectory(result)
        End If
    
        Return result
      End Function
    
      Public ReadOnly Property LastSuccessfullSyncTime() As Date
        Get
          Return m_LastSuccessfulSyncTime
        End Get
      End Property
    
      Public ReadOnly Property SyncState() As SyncOrchestratorState
        Get
          If (m_SyncOrchestrator IsNot Nothing) Then
            Return m_SyncOrchestrator.State
          Else
            Return SyncOrchestratorState.Canceled
          End If
    
        End Get
      End Property
    
      Private ReadOnly Property ScopeFilter() As FileSyncScopeFilter
        Get
          If m_ScopeFilter Is Nothing Then
            m_ScopeFilter = New FileSyncScopeFilter()
            m_ScopeFilter.AttributeExcludeMask = Nothing
            m_ScopeFilter.FileNameIncludes.Clear()
            m_ScopeFilter.FileNameExcludes.Clear()
            m_ScopeFilter.SubdirectoryExcludes.Clear()
          End If
          Return m_ScopeFilter
        End Get
      End Property
    
      Private ReadOnly Property LocalFileSyncProvider() As FileSyncProvider
        Get
          If m_LocalFileSyncProvider Is Nothing Then
            Try
    
              m_LocalFileSyncProvider = New FileSyncProvider( _
                  m_SyncGroup.LocalDir, _
                  Me.ScopeFilter, _
                  m_SyncGroup.SyncOptions, _
                  Me.MetadataStoragePath, _
                  GetLocalMetadataFileName(), _
                  GetLocalTempPath(), _
                  Nothing)
    
            Catch exDir As DirectoryNotFoundException
              ' This exception is expected when the local directory is not available
              Console.WriteLine("LocalFileSyncProvider: Could not create Local Provider because Local Directory not available: " + vbCrLf + exDir.ToString)
              m_LocalFileSyncProvider = Nothing
    
            Catch ex As Exception
              Console.WriteLine("LocalFileSyncProvider: Could not create Local Provider: " + vbCrLf + ex.ToString)
              m_LocalFileSyncProvider = Nothing
            End Try
    
          End If
          Return m_LocalFileSyncProvider
        End Get
      End Property
    
      Private ReadOnly Property RemoteFileSyncProvider() As FileSyncProvider
        Get
          If m_RemoteFileSyncProvider Is Nothing Then
            Try
    
              m_RemoteFileSyncProvider = New FileSyncProvider( _
                  m_SyncGroup.RemoteDir, _
                  Me.ScopeFilter, _
                  m_SyncGroup.SyncOptions, _
                  Me.MetadataStoragePath, _
                  GetRemoteMetadataFileName(), _
                  GetRemoteTempPath(), _
                  Nothing)
    
            Catch exDir As DirectoryNotFoundException
              ' This exception is expected when the local directory is not available
              Console.WriteLine("RemoteFileSyncProvider: Could not create Remote Provider because Remote Directory not available: " + vbCrLf + exDir.ToString)
              m_RemoteFileSyncProvider = Nothing
    
            Catch ex As Exception
              Console.WriteLine("RemoteFileSyncProvider: Could not create Remote Provider: " + vbCrLf + ex.ToString)
              m_RemoteFileSyncProvider = Nothing
            End Try
    
          End If
          Return m_RemoteFileSyncProvider
        End Get
      End Property
    
      Private ReadOnly Property SyncOrchestrator() As SyncOrchestrator
        Get
          ' This is here because sometimes the network is not available
          ' or the network path is not available during startup.
          ' When this happens, the remote path will not be accessable
          ' and the RemoteFileSyncProvider will be Nothing.
          ' Using the properties and assigning them each time will attempt
          ' to create the RemoteFileSyncProvider in the case where it
          ' is Nothing. If RemoteFileSyncProvider is valid, it just
          ' returns the m_RemoteFileSyncProvider class variable.
          m_SyncOrchestrator.LocalProvider = LocalFileSyncProvider()
          m_SyncOrchestrator.RemoteProvider = RemoteFileSyncProvider()
          Return m_SyncOrchestrator
        End Get
      End Property
    
    
    End Class
    
    
    
    

    Tuesday, November 2, 2010 6:13 PM
  • Hi,

    Thanks for sharing the implemetation, but I may not be able to create a repro with your code because it looks like a combination of multiple partial files. It is difficult to create a project with them.

    If you can confirm below points in your implementation, I can try to repro again:

    1. Sync is started in a separate thread, and Cancel() is called from main thread after some wait time.

    2. SyncOrchestrator instance is re-created for every sync

    3. SyncOptions is RecycleConflitLoserFiles & RecyclDeletedFiles & RecyclePreviousFileOnUpdates. For remote share, recycle bin is not available. Did you set the same SyncOptions for remote provider?

    4. Metadata Folder for both providers are in local directory

    5. Need multiple retries to hit one repro, or repro is intermittent.

     

    I have two more quesitons for you:

    1. When hit a repro, did every sync after the SyncAbortException take very long time or just the first one?

    2. You didn't explicitly call ChangeDetection method and didn't disable provider implicit change detection. It means the cancel() may happen in the middle of change detection, is it correct understanding?

    Thanks,
    Dong


    This posting is provided AS IS with no warranties, and confers no rights.
    Wednesday, November 3, 2010 1:40 AM
    Moderator
  • Yes the project is 3 files; 1 form and 2 classes.  Each class is its own file.  I tried to draw the layout of the form but it did not come through very clearly.

    I tried to reproduce, in a temporary project, the same code that my full project is using.

     

    1. Yes, Sync is started in a separate thread and the cancel is triggered from a button click in the UI.

    2. SyncOrchestrator is NOT recreated each time.  My testing has confirmed that this is irrelevant. Using the same SyncOrchestrator or creating a new one does not make any difference as far as this issues is concerned.

    3. SyncOptions = FileSyncOptions.RecycleConflictLoserFiles Or FileSyncOptions.RecycleDeletedFiles Or FileSyncOptions.RecyclePreviousFileOnUpdates

    This is true for both the local and remote provider.

    4. Yes, metadata is stored locally for both providers.

    5. True.  In some cases I have reproduced the problem in one try.  I provided my test data "as is".  I have no explanation for why it took multiple attempts to repro.

     

    Questions:

    1. Just the first one.  After the long sync time, things return normal.

    2. I removed the explicit change detection code.  However, the cancel is most likely happening during the change detection.  If I remember correctly, using explicit change detection, the cancel happened during the DetectChanges() function.  This resulted in a SyncAbortedExctption.  On the following sync attempt, the DetectChanges happened very quickly but the Synchronize() took a very long time.

    My comment on Monday, November 01, 2010 4:36 PM, answer to question 4) has the details of what happened when I used the explicit detect changes call.


     

    Wednesday, November 3, 2010 3:54 PM
  • Hi,

    I can repro your scenario now. And it is a bug in FileSyncProvider metadata handling during change detection. It only happens when change detection fails in the middle. We will investigate this issue and consider a fix in the next SyncFx release. Thanks a lot for help us find this defect.

    With current FileSyncProvider, is it possible for you to avoid cancelling or failing a sync during provider ChangeDetection call? If not, you may want to call FileSyncProvider.ChangeDetect explicitly, and keep a copy of the .metadata file aside to replace the bad one when change detection fails in the middle. Considering there is no data lost but just performance impact, you can decide if this workaround is needed for you or not.

    Thanks,
    Dong 


    This posting is provided AS IS with no warranties, and confers no rights.
    Thursday, November 4, 2010 5:56 PM
    Moderator
  • I am currently using a work-a-round similar to the one you suggested. 

    1) I save the metadata after a successful synchronize().

    2) If any exception happens during synchronize()

       a. Delete the metadata and replace with the last know good metadata.

    This has been working quite well and I have not yet experienced any performance problems.

     

    Thank you for taking the time to reproduce the problem. 

    Will you let me know if or when this issue gets resolved in future releases?

    Thanks, Jim

    • Marked as answer by jim.software Thursday, November 4, 2010 6:20 PM
    Thursday, November 4, 2010 6:19 PM