locked
Regarding FileSyncProvider RRS feed

  • Question

  • Achieve:

    I am trying to sync 2 folders (source and destination) it is one way sync i.e. upload, which I can do successfully.

    After synchronizing I want to delete the files from the source folder.

    In short I want to use Sync Framework to move files.  Is this possible? If yes then how can this be achieved?

     

     

    Thursday, January 12, 2012 3:06 PM

Answers

  • my apologies for the late reply. the code is a bit lengthy and contains some other stuff.

    anyway to answer your other question, files that are in use are actually skipped by Sync Framework.

    If you subscribe to the SkippedChange event, you'll get the name of the file being skipped. Files that are in use will cause a read error and the event argument's SkipReason will be set to ReadError.

    You can use this event to set a flag to determine if files has been skipped in a sync. If files has been skipped, you can call Synchronize for another pass to retry synching the skipped files.

     

    • Marked as answer by arm007 Monday, January 30, 2012 2:47 PM
    Thursday, January 26, 2012 2:22 AM

All replies

  • are you just trying to sync new files?

    Friday, January 13, 2012 9:58 AM
  • yes newly created files in the source folder.
    Friday, January 13, 2012 1:58 PM
  • Here's what I am trying to do : 

    I want to transfer files (move) over the network from 2 different machines (as new files are created on these machine) to a server.

    - My first approach was to use a filesystemwatcher to watch the folder on the machine and on new file create event do a simple move(System.io). But these files are moving through network and I dont trust move.

    - My second approach is to use Sync framework as it elegantly handles file transfer. So, to start with I still have a filesystemwatcher to watch the folder on the machine.

     - Once the event is triggered I am doing a one way sync(upload) between the folder and temporary folder on the server,  - after successful sync I do a move on the files in the temporary folder to the actual server folder where the files suppose to             be,

     - then I do a sync(download) again so the source would not contain any files.

    following is my code

     

    //file watcher event handler when new file is created
    private void Onchange(object source, FileSystemEventArgs e)
    {
        FileSyncProvider sourceProvider = null;
        FileSyncProvider destinationProvider = null;
    
        if (!Directory.Exists(this.TempSyncFolderPath))
        {
            Directory.CreateDirectory(this.TempSyncFolderPath);
        }
    
        int fileCount = 0;
        try
        {
            StopWatcher();
    
            sourceProvider = new FileSyncProvider(this.FromPath, null, FileSyncOptions.None);
            destinationProvider = new FileSyncProvider(this.TempSyncFolderPath, null, FileSyncOptions.None);
    
            sourceProvider.DetectChanges();
            destinationProvider.DetectChanges();
    
            SyncOrchestrator agent = new SyncOrchestrator();
            agent.LocalProvider = sourceProvider;
            agent.RemoteProvider = destinationProvider;
            agent.Direction = SyncDirectionOrder.Upload; // Sync source to destination
            agent.Synchronize();
            
            //move files from tempsyncfolder to topath
            foreach (string file in Directory.GetFiles(this.TempSyncFolderPath, "*", SearchOption.TopDirectoryOnly)
                .Where(s => !s.EndsWith(".metadata")))
            {
                File.Move(file, Path.Combine(this.ToPath, Path.GetFileName(file)));
                fileCount++;
            }
    
            sourceProvider.DetectChanges();
            destinationProvider.DetectChanges();
    
            agent.Direction = SyncDirectionOrder.Download;
            agent.Synchronize();
        }
        catch (Exception ex)
        {
            System.Console.WriteLine("Error : {0}", ex.Message);
            StopWatcher();
        }
        finally
        {
            StartWatcher();
            // Release resources
            if (sourceProvider != null) sourceProvider.Dispose();
            if (destinationProvider != null) destinationProvider.Dispose();
            System.Console.WriteLine("From {0} - To {1} : {2} files copied",this.FromPath,this.ToPath,fileCount);
        }
    }
    

    This code works but if a big file is created this code does not work? Any suggestions?

     

     




    • Edited by arm007 Friday, January 13, 2012 3:55 PM
    Friday, January 13, 2012 3:47 PM
  • What do mean it doesn't work if a big file is created?  What is the error?

    -Jesse

    Friday, January 13, 2012 9:26 PM
  • Actually I figured where the problem was -

    - My function gets call when a new file is created. The first step I do stop the "FileSystemWatcher" and do sync and then again start the "FileSystemWatcher".

    - At a given moment in the FolderWatcher machine multiple files are created. 

    - So the event triggers at first file created. In sometimes while the sync is going on new file are still been created which the event is not triggered as I am setting the  FileSystemWatcher false. Some files stay their at the source location, which will be picked up at next file create event

    - But then I have to depend on the next create event to pull those files from the source folder

     

    So my question is there a way I can get away with this filesystemwatcher and still can continuously monitor the source folder?

     

    Friday, January 13, 2012 9:49 PM
  • Can you just kick off a sync every time the event is called instead of disabling filesystemwatcher?  Or would that be too much overhead?

    Friday, January 13, 2012 10:06 PM
  • unfortunately, there is no facility in sync framework to continously monitor the source folder. the synchronization needs to be explicitly invoked.

    even if you invoke the synchronization 2x or more in the same filesystemwatcher event, there is still a chance a file is created after sync framework is done enumerating changes.

    if you're worried that a file is not immediately synched because the next file create event takes long to happen, try a scheduled sync instead at fix intervals (say every 5 mins). that way at least you can predict when a file will be synched.

    Saturday, January 14, 2012 1:45 AM
  • @Jesse L - I am new to this sync framework so according to my reading and research on this framework I didn't find any way of doing this.

    @JuneT - That seems to be a drawback. I believe having a timed system will still create problems as suppose if I am doing a sync after every 5 mins (i.e. as per my above function upload first,then move followed by download)  and at that given point a new file is been created which is big enough and might not be completely created during this sync operation. This will create conflicts and the whole file would not be synced properly(as I would be moving during this process). 

    I might be thinking too much but I care about the data and want to handle worst possible situation.

    Now this makes me think about using the sync framework!

     

     

     

    Monday, January 16, 2012 1:57 PM
  • you can put some code in your conflict handler to check if the file is still in use by another process (assuming the write operation of the source app is still writing the file) and do a wait before setting conflict resolution.

    i have done something similar where i have a loop trying to open the file and if there's an access exception, i assume the file is in use by another process and simply wait for it to be freed up.

    Tuesday, January 17, 2012 1:52 AM
  • @JuneT - can you give me a sample code for handling the file conflict.

     

    Tuesday, January 17, 2012 3:48 AM
  • let me look it up when i get home.
    Tuesday, January 17, 2012 10:24 AM
  • @ JuneT - do you still have an example of handling file conflict 
    Friday, January 20, 2012 7:58 PM
  • How can I restrict FileSyncProvider/SyncOrchestrator skip sync new files which are not done creating during the process of sync? 

    Wednesday, January 25, 2012 4:45 PM
  • my apologies for the late reply. the code is a bit lengthy and contains some other stuff.

    anyway to answer your other question, files that are in use are actually skipped by Sync Framework.

    If you subscribe to the SkippedChange event, you'll get the name of the file being skipped. Files that are in use will cause a read error and the event argument's SkipReason will be set to ReadError.

    You can use this event to set a flag to determine if files has been skipped in a sync. If files has been skipped, you can call Synchronize for another pass to retry synching the skipped files.

     

    • Marked as answer by arm007 Monday, January 30, 2012 2:47 PM
    Thursday, January 26, 2012 2:22 AM
  • Great that's what I figured out experimenting.

    One more question is there any more good resources for File Sync Framework other than Microsoft Sync documentation. Also, where I can find some sample examples to experiment on. 

    Appreciate for your help.     

    Thursday, January 26, 2012 1:58 PM
  • So in what scenarios files are deleted or send to recycle bin? Does file with same name the only possible reason?

    In short what are the possible scenarios?

     

     

    Thursday, January 26, 2012 7:38 PM
  • There a few scenarios depending on which flags you set on the FileSyncOptions, which are fairly self explantory:

    FileSyncOptions.RecycleDeletedFiles
    FileSyncOptions.RecyclePreviousFileOnUpdates
    FileSyncOptions.RecycleConflictLoserFiles

    You can find a few File Sync samples on Code Gallery:

    http://code.msdn.microsoft.com/site/search?f%5B0%5D.Type=SearchText&f%5B0%5D.Value=Microsoft%20Sync%20Framework&f%5B1%5D.Type=Affiliation&f%5B1%5D.Value=Official&f%5B1%5D.Text=Microsoft

    • Marked as answer by Yunwen Bai Monday, January 30, 2012 6:58 AM
    • Unmarked as answer by arm007 Monday, January 30, 2012 2:47 PM
    Thursday, January 26, 2012 7:53 PM