Create the Hash DB only option? RRS feed

  • Question

  • I have a 2TB drive i am trying to keep in sync with another 2TB drive at my parents house. About 1.4TB of data. Currently out of Sync but all changes are going to be from my PC to my Dad's at this point due to the fact i didn't let the other software i have been using do it's sync last night because i wanted to try SyncToy. I have been using another application to do this but it does not sync changes like renames or moves so I wanted to give SyncToy a try. Other than that I have had no issues syncing both ways through the VPN. Just sick of sending 2GB every night just because I moved or renamed a file.

    This drive mainly has media files on it both video and audio.

    When doing the preview, (Watching in Process Explorer to see what files it's on [lets get the SyncToy GUI to show us this info in future versions!]) it is going so very slow checking the files over the VPN. I don't know when it will finish if ever! It is getting through the local files OK considering this is the 1st "scan" and it needs to create the hash for each file.


    What I would like to know is, is there a way to make the SyncToy database locally on my Dad's computer so it doesn't have to make it through the VPN and then just schedule SyncToy on my PC to sync to the shared drive at 1AM and have it use the pre made hash DB? It has been stuck on one .avi video file for the last 3 hours and that is only the 1st of many files!

    Tuesday, November 30, 2010 8:01 PM

All replies

  • A day later the Preview is finally compleate. Sort of..


    New 14,942 Create folder 746

    Total bytes to copy 1,458,307,396,406

    Avoided copying 54,599,888,256 bytes in 61 files.

    There is no way I am going to be able to sync 1.5TB over the VPN! The preview window shows no last modified date for anything on the right side.

    At this point I don't think it fully scanned the right side which was what I was afraid of yesterday when I saw how long it was taking to scan just one file.

    Here is the log if it's of any help:


    SYNC: 11/30/2010 09:42:52:755: -------------------------------------------------------------------------
    SYNC: 11/30/2010 09:42:52:758: Starting SyncToy, version, built 10/19/2009 4:04:38 AM.
    SYNC: 11/30/2010 09:43:26:557: Started scanning directory : E:\
    SYNC: 11/30/2010 09:43:26:557: Started scanning directory : X:\
    SYNC: 11/30/2010 23:17:24:036: Stopped scanning directory : E:\
    SYNC: 12/01/2010 02:16:37:653: Stopped scanning directory : X:\
    SYNC: 12/01/2010 02:22:36:556: Preview of Media Sync - DAD (E:\, X:\) in time 16:39:09:420.
    SyncToy action was 'Synchronize'
    Found 15678 actions to perform.
    Found 61 files that did not require action.
    Analyzed 0.0 files per second.
    Avoided copying 54,599,888,256 bytes in 61 files.
    Saved approximately 06:19:59:00 by not copying any files.


    As you can see it says "Analyzed 0.0 files per second." which isn't very encouraging.

    Anyone have any ideas what I should do next? Is SyncToy just not going to work for me? Any help would be appreciated!


    Wednesday, December 1, 2010 4:17 PM
  • So I went ahead and tried to sync.

    It ran for over 24hrs and didn't sync a thing! I tried to cancel the sync...

    At this point Process Explorer wasn't even showing SyncToy accessing any of the files so i figured it was hung.

    I eventually had to kill it using Process Explorer.


    Is this just more data than SyncToy can handle?


    I deleted all the logs/bin files and will try again this weekend.

    Friday, December 3, 2010 6:57 PM
  • So I got another 2TB drive and figured I would try to use SyncToy localy only and see if I still ran into issues.


    Well... this is just the Preview log.

    SYNC: 12/06/2010 06:55:42:170: Preview of Media Sync - Brother (E:\, Y:\) in time 13:18:27:614.

    SyncToy action was 'Synchronize'
    Found 15917 actions to perform.
    Found 39 files that did not require action.
    Analyzed 0.0 files per second.
    Avoided copying 10,474,936,246 bytes in 39 files.
    Saved approximately 05:49:09:00 by not copying any files.

    Over 13 Hours!! Just for the Preview!?

    I Started the Sync this morning and will see how long it takes. Hopefully less than 13 hours!

    So far it has transferred 285GB and the completion bar isn't even 1/10 full. And it's been 6 hours already.

    It says failed on one file too but guess I can't find out what file that is till it's done.

    Monday, December 6, 2010 9:56 PM
  • The sync finally finished with 1 error and it appears it was just a locked file.

    This is a Local sync from 1 USB drive to another.

    SYNC: 12/07/2010 09:15:50:516: SyncToy run of Media Sync - Brother (E:\, Y:\) completed at 12/7/2010 9:15:50 AM.

    SyncToy action was 'Synchronize'.
    SyncToy options were:
        Active for run all
        All files included
        No files excluded
        Check file contents
        Include read-only files
        Do not include hidden files
        Do not include system files
        Do not backup older files
        All subfolders included
    SyncToy run took 22:26:12:306.
    Copied 1,533,689,621,243 bytes in 15,146 files in 22:26:12:306.
    Bytes per second 18,987,815.2, files per second 0.2.
    Avoided copying 10,474,936,246 bytes in 39 files that did not require action.
    Saved approximately 00:09:11:666 by not copying all files.
    Warning: 1 failures occured.
    You can retry by selecting "Run" again or select "Preview" to see
    the operations that remain to be performed.

    OK. It took much longer than expected but this was the 1st full drive sync. It should be faster now that the drives are in sync (minus the one file). Correct?

    I hit Sync again to get the one file it missed however, it's taking forever! It just needs to sync one file that's it. Why is it taking so long for such a simple task? I was going to wait for the results before posting this but I'm afraid the site will time out before i get a chance so I will post this as is.

    Tuesday, December 7, 2010 6:33 PM
  • Awesome! Couldent even manage to sync ONE file!

    Notice the start and stop times on the scans. Again this was a follow up scan to sync ONE file that was missed on the original pass as it was locked by another app. Sync time should have been under 5 min total.


    SYNC: 12/07/2010 11:17:31:988: Started scanning directory : E:\
    SYNC: 12/07/2010 11:17:31:991: Started scanning directory : Y:\
    SYNC: 12/08/2010 07:12:37:660: Stopped scanning directory : Y:\
    SYNC: 12/08/2010 07:37:05:415: Stopped scanning directory : E:\
    SYNC: 12/08/2010 07:37:09:120: *** Exception during preview: {0} :
     System.ArgumentOutOfRangeException: Not a valid Win32 FileTime.
    Parameter name: fileTime
       at System.DateTime.FromFileTimeUtc(Int64 fileTime)
       at Microsoft.Synchronization.Files.FileData.FileTimeToDateTime(FILETIME fileTime)
       at Microsoft.Synchronization.Files.FileSyncProvider.FileSyncProviderCallback.OnApplyingChange(String newDirectoryPath, String currentDirectoryPath, _WIN32_FIND_DATAW newFileInfo, _WIN32_FIND_DATAW currentFileInfo, FILESYNC_CHANGE_TYPE dwChangeType)
       at Microsoft.Synchronization.CoreInterop.ISyncSession.Start(CONFLICT_RESOLUTION_POLICY resolutionPolicy, _SYNC_SESSION_STATISTICS& pSyncSessionStatistics)
       at Microsoft.Synchronization.KnowledgeSyncOrchestrator.DoOneWaySyncHelper(SyncIdFormatGroup sourceIdFormats, SyncIdFormatGroup destinationIdFormats, KnowledgeSyncProviderConfiguration destinationConfiguration, SyncCallbacks DestinationCallbacks, ISyncProvider sourceProxy, ISyncProvider destinationProxy, ChangeDataAdapter callbackChangeDataAdapter, SyncDataConverter conflictDataConverter, Int32& changesApplied, Int32& changesFailed)
       at Microsoft.Synchronization.KnowledgeSyncOrchestrator.DoOneWayKnowledgeSync(SyncDataConverter sourceConverter, SyncDataConverter destinationConverter, SyncProvider sourceProvider, SyncProvider destinationProvider, Int32& changesApplied, Int32& changesFailed)
       at Microsoft.Synchronization.KnowledgeSyncOrchestrator.Synchronize()
       at Microsoft.Synchronization.SyncOrchestrator.Synchronize()
       at SyncToy.SyncEngine.CreateAndStartSession(FileSyncProvider destinationProvider, FileSyncProvider sourceProvider)
       at SyncToy.SyncEngine.RunSyncSessionsForChosenSyncType(FileSyncProvider leftProvider, FileSyncProvider rightProvider, SyncMode syncMode)
       at SyncToy.SyncEngine.Run(Boolean previewMode)
       at SyncToyUI.PreviewDialog.PreviewOnePair(SyncEngine she)
    SYNC: 12/08/2010 07:37:09:123: *** Exception during preview: Not a valid Win32 FileTime.
    Parameter name: fileTime


    Ill give it one more go then I think I have to give up on SyncToy as I have yet to receive any responses to my cry's for help!

    Wednesday, December 8, 2010 6:00 PM
  • It ran for over 24 hours and never managed to sync the one file that was missing.

    I guess SyncToy just isn't going to work for me.


    Going to give http://www.superflexible.com/index.htm a try as it has all the features of SyncToy and more including the Remote Service that looks like what I was hoping SyncToy would have.

    Would have been nice to use a Free tool though.

    Thursday, December 9, 2010 5:07 PM
  • Wow! One day with Super Flexible File Synchronizer and not only are my two local 2TB drives in sync but the remote 2TB looks like it will finish up in the next hour or so.

    It's not free, but if you can't get SyncToy to work as I never did. You really should take a look at Super Flexible File Synchronizer. The remote file monitoring service works magic with low bandwidth connections!




    And no I am not getting anything for recommending it. After trying many synchronization options I just found it to be the best and most feature rich of any of the applications I tried and I just want to share my findings.

    Friday, December 10, 2010 12:07 AM
  • SyncToy takes forever to sync if one end is a NAS with lots and lots of files.  It took 3 days just to go through (not copy) the files on NAS to create its database of file to be copied.  I eventually stopped it. I have not tried it with two local or USB disks, but I wouldn't waste my time on it.
    Wednesday, December 15, 2010 12:52 AM