Sign in
Microsoft.com
 
Microsoft
 
 
 

locked
Large Sync Using SyncToy 2.1 RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    1
    Sign in to vote

    I'm trying to use SyncToy 2.1 to Sync 2 very large folders. The left hand folder has about 3.5 million objects (files or folders). SyncTool runs the preview for a number of hours and then fails with the error - "SyncToy cannot access its internal metadata file in one of the folders being synchronized". I've tried this on a number of machines and the results are the same.
    I've noticed that there are 2 "*.dat" files created by SyncToy under "Local Settings\Application Data\SynceToy\2.0", which I assume hold the metadata for each half of the pair. During a failing run, these grow as you might expect. Some time after the first file reaches a Gigabyte or so it get's truncated back to 92KB! However the application continues to run for some considerable time. I assume it's working on the remote data because at dome point the application displays the error message shown above and both DAT files are truncated to 92KB.
    Am I breaking a documented (or undocumented) limit in SyncToy?

    Monday, November 16, 2009 4:09 PM

Answers

  • Question
    Sign in to vote
    1
    Sign in to vote

    Hi Kjen,
         I did repro your problem, as folder pairs with 3.5 million objects really is extreme case. 3.5 million objects may have hit the limitation.

    For  now, my suggestion for you is split those into more folde pairs, at least I have tested with 840 thousands files could be synced successfully.


    Thanks,
    Ping

    This posting is provided "AS IS" with no warranties, and confers no rights.
    • Proposed as answer by Danish Wang Friday, November 27, 2009 7:58 AM
    • Marked as answer by Yunwen Bai Wednesday, December 2, 2009 12:57 AM
    Friday, November 27, 2009 5:59 AM

All replies

  • Question
    Sign in to vote
    1
    Sign in to vote

    Hi Kjen,
         I did repro your problem, as folder pairs with 3.5 million objects really is extreme case. 3.5 million objects may have hit the limitation.

    For  now, my suggestion for you is split those into more folde pairs, at least I have tested with 840 thousands files could be synced successfully.


    Thanks,
    Ping

    This posting is provided "AS IS" with no warranties, and confers no rights.
    • Proposed as answer by Danish Wang Friday, November 27, 2009 7:58 AM
    • Marked as answer by Yunwen Bai Wednesday, December 2, 2009 12:57 AM
    Friday, November 27, 2009 5:59 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    Hi Ping Lu

    Thanks for the reply. When you say that I "may have hit the limitation", what is the limitation?

    The problem that I have with splitting the job into more folders pairs is that the top level folder contains a few thousand folders. So I would need to add a new pair for each folder, which is not practical. Also every time a new folder is added to the top level folder, I would need to detect this and add another folder pair. Again not practical.

    I guess on this occasion, SyncToy is not the right tool for the job. However it looks like a really useful tool, so keep up the good work.
    Tuesday, December 1, 2009 4:15 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    Hi kjen,
         Sorry if the limitation words confused you, as that was my original guess, in fact it should not be the reason, as the metadata has
    For your problem, another better solution is you can create a few folder pairs which actually have the same left and right folder(such as it is the root folder), then you could exclud some subfolders to make the folder pairs combination cover all of your sub folders.
    For these new created folders, it could be included in the one which selected the root folder.

    Hope this could be helpful!


    Thanks,
    Ping
    This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, December 16, 2009 3:46 AM