locked
Large data sync gets stuck RRS feed

  • Question

  • Hi,

    We've made a solution for synchronization of 2 MS SQL databases (one enterprise one express edition).
    And tried to sync small tables, and faced no problems.
    Now we are trying to sync with a production database. And a lot of data.
    This gives a lot of problems. First with memory, because sync framework gets all changes in one dataset
    and the client computer is running out of memory.
    So we've changed the code: 1 table sync at the time.(1 for a sync group)
    But still if we are at about the end of our synchronisation, it get's stuck and gives exeptions like
    a servere error occured, connection with the local database lost.
    Or no exceptions at all, but it just doesn't insert any records anymore.

    So I was wondering ar there any people who have the same problem. And where able to fix it?
    • Edited by mn1aC Friday, April 17, 2009 10:44 AM
    • Moved by Max Wang_1983 Friday, April 22, 2011 12:12 AM forum consolidation (From:SyncFx - Microsoft Sync Framework Database Providers [ReadOnly])
    Thursday, April 16, 2009 1:17 PM

Answers

  • Hi mn1ac,
    You can use batching to alleviate the problem of downloading large data in one shot. Please refer to the MSDN link to learn more about how to download changes in batches.
    http://msdn.microsoft.com/en-us/library/bb902828.aspx

    Please note that for Server-Client sync, batching is only supported for the server. i.e there is no support to enable the client to upload changes in batches.
    Maheshwar Jayaraman - WCF -- http://blogs.msdn.com/dotnetremoting
    Wednesday, August 12, 2009 11:04 PM
    Moderator

All replies

  • I'm currently working on the same project as mn1aC.

    When we try to sync small tables, everything goes according to plan. When you try to sync the same table for a 2nd time (and no data has been changed on either of the two databases) nothing is synchronized.

    When we try to initially sync a larger table (total size about 60Mb or larger) the data is inserted in the client database like it should. When we try a 2nd sync (with no data changed at all) the client tries to insert all the data it received previously from the server, again on the server. This causes a Client Insert - Server Insert conflict.

    The conflicts only occur when synchronizing a large table.

    Does anybody now what the cause of this problem could be?

    Thx in advance.
    Friday, April 17, 2009 10:12 AM
  • I assume you are using OCS V2 DbSyncProvider in a peer to peer fashion. For the small table and the large table you mentioned, do they have the same schema or different one? If they have different schemas, could you please check if the SelectIncrementalChangesCommand of DBSyncAdapter works for your large table or not?

    Thanks,


    Ann Tang
    Friday, April 17, 2009 8:42 PM
  • Hi Dann,

    Are you able to track down the issue? I am facing the same problem with large tables.



    Friday, August 7, 2009 2:28 PM
  • Hi mn1ac,
    You can use batching to alleviate the problem of downloading large data in one shot. Please refer to the MSDN link to learn more about how to download changes in batches.
    http://msdn.microsoft.com/en-us/library/bb902828.aspx

    Please note that for Server-Client sync, batching is only supported for the server. i.e there is no support to enable the client to upload changes in batches.
    Maheshwar Jayaraman - WCF -- http://blogs.msdn.com/dotnetremoting
    Wednesday, August 12, 2009 11:04 PM
    Moderator
  • Hi daan,
    What are the SyncDirections setting for each of those sync. The behavior you mention can only happen if you do the two syncs in an unsupported order. Like 1st sync with DownloadOnly and then the second Sync with UploadOnly/Bidirectional.

    Maheshwar Jayaraman - WCF -- http://blogs.msdn.com/dotnetremoting
    Wednesday, August 12, 2009 11:12 PM
    Moderator