locked
Is there any limit for size of data that can be sync RRS feed

  • Question

  •  

    I am trying to synch one table fromSQL server 2005 to SQL CE. The table got around 10 million records. When I start the sync process it says out of memory after reaching close to 1 million records and it stops there. I am not sure whether any limit is there about the size of data to be sync?

     

    Regards

    Pratip

     

     

    • Moved by Max Wang_1983 Thursday, April 21, 2011 10:05 PM forum consolidation (From:SyncFx - Technical Discussion [ReadOnly])
    Wednesday, May 14, 2008 4:56 AM

Answers

  • Hi Pratip,

     

    The Sync Service will sync all changes in one dataset by default. Since you tried to sync 10 million records from SQL server to a SQL compact database, I think that your out of memory exception was caused by the huge dataset which cannot be fit into your memory. Since your scenarios is a download sync, there is a solution to enable batching. Please take a look at the link: http://msdn.microsoft.com/en-us/library/bb902828(SQL.100).aspx. It has a sample for how to configure batch size for the download sync.

     

    Thanks,

    Dong

     

    Wednesday, June 11, 2008 5:23 PM
    Moderator

All replies

  • First, SQL CE has a limitation of 4GB size.

    Second, if you are using one or few transactions for the million records synchonization then the SQL CE may hit other limitation such as temp db size limitation.

    Wednesday, May 14, 2008 6:45 AM
    Answerer
  • Some other information also would be helpful.

    How do you use the Sync Framework in your sync project? 

    - Are you using Sync Services for ADO.Net to Sync?

    - Are you writing a Sync solution with your own provider and your own meta data management?

    - Are you writing a Sync solution with your own provider and Sync Framework Metadata store?

     

    Wednesday, May 14, 2008 2:30 PM
    Answerer
  • Thanks for the input.

    I am aware about 4GB limit. My .sdf file size is only 300 mb. Only one table contains 10 million records.I am using the Sync Services for ADO.net for sync.

     

    I am just worried as sync start in a batch or group. So in case of table having 10 million record it will first gets all the changes in memory then start updating the local db. Is there any possibility that it can store the changes in hardisk in tmp location and after that start updating.

     

    Even I tried with a table having 2 million records. It starts synching and when it reaches the half million records it says out of memory.

     

     

     

    Thursday, May 22, 2008 5:52 AM
  • Hi Pratip,

     

    The Sync Service will sync all changes in one dataset by default. Since you tried to sync 10 million records from SQL server to a SQL compact database, I think that your out of memory exception was caused by the huge dataset which cannot be fit into your memory. Since your scenarios is a download sync, there is a solution to enable batching. Please take a look at the link: http://msdn.microsoft.com/en-us/library/bb902828(SQL.100).aspx. It has a sample for how to configure batch size for the download sync.

     

    Thanks,

    Dong

     

    Wednesday, June 11, 2008 5:23 PM
    Moderator