locked
OutOfMemoryException on SyncAgent.Synchronize() - SSfADO.Net - Mobile Devices RRS feed

  • Question

  •  

    I've got a windows mobile 6 standard device (samsung blackjack i607) that I use for GPS track logging.  Synchronization of the data has been great for a while, until recently when I went on a long trip and didn't synchronize until I got back from the trip and tried synchronizing again.

     

    There are two tables that are being synchronized (both upload only)

    Table A: 0 Records

    Table B: ~40,000 Records

     

    Here's a trace of the SqlCeClientSyncProvider's SyncProgress event before the exception is thrown:

    152243 | Stage: GettingInserts for Table A (ChangesPending: 0)

    152603 | Stage: GettingInserts for Table B (ChangesPending: 39332)

    152604 | Stage: GettingUpdates for Table A (ChangesPending: 0)

    152746 | Exception: OutOfMemoryException

     

    Before the SyncProgress event gets a chance to fire with a stage of GettingUpdates for Table B, I get a OutOfMemoryException.  It seems these events are being fired after the sync provider fetches the list of changes from the database, so my device can't allocate enough memory to fetch the list of updates. 

     

    I've looked around and it seems that synchronizing in batches isn't supported in the device version of sync services. 

     

    I've also tried cleaning up unused resources in the application code before synchronizing and tried synchronizing after a clean soft reset.

     

    Is my only option to create a synchronization application for my PC, copy the database over there and have it synchronize the data? Or is there anything I can do (or something planned in CTP2/RTM) to get around this problem?

     

    Thanks!

    • Moved by Max Wang_1983 Friday, April 22, 2011 4:46 PM forum consolidation (From:SyncFx - Microsoft Sync Framework Database Providers [ReadOnly])
    Wednesday, September 10, 2008 7:56 PM

All replies

  •  

    Additional thoughts:

     

    I've added a line to GC.GetTotalMemory(false)  between each of these events and here's how it looks:

     

    Stage: GettingInserts for Table A (ChangesPending: 0)

    00695880 bytes in use

    Stage: GettingInserts for Table B (ChangesPending: 39332)

    13536200 bytes in use (Calling GC.Collect)

    12418592 bytes in use (after collection)

    Stage: GettingUpdates for Table A (ChangesPending: 0)

    12440892 bytes in use (Calling GC.Collect)

    12421928 bytes in use (after collection)

    Exception: OutOfMemoryException

     

    I don't like calling collect manually, and it didn't seem to do any good anyways.

     

     

    Thanks!

    Wednesday, September 10, 2008 9:14 PM
  • Hi Jonathan

     

    Synchronising 40,000 items to a mobile device is never going to be easy.

     

    Below are some ways of reducing your memory restrictions with out adding lots of code

     

    1: Use GC.GetTotalMemory(True) instead of false. This should improve the collection of data. Also you can call the GC.Collect method as many times as you like within your code.

     

    2: Work out approx how much ram each row in your database takes.

     

    To do this: Create a text file and type in what each cell contains in one row, plus the LastEditDate and GUID Cells etc

    i.e.  john, galvin, backupearth, Microsoft, support, etc, etc, 01/01/01 01:01:01, 0000-0000-0000-00042c

    Now save the text file and check its “REAL” file size.

    Now multiple this file size X number of rows to get the amount of data that needs to be downloaded. I.E

    FileSize = 1Kb X 40000 = 40Mb that’s allot of data to synchronise in one go to mobile device

    But this is only approx value and can be “out of touch” with the real value by a mile. But it will give you an idea of what your synchronisation is facing.

     

    3: Use a Block Transfer to transfer only X amount of rows at a time.

     

    This requires a lot of code, please see this link for an example.

    http://www.syncguru.com/projects/SyncServicesDemoBatching.aspx

     

    I can do an example in VB if you like, just leave a message here.

     

    4: Database Break Down

     

    Allot of software companies place all the data in one big table, this is fine when you are using SQL Server, but when it comes to SQL CE Compact, things are a little different. SQL CE Compact is not very good at resizing itself for new data, hence the slow insert rates compared to its big brother SQL Server, but fair play to Microsoft they are working on ways to improve this.

     

    Break down your database into as many tables as necessary. This will not effect the performances of database and application, Databases effect the performance not tables.

     

    Use a common column across the tables, a Identifier Column i.e. UserName that is common to all tables. DO NOT use the ID or Primary Key Column, The GUID or LastEditColumn as SQL Tracking will changes these.

     

    When Synchronising only Sync the Data required for the Username.

    To Do This Change the SelectIncrementalInsertsCommand, SelectIncrementalUpdatesCommand for each table under the SyncCache.Designer.vb . Only change these commands, do not change the Delete commands etc unless you understand SQL programming.

     

    In the SelectIncrementalInsertsCommand, and SelectIncrementalUpdatesCommand command text you will find “…WHERE ID = @ID” add the following “ AND [UserName] = ‘***WhatEverUsername***’

     

    --> WHERE ID = @ID AND [UserName] = 'jgalvin'...

     

    Now your appication will only sync with data that is for your mobile decvice (Username).

     

     

    5:  By Breaking down the Database into Tables you can track the Sync Rate using SyncProvider_SyncProgress. Once again you will find this Partial Class under SyncCache.Designer.vb. Add the Event SyncProgress. Use the e.tableprocess to give you the information on which table is been sync and how many inserts etc have been complete.

     

    Also by having Tables the Sync will Download and Apply changes per table, this will reduce RAM usage.

     

    Finally I hope the above was of some help, let me know how you got on, if you are anyone needs a good example, let me know and I will do one up for you and place it here.

     

    Kind Regards

     

    John Galvin

     

     

     

     

      

     

    Execute

     

     

     

     

    Sunday, September 28, 2008 12:35 PM