none
OutOfMemoryException in EnumerateUpdates / EnumerateInserts / EnumerateDeletes - Sync Services for Devices RRS feed

  • Question

  • We have Compact Framework application that synchronizes a SQL Server database over WCF.

    We have optimized the synchronization of data from the server in batches so we don't run into any memory issues when synchronizing large data sets but we now have an issue when synchronizing from the client.

    I our use case a customer will be offline for many hours taking pictures and notes which at the end of the day need to be synchronized, it seems that the EnumerateInserts / EnumerateUpdates and EnumerateDeletes methods in the SqlCeClientSyncProvider creates a command that is used to fill a dataset with all the changes from the database, in our scenario this causes a lot of pictures to be  loaded from the DB that causes an OutOfMemoryException

    The line is similar to this: sqlCeCommand2.CommandText = string.Format((IFormatProvider) CultureInfo.InvariantCulture, " SELECT ut.__sysRK  FROM    (select ut0.* from __sysOcsDeletedRows as ut0 where      (ut0.__sysInsertTxCsn IS NULL OR ut0.__sysInsertTxCsn < @LCSN) and      ut0.__sysTName = N'{0}'   )

    As you can see it is doing a SELECT ut0.* and there is no extensibility point to change this SQL before it is executed.

    1. Have I missed something and is it possible to modify the select statement used to load data for updates/inserts/deletes?
    2. If number 1 is not possible what other options do we have?
    3. If we were to sign an NDA would it be possible to have the source for the SyncFramework so we can make the changes ourselves as this is a critical bug and from my investigation there is no workaround?

    Regards

    Russell

    Monday, April 23, 2012 8:50 AM

All replies

  •  afaik, there are no publicly exposed interface to change the sqlceclientsyncprovider selectincrementalxxx statements...and am not sure you're going to get the source code either

    can you compress the images before storing in db? or maybe even do the image sync outside of sync fx.

    Thursday, April 26, 2012 11:11 PM
    Moderator
  • Thanks for your reply.

    I have been banging my head against a wall trying to solve this problem. I have solved the problem when retrieving data from the server by setting a hash for the picture column instead of the image byte array server side which I then check client side by checking the hash and downloading the images one by one. I have also thought about serving up smaller images for the devices as they don't need to be viewing 2048 x 1620 pictures on the device but this problem is slightly different.

    Our customer is collecting asset details for many buildings and the majority of the time they will be in a basement so will be offline. They are taking quite hires pictures of collected assets which means when the sync occurs it tries to load all the images into a dataset that then fails!

    I have even spiked trying to use ildasm to make certain methods in SqlCeClientSyncProvider virtual which I can then override after running ilasm on it, this I may add has had limited success.

    I understand Microsoft have recently release the source for the 4.0 sync framework so was looking for any solutions to the problem.

    Out of curiosity would you know if raising a support request for this would have much success?

    Regards

    Russell

    Thursday, April 26, 2012 11:26 PM
  • i cant speak for MS (i dont work for MS) but i doubt you'll get much success considering that the SqlCeClientSyncProvider has not had any improvements over the last couple of years and if youre not working with a Windows CE device, it's highly recommended to use the much later SqlCeSyncProvider.

    Allow me to clarify some confusion about MS releasing code for Sync Fx 4.0.

    The supposedly v4 is actually now the Sync Fx Toolkit. the toolkit was primarily designed for clients that cant install the full Sync Fx SDK/Runtime(Silverlight, WP7, Android, iPhone, HTML5). the server side component of the toolkit is still running Sync Fx 2.1 but the client side has custom codes that suits the client platform. Sync Fx Toolkit source is NOT the Sync Fx code.

    Thursday, April 26, 2012 11:48 PM
    Moderator