none
SyncFx Devices need to limit records that are uploaded to the server RRS feed

  • Question

  • We are running a SyncFx 1.0 for devices application on WM6 over WCF to a SQL2005 database. I have 1 major table that is upload only and when uploading 2000+ records memory on the device gets close to 24MB and the application shuts down.

    I really need a way to batch upload these guys in 500 row chunks.

    My understanding is that batching is not supported on SyncFx 1.0 for devices. How can I accomplish this??

    Wednesday, December 8, 2010 7:40 PM

All replies

  • Batching is not supported for Upload with the hub-spoke sync provider.

    you can try to use smaller transactions ( ie. less data change perf trans ) and more frequent sync to get the smaller syncs.

    with 24 MB data, I suppoe it should work. is there any error or as you said, it just crashed ? can you check in the app to see what caused this ?

    thanks

    Yunwen


    This posting is provided "AS IS" with no warranties, and confers no rights.
    Friday, December 10, 2010 3:56 PM
    Moderator
  • Well this is a limitation that we can not deal with. In a disconnected environment we cannot "force" users to sync often. That somewhat defeats the purpose of allowing them to work disconnected.

    What about SyncFx 4.0? The docs claim that it can work on any platform including mobile and moves all sync logic to the server. Looks like batching is supported on that version.

    My only other option seems to be that I would have to remove all uploads from the SyncFx and handle them myself so that I could implement the batching required for our application. I really do not want to do this because I think Sync works very well for download and bidirectional, at least when not dealing with a lot of data.  

    Friday, December 10, 2010 5:15 PM
  • have you tried playing around and setting the SetTableSentAnchor yourself? i remember doing a similar implementation for SQL Express where we intercept the changes in ChangesSelected and trim the dataset based on a predefined batch size then manually setting the last sent anchor to the max value in the trimmed dataset...a bit ugly though...
    Monday, December 13, 2010 3:53 PM
    Moderator
  • Thanks for the reply. How exactly would that work? It seems as if I removed records from the dataset based on a batch size (lets say 500) and then modify the SentAnchor to the max of the 500 then the additional records (batches) would not get picked up until the next Synchronize call.

    This would not work because the record set would never catch up.

    Monday, December 13, 2010 6:47 PM
  • in our case, i have to persist the lastsentanchor to make sure we could restart the sync where we left off just in case something goes wrong. if i remember it right, we trim the dataset in the GetChanges or ChangesSelected. it was for SQL Express and we have to enable batching as well so that the GetChanges get;s called repeatedly. the idea is to let it get Sync Fx to select the changes from the last sent anchor onwards then trimming anything above the batch size specified. it was to control the data transfer over the wire. i just read your original post and it has something to do with memory.

    have you checked if its crashing before uploading the data or after it has uploaded?

    if its after uploading, i'm guessing its because of the extra dataset in the SyncContext. if i remember it right, the SyncContext returned after applying the changes on the server would have a separate dataset for the data originally uploaded, another dataset for those that has a conflict and there's another extra dataset (SyncProgress i think).

    Monday, December 13, 2010 11:39 PM
    Moderator
  • Thanks JuneT, I think your solution may work for us.

    A  few questions regarding your implementation.

    1. You said that you had to enable batching to get GetChanges to fire repeatedly. So to enable batching do I simple set the batchsize on the server and that will cause the ClientSyncProvider's GetChanges to fire repeatedly? The tables that I am trying to "batch" are upload only.

    2. If the ClientSyncProvider GetChanges fires repeatedly, what causes it to stop?

    3. Finally, what exactly would the LastSentAnchor be set to? Im not positive how SyncFx determines what changes to send up from the client but I thought that it compares the LastSentAnchor with the __sysInsertTxBsn hidden column to determine which records were newly inserted and need to be sent to the server.

    Tuesday, January 4, 2011 8:25 PM