locked
batching RRS feed

  • General discussion

  • I previously posted about my hastles on trying to get a filtered subset of data to be batched for incremental biderectional updates.
    Ive had a mini breakthrough in batching that i thought i should share here.
    Ive found that the reason why batching as implimented on MSDN behaves erratically in my tests on large data tables is because the 2 tracking fields 'lasteditdate timestamp' and 'creationdate bigint' were added to an existing table which was already filled with data. This means that while lasteditdate is unique over the 350000 records, the creationdate value is the same for all the records i.e. @@dbts+1(this parameter is the same as the last active timestamp in the db).
    So when the framework uses the usp_GetNewBatchAnchor procedure,if the @synch_last_received_anchor starts at zero, it works out that there should be x amount of batches in @batch_count but each batched query brings back nothing because the where clause in the incremental insert command 'select * from table where creationdate>@synch_last_received and creationdate<@synch_new_received_anchor' is always too small to reach the value of the creationdate field.It only reaches a match on the very last batched query and when it does, the amount of data it pulls out is far bigger than batchsize specified, resulting in memory errors on the mobile device.

    So the solution is : when  the creationdate field is added to your EXSITING table(either by visual studio or manually), its best to update its value on each row to the value of lasteditdate. e.g.
    update account
    set creationdate=lasteditdate
    Maybe this is an obvious solution , but when using the example straight out of msdn this is not really made clear.
    So far it seems to be working. Still not happy with the speed, but maybe i need to play around with it a bit more.

    Im using SyncFramework CTP2,vs2008, SQL 2005, windows mobile 5, SQL CE 3.5 sp1
    Ive also implimented SyncCom which enables GZip compression over WCF on mobile devices
    • Moved by Hengzhe Li Friday, April 22, 2011 2:21 AM (From:SyncFx - Microsoft Sync Framework Database Providers [ReadOnly])
    Wednesday, July 29, 2009 4:03 PM