locked
Bidirectional Sync and Batching with SQL Server 2008 and Change Tracking RRS feed

  • Question

  • Is it possible to modify the sync process so that all client changes that are part of a sync don't end up with the same version number on the server?  I want to implement batching so that even though one user inserts 500 records in a sync another user will still download the 500 records in smaller batches.  Can you override how changes are applied to SQL Server 2008 so that each of the 500 records generates it's own version number rather than one all together?
    • Moved by Max Wang_1983 Thursday, April 21, 2011 11:21 PM forum consolidation (From:SyncFx - Microsoft Sync Framework Database Providers [ReadOnly])
    Wednesday, July 29, 2009 7:29 PM

Answers

  • Hi Stewart,
    We dont have any samples published that demonstrates how to do server side batching with change tracking enabled. I dont want to say its not supported but it will take some creative work for you to break all changes with same version. You are right in that with change tracking all inserts/updates done in a transaction share the same version information and hence there is no way to batch them based on versions. This is why the CE client provider doesnt support batching. CE change tracking uses the same version number for all changes in a given transaction. THe only other way for you to achieve this would be to do what Leo suggested. Dont use Change tracking for the big table. Use two timestamp columns to track changes manually and use those columns to choose your sent anchor from the selectNewAnchor comamnds.

    One way you can achieve this is after the client uploads, just dummy touch the rows on the server with a different transactions so you end up with different versions.
    Maheshwar Jayaraman - WCF -- http://blogs.msdn.com/dotnetremoting
    Wednesday, August 12, 2009 11:18 PM
    Moderator

All replies


  • When changes are uploaded to the server, all changes in one sync group are applied into one SQL transaction.
    So what you try is to do dummy update on these rows after they got applied on the server (such as from the Server side ChangeApplied event handler).

    Thanks.
    Leo Zhou ------ This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, July 29, 2009 8:13 PM
    Answerer
  • Thanks L Zhou.  How do you do that and make sure the same user does not download the 'dummy' updates?  Do you have an example?
    Wednesday, July 29, 2009 8:17 PM
  • Also, I thought this might help you understand my situation:

    TABLE:

    CREATE TABLE [dbo].[DataBlob]
    (
    	DataBlobId uniqueidentifier NOT NULL,
    	DataObjectId uniqueidentifier NOT NULL,
    	DataBlobSort int NOT NULL,
    	DataBlobType nvarchar(50) NOT NULL,
    	DataBlobData image NULL,
    	DataBlobCreated datetime NOT NULL,
    	DataBlobLastModified datetime NOT NULL
    )
    
    When a user uploads a document to my application the document is chunked (aka broken down) into multiple records (each capped at 250K of image field) and inserted into this table.  This table only has inserts and deletes. 

    When the user who uploaded documents syncs with the server I would like to make sure the new records inserted into the server database each have thier own change tracking version.  That way when the next user syncs batching will take effect.

    Normally I would just use CTP2 but I have to release at the end of next month.

    Thanks!!
    Stewart
    Wednesday, July 29, 2009 8:30 PM
  • You have 

    DataBlobCreated datetime NOT NULL,
    DataBlobLastModified datetime NOT NULL

    to track when the create or update happens, right?

    You may also need to have 2 more columns to track who does the create and update.  As long as the dummy done by the same the client, you should be able to prevent the loop back by adding LastUpdater <> 'me' to your select incremental insert/update query.

    Thanks.


    Leo Zhou ------ This posting is provided "AS IS" with no warranties, and confers no rights.
    Wednesday, July 29, 2009 8:35 PM
    Answerer
  • Those two fields are not used in the sync.  Sorry I wasn't clear:  I am using SQL Server Native Change Tracking.  Are you saying I should implement my own custom sync logic?  I was trying to avoid that.  Just wanted to see if there was a way to trick the inserts into the server side database so they weren't applied in one transaction and all ended up with different SYS_CHANGE_CREATION_VERSION values in the underlying SQL Server Change Tracking Changes table. 

    My experience with the sync framework is not the deepest right now.  Is there a safe way to override the server side ApplyChanges and manually force each insert to happen in a separate transaction?  If I do this will it interfere with any default downstream logic for getting server side changes for the client that is sync'ing?

    OR as you stated in your first response, can I just run an update in a server side ChangeApplied event handler that updates the records inserted?  If this works, I was just hoping you or someone could point me to a code sample of what is the best way to do that.
    Wednesday, July 29, 2009 8:46 PM
  • You are correct - you do not need to create those 2 columns.  Instead, SQL Server Change Tracking should do it for you once you specify the DML as
    ";WITH CHANGE_TRACKING_CONTEXT ( ..some binary number to represent this client ...)".

    In this case, your app can download updated rows from the server which made by other clients (or server iteslf), when the select incremental insert/update/delete TSQL has the below condition in the where-clause.
    CT.SYS_CHANGE_CONTEXT IS NULL OR CT.SYS_CHANGE_CONTEXT <> @sync_client_id_binary.

    In case you worry the TSQL command syntax, please use SyncAdapter object to help you to build the upload/download commands for your.

    For more info, you can refer to, "How to: Use SQL Server Change Tracking" - http://msdn.microsoft.com/en-us/library/cc305322.aspx.

    Thanks.


    Leo Zhou ------ This posting is provided "AS IS" with no warranties, and confers no rights.
    Thursday, July 30, 2009 12:50 AM
    Answerer
  • Thanks, but this isn't exactly what I was talking about.  I am more concerned with modifying the way inserts are applied on the server side. Here is the problem I am facing right now:

    1. Client AAA takes a week and makes changes to thier local database by uploading several files to the DomainBlob table (say 30 records get added each ~ 200K's worth of binary data in the DataBlobData image field).
    2. Client AAA syncs with the server and all 30 records get pushed to the server in one batch because there is no batching on the client side.
    3. All 30 newly inserted records in the server side database end up with the same SYS_CHANGE_CREATION_VERSION.
    4. Client BBB syncs.
    5. All 30 records are downloaded to Client BBB in one batch because the default filter for SelectIncrementalInserts contains this where clause below with the condition in the fourth line (remember in step 3 all records ended up with the same value in this field).

    JOIN CHANGETABLE(CHANGES dbo.DataBlob, @sync_last_received_anchor) CT 
    ON CT.[DataBlobId] = dbo.DataBlob.[DataBlobId] 
    WHERE (CT.SYS_CHANGE_OPERATION = 'I' 
    AND CT.SYS_CHANGE_CREATION_VERSION <= @sync_new_received_anchor 
    AND (CT.SYS_CHANGE_CONTEXT IS NULL 
    OR CT.SYS_CHANGE_CONTEXT <> @sync_client_id_binary)
    


    So given this I was looking for a way to change the processing step 3.  Is it possible to change the way the records are inserted in the server database so that each record gets a different (incrementing) value for the SYS_CHANGE_CREATION_VERSION?  If I can do this then the batching I have setup will take effect and step 5 will pull down 6 batches of 5 records instead of one batch of 30.
    • Edited by Stewart Armbrecht Thursday, July 30, 2009 1:15 AM formatting wasn't working
    Thursday, July 30, 2009 1:13 AM
  • Hi Stewart,
    We dont have any samples published that demonstrates how to do server side batching with change tracking enabled. I dont want to say its not supported but it will take some creative work for you to break all changes with same version. You are right in that with change tracking all inserts/updates done in a transaction share the same version information and hence there is no way to batch them based on versions. This is why the CE client provider doesnt support batching. CE change tracking uses the same version number for all changes in a given transaction. THe only other way for you to achieve this would be to do what Leo suggested. Dont use Change tracking for the big table. Use two timestamp columns to track changes manually and use those columns to choose your sent anchor from the selectNewAnchor comamnds.

    One way you can achieve this is after the client uploads, just dummy touch the rows on the server with a different transactions so you end up with different versions.
    Maheshwar Jayaraman - WCF -- http://blogs.msdn.com/dotnetremoting
    Wednesday, August 12, 2009 11:18 PM
    Moderator
  • I was able to do this by modifying the server side procs to use row_number().  I batched off the row_number() value and not the anchor value.  I even posted some sample code on this forum explaining how it works.
    Friday, September 18, 2009 3:05 PM