Can we sync two database with different schemas and different tables using Sync Framework
-
18 sierpnia 2011 08:24
Hi
I have two database and there are certain tables in them which i need to synchronise using Sync framework with the help of WCF Data Service.
At the source DB side tables have different column names as compared to the destination DB side.
e.g. Source DB has Order table with Order_ID as the column which i need to synchronise with Destination DB having OrderData table with Order_Num as the column. I want to synchronise these columns and many other columns like this.
Can we achieve that using Sync Framework? Also i want to use WCF data service to achieve this.
Any help in this.
Wszystkie odpowiedzi
-
18 sierpnia 2011 08:35Moderator
Sync Framework doesnt integrate with WCF Data Services.
yes, you can sync different table names or column names. even different PKs or not use the table PKs at all.
see the following blog posts:
Part 1 – Upload Synchronization where the Client and Server Primary Keys are different
Part 2 – Bidirectional Synchronization where the Client and Server Primary Keys are different
Part 3 – Synchronizing tables where Client and Server Primary Keys are different
Synching tables with different table names
-
18 sierpnia 2011 10:26
Thanks June for sharing your blogs but in your blog "Synching tables with different table names" you have given the soruce code but i am not able to understand whether this applies to the Server side or Client side.
Please NOTE i created the project as guided on this link http://msdn.microsoft.com/en-us/library/ff928700(v=SQL.110).aspx.
Could you please help me in understanding where exactly the piece of code (given in ur blog) i need to put either in Server side project or Client side project.
-
18 sierpnia 2011 10:33Moderatoryou assign the table GlobalName during provisioning.
-
18 sierpnia 2011 11:08
I did the same thingbut now what has happened is that initially SourceDB had Source table and DestinationDb had Destination table (empty table). Now after synchronisation DestinationDb had Destination table (with the synchronised data -- Desired result) but it created one more table Source with the synchronised data -- Not required.
How to remove this extra table creation. I am sharing my code below.
Server Side
SqlConnection serverConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncSource;User Id=asapteam;Password=welcome"); // define a new scope named ProductsScope DbSyncScopeDescription scopeDesc = new DbSyncScopeDescription("SourceScope"); // get the description of the Products table from SyncSource database DbSyncTableDescription tableDesc = SqlSyncDescriptionBuilder.GetDescriptionForTable("Source", serverConn); // giving a global name to this table tableDesc.GlobalName = "GlobalSourceTable"; // add the table description to the sync scope definition scopeDesc.Tables.Add(tableDesc); // create a server scope provisioning object based on the SourceScope SqlSyncScopeProvisioning serverProvision = new SqlSyncScopeProvisioning(serverConn, scopeDesc); // skipping the creation of table since table already exists on server serverProvision.SetCreateTableDefault(DbSyncCreationOption.Skip); // start the provisioning process serverProvision.Apply();Client Side
SqlConnection clientConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncDestination;User Id=asapteam;Password=welcome"); SqlConnection serverConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncSource;User Id=asapteam;Password=welcome"); // get the description of SourceScope from the SyncSource server database DbSyncScopeDescription scopeDesc = SqlSyncDescriptionBuilder.GetDescriptionForScope("SourceScope", serverConn); // get the description of the Products table from SyncSource database DbSyncTableDescription tableDesc = SqlSyncDescriptionBuilder.GetDescriptionForTable("Destination", clientConn); // giving a global name to this table tableDesc.GlobalName = "GlobalSourceTable"; // add the table description to the sync scope definition scopeDesc.Tables.Add(tableDesc); // create client provisioning object based on the SourceScope SqlSyncScopeProvisioning clientProvision = new SqlSyncScopeProvisioning(clientConn, scopeDesc); // starts the provisioning process clientProvision.Apply();
Synchronised CodeSqlConnection clientConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncDestination;User Id=asapteam;Password=welcome"); SqlConnection serverConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncSource;User Id=asapteam;Password=welcome"); // create the sync orhcestrator SyncOrchestrator syncOrchestrator = new SyncOrchestrator(); // set local provider of orchestrator to a sync provider associated with the // SourceScope in the SyncDestination client database syncOrchestrator.LocalProvider = new SqlSyncProvider("SourceScope", clientConn); // set the remote provider of orchestrator to a server sync provider associated with // the SourceScope in the SyncSource server database syncOrchestrator.RemoteProvider = new SqlSyncProvider("SourceScope", serverConn); // set the direction of sync session to Upload and Download syncOrchestrator.Direction = SyncDirectionOrder.Download; // execute the synchronization process SyncOperationStatistics syncStats = syncOrchestrator.Synchronize(); // print statistics Console.WriteLine("Start Time: " + syncStats.SyncStartTime); Console.WriteLine("Total Changes Uploaded: " + syncStats.UploadChangesTotal); Console.WriteLine("Total Changes Downloaded: " + syncStats.DownloadChangesTotal); Console.WriteLine("Complete Time: " + syncStats.SyncEndTime); Console.WriteLine(String.Empty);See if you can suggest something.
-
18 sierpnia 2011 11:23Moderator
this line is grabbing the scope definition on the server:
// get the description of SourceScope from the SyncSource server database
DbSyncScopeDescription scopeDesc = SqlSyncDescriptionBuilder.GetDescriptionForScope("SourceScope", serverConn);
the scope definition on the server has the "Source" table and in the client you added the "Destination" table to the scope. your scope definition now contains both "Source" and "Destination" table by the time you create the scope.on the client, replace
DbSyncScopeDescription scopeDesc = SqlSyncDescriptionBuilder.GetDescriptionForScope("SourceScope", serverConn);
with
DbSyncScopeDescription scopeDesc = new DbSyncScopeDescription("SourceScope");
- Zmodyfikowany przez JuneTMVP, Moderator 19 sierpnia 2011 05:53
-
18 sierpnia 2011 11:58
Thnaks June it worked as expected, you are gr8 :)
Now one more thing i want to try is that Suppose Source Table has Order_ID and Order_Desc as there columns and Destination table has Order_Num, Order_Detail, Order_Qty as columns.
Now i want to sync only Order_ID --> Order_Num and Order_Desc-->Order_Detail columns only. Can we achieve this. If yes what all changes i need to make in my program.
-
18 sierpnia 2011 14:59Moderator
subscribe to the ChangesSelected event, loop thru the change dataset and rename the column.
-
19 sierpnia 2011 13:54Moderatorfound some time to blog, so here you go: Part 4 – Synchronizing Tables With Different Table Names and Column Names
-
23 sierpnia 2011 07:03
I tried your blog but it's not able to synchronise the data andsyncStats.DownloadChangesTotal = 0
is the answer i.e. no rows is getting synchronised.
Server
SqlConnection serverConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncSource;User Id=asapteam;Password=welcome"); string scopeName = "CommonSchemaScope"; SqlSyncScopeProvisioning serverProvision = new SqlSyncScopeProvisioning(serverConn); if (!serverProvision.ScopeExists(scopeName)) { DbSyncScopeDescription serverScopeDesc = new DbSyncScopeDescription(scopeName); // add the server table DbSyncTableDescription serverTableDesc = SqlSyncDescriptionBuilder.GetDescriptionForTable("Source", serverConn); // our server and client tables has different names, so let’s setup a common name serverTableDesc.GlobalName = "GlobalCommonTable"; serverScopeDesc.Tables.Add(serverTableDesc); serverProvision.PopulateFromScopeDescription(serverScopeDesc); //apply the scope definition serverProvision.Apply(); }Client
SqlConnection clientConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncDestination;User Id=asapteam;Password=welcome"); string scopeName = "CommonSchemaScope"; SqlSyncScopeProvisioning clientProvision = new SqlSyncScopeProvisioning(clientConn); if (!clientProvision.ScopeExists(scopeName)) { DbSyncScopeDescription clientScopeDesc = new DbSyncScopeDescription(scopeName); // add the client table DbSyncTableDescription clientTableDesc = SqlSyncDescriptionBuilder.GetDescriptionForTable("Destination", clientConn); clientTableDesc.GlobalName = "GlobalCommonTable"; clientScopeDesc.Tables.Add(clientTableDesc); // remove the extra column from Destination table which Source table doesnt have it clientScopeDesc.Tables["GlobalCommonTable"].Columns.Remove(clientScopeDesc.Tables["GlobalCommonTable"].Columns["Temp1"]); clientScopeDesc.Tables["GlobalCommonTable"].Columns.Remove(clientScopeDesc.Tables["GlobalCommonTable"].Columns["Temp2"]); clientProvision.PopulateFromScopeDescription(clientScopeDesc); //apply the scope definition clientProvision.Apply(); }Synchronising
SqlConnection clientConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncDestination;User Id=asapteam;Password=welcome"); SqlConnection serverConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncSource;User Id=asapteam;Password=welcome"); string scopeName = "CommonSchemaScope"; //setup providers SqlSyncProvider localProvider = new SqlSyncProvider(scopeName, clientConn); SqlSyncProvider remoteProvider = new SqlSyncProvider(scopeName, serverConn); // lets intercept the changes so we can rename the columns remoteProvider.ChangesSelected += new EventHandler<DbChangesSelectedEventArgs>(remoteProvider_ChangesSelected); SyncOrchestrator syncOrchestrator = new SyncOrchestrator(); syncOrchestrator.LocalProvider = localProvider; syncOrchestrator.RemoteProvider = remoteProvider; // set the direction of sync session Download syncOrchestrator.Direction = SyncDirectionOrder.Download; // execute the synchronization process SyncOperationStatistics syncStats = syncOrchestrator.Synchronize(); // print statistics Console.WriteLine("Start Time: " + syncStats.SyncStartTime); Console.WriteLine("Total Changes Downloaded: " + syncStats.DownloadChangesTotal); Console.WriteLine("Complete Time: " + syncStats.SyncEndTime); Console.WriteLine(String.Empty); Console.ReadKey(); static void remoteProvider_ChangesSelected(object sender, DbChangesSelectedEventArgs e) { // check to see whether we are synchronising the table we are interested if (e.Context.DataSet.Tables.Contains("GlobalCommonTable")) { DataTable dataTable = e.Context.DataSet.Tables["GlobalCommonTable"]; //rename the columns to match the destination table’s column names dataTable.Columns["first_name"].ColumnName = "Start_Name"; dataTable.Columns["last_name"].ColumnName = "End_Name"; } }
My Database StructureSource DB
Destination DBWhen iam running the server and client part it's working correctly and all the database is getting provisioned (you can see in the snapshot of the DB) but when iam trying to run the Synchronising part it's havong the problem as mentioned above.
Anything wrong in my code.
-
23 sierpnia 2011 08:30Moderator
put a breakpoint on the ChangesSelected code and see if it's actually getting a change.
if it has changes, subscribe to the ApplyChangesFailed event and see if there are errors or conflicts.
-
23 sierpnia 2011 08:47
I checked in the ChangesSelected event and the changes were getting reflected. I also subscribed to the ApplyChangesFailed event and put a break point there also but it was not going there and also code was executing properly but thesyncStats.DownloadChangesTotal value was still 0
Any other suggestion. -
23 sierpnia 2011 10:38Moderator
did you subscribe to the local provider's ApplyChangesFailed or the remote provider?
run Sql Profiler on your local db and see if its sending commands to update your local db.
-
23 sierpnia 2011 12:01I tried on remote provider. Do i need to subscribe to local provider's ApplyChangesFailed event.
-
23 sierpnia 2011 12:07Moderatorthe changes is being applied on the local provider, so you should check for errors in applying it on the local provider's ApplyChangesFailed.
-
23 sierpnia 2011 12:31
I ran the SQL profiler and there it seems to work fine as there were this data which shows that data was getting inserted to the table.
Below is a part of the SQL profiler report.
declare @p3 dbo.Destination_BulkType insert into @p3 values(301,N'Shaleen123',N'Pandiya',2695,1,2695,1) insert into @p3 values(302,N'Shaleen123',N'Pandiya',2696,1,2695,1) insert into @p3 values(305,N'Shaleen456',N'Pandiya',2697,1,2695,1) insert into @p3 values(306,N'Shaleen789',N'Pandiya',2698,1,2695,1) insert into @p3 values(307,N'Shaleen7890',N'Pandiya',2699,1,2695,1) insert into @p3 values(315,N'ShaleenTest',N'Pandiya',2700,1,2695,1) insert into @p3 values(400,N'shruthi',N'h',2701,1,2695,1) insert into @p3 values(888,N'priyanka',N'priyanka',2702,1,2695,1) insert into @p3 values(6666,N'shruthi',N'h',2703,1,2695,1) insert into @p3 values(8888,N'sushma',N'Palled',2704,1,2695,1) insert into @p3 values(31352,N'Paresh',N'Mohanty',2705,1,2695,1) insert into @p3 values(31356,N'O',N'Wong',2706,1,2695,1) insert into @p3 values(31357,N'sasd',N'sdsa',2707,1,2695,1) insert into @p3 values(31358,N'sasd',N'sdsa',2708,1,2695,1) insert into @p3 values(31359,N'g',N'g',2709,1,2695,1) insert into @p3 values(31360,N'Victor',N'Kouznetsov',2710,1,2695,1) insert into @p3 values(31361,N'Oyland',N'Wong',2711,1,2695,1) insert into @p3 values(31362,N'HowNOW',N'BrownCow',2712,1,2695,1) insert into @p3 values(31363,N'Paresh',N'Mohanty',2713,1,2695,1) insert into @p3 values(31364,N'Me',N'Myself',2714,1,2695,1) insert into @p3 values(31366,N'123456',N'123456',2715,1,2695,1) insert into @p3 values(31367,N'Julius',N'Gabby',2716,1,2695,1) insert into @p3 values(31368,N'Gerard',N'de L`homme',2717,1,2695,1) insert into @p3 values(31369,N'William',N'Grove',2718,1,2695,1) insert into @p3 values(31370,N'Jack',N'Clark',2719,1,2695,1) insert into @p3 values(31371,N'Francis',N'Refol',2720,1,2695,1) insert into @p3 values(31372,N'asdf',N'asdf',2721,1,2695,1) insert into @p3 values(31373,N'Dennis',N'Callan',2722,1,2695,1) insert into @p3 values(31374,N'Marc',N'Veary',2723,1,2695,1) insert into @p3 values(31375,N'David',N'Sherry',2724,1,2695,1) insert into @p3 values(31376,N'Stuart',N'Schillinger',2725,1,2695,1) insert into @p3 values(31377,N'Guillaume',N'Ardoise',2726,1,2695,1) insert into @p3 values(31378,N'Andriescu',N'Alexandru',2727,1,2695,1) insert into @p3 values(31379,N'Alex',N'Kachanov',2728,1,2695,1) insert into @p3 values(31380,N'Dan',N'Melchione',2729,1,2695,1) insert into @p3 values(31381,N'Ted',N'Johnson',2730,1,2695,1) insert into @p3 values(31382,N'Mario',N'Rose',2731,1,2695,1) insert into @p3 values(31383,N'Brian',N'Mann',2732,1,2695,1) insert into @p3 values(31384,N'Robot',N'Man',2733,1,2695,1) insert into @p3 values(31385,N'hello',N'howdy',2734,1,2695,1) insert into @p3 values(31386,N'Niels',N'Frederiksen',2735,1,2695,1) insert into @p3 values(31387,N'Scott',N'Wolf',2736,1,2695,1) insert into @p3 values(31388,N'greg',N'dingas',2737,1,2695,1) insert into @p3 values(31389,N'Guzou',N'Patrice',2738,1,2695,1) insert into @p3 values(31390,N'INAS',N'MAKAR',2739,1,2695,1) insert into @p3 values(31391,N'David',N'Mortman',2740,1,2695,1) insert into @p3 values(31392,N'Richard',N'White',2741,1,2695,1) insert into @p3 values(31393,N'Chris',N'Littell',2742,1,2695,1) insert into @p3 values(31394,N'rob',N'wortman',2743,1,2695,1) insert into @p3 values(31395,N'Chris',N'Browning',2744,1,2695,1) insert into @p3 values(31396,N'Barry',N'Fredericks',2745,1,2695,1) insert into @p3 values(31397,N'Sergey',N'Fidotov',2746,1,2695,1) insert into @p3 values(31398,N'sigridur ',N'ingimundardottir',2747,1,2695,1) insert into @p3 values(31399,N'Drew',N'Cohen',2748,1,2695,1) insert into @p3 values(31400,N'Jeff',N'Platon',2749,1,2695,1) insert into @p3 values(31401,N'Bert',N'Kashyap',2750,1,2695,1) insert into @p3 values(31402,N'KS',N'Loh',2751,1,2695,1)
Also i subscribed to the local provider's ApplyChangesFailed event but while debugging it was not going to the event itself. Please see the below code for your reference.
class Program { static void Main(string[] args) { //setup the connections var serverConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncSource;User Id=asapteam;Password=welcome"); var clientConn = new SqlConnection(@"Data Source=IIS08TOPSDEVDB1\IIS08TOPSDEVDB1; Initial Catalog=SyncDestination;User Id=asapteam;Password=welcome"); //setup scope name const string scopeName = "DifferentSchemaScope"; //provision server var serverProvision = new SqlSyncScopeProvisioning(serverConn); if (!serverProvision.ScopeExists(scopeName)) { var serverScopeDesc = new DbSyncScopeDescription(scopeName); // add the server table var serverTableDesc = SqlSyncDescriptionBuilder.GetDescriptionForTable("Source", serverConn); // our server and client tables has different names, so let’s setup a common name serverTableDesc.GlobalName = "OrderTable"; serverScopeDesc.Tables.Add(serverTableDesc); serverProvision.PopulateFromScopeDescription(serverScopeDesc); //apply the scope definition serverProvision.Apply(); } //provision client var clientProvision = new SqlSyncScopeProvisioning(clientConn); if (!clientProvision.ScopeExists(scopeName)) { var clientScopeDesc = new DbSyncScopeDescription(scopeName); // add the client table var clientTableDesc = SqlSyncDescriptionBuilder.GetDescriptionForTable("Destination", clientConn); clientTableDesc.GlobalName = "OrderTable"; clientScopeDesc.Tables.Add(clientTableDesc); // remove the OrderQty column since the Source table doesnt have it clientScopeDesc.Tables["OrderTable"].Columns.Remove(clientScopeDesc.Tables["OrderTable"].Columns["Temp1"]); clientScopeDesc.Tables["OrderTable"].Columns.Remove(clientScopeDesc.Tables["OrderTable"].Columns["Temp2"]); clientProvision.PopulateFromScopeDescription(clientScopeDesc); //apply the scope definition clientProvision.Apply(); } // create the sync orchestrator var syncOrchestrator = new SyncOrchestrator(); //setup providers var localProvider = new SqlSyncProvider(scopeName, clientConn); var remoteProvider = new SqlSyncProvider(scopeName, serverConn); // lets intercept the changes so we can rename the columns remoteProvider.ChangesSelected += new EventHandler<DbChangesSelectedEventArgs>(remoteProvider_ChangesSelected); localProvider.ApplyChangeFailed += new EventHandler<DbApplyChangeFailedEventArgs>(localProvider_ApplyChangeFailed); syncOrchestrator.LocalProvider = localProvider; syncOrchestrator.RemoteProvider = remoteProvider; // set the direction of sync session Download syncOrchestrator.Direction = SyncDirectionOrder.Download; // execute the synchronization process var syncStats = syncOrchestrator.Synchronize(); // print statistics Console.WriteLine("Start Time: " + syncStats.SyncStartTime); Console.WriteLine("Total Changes Downloaded: " + syncStats.DownloadChangesTotal); Console.WriteLine("Complete Time: " + syncStats.SyncEndTime); Console.WriteLine(String.Empty); Console.ReadKey(); } static void remoteProvider_ChangesSelected(object sender, DbChangesSelectedEventArgs e) { //let’s check if we’re synching the table we’re interested if (e.Context.DataSet.Tables.Contains("OrderTable")) { var dataTable = e.Context.DataSet.Tables["OrderTable"]; //rename the columns to match the destination table’s column names dataTable.Columns["first_name"].ColumnName = "Start_Name"; dataTable.Columns["last_name"].ColumnName = "End_Name"; } } static void localProvider_ApplyChangeFailed(object sender, DbApplyChangeFailedEventArgs e) { //let’s check if we’re synching the table we’re interested if (e.Context.DataSet.Tables.Contains("OrderTable")) { var dataTable = e.Context.DataSet.Tables["OrderTable"]; //rename the columns to match the destination table’s column names dataTable.Columns["first_name"].ColumnName = "Start_Name"; dataTable.Columns["last_name"].ColumnName = "End_Name"; } } }
Please suggest if any implementation is going wrong.
Thanks in advance.
-
23 sierpnia 2011 13:26Moderator
the insert you posted is against the UDT. check the commands in the profiler if its even calling the insert or update stored procs.
i notice you're synching the same tables as your original code, have you deprovisioned the previous scopes before creating the new ones? if you didnt deprovision the old scopes using the same table, the insert/update stored procs and the UDT from the first scope created on this tables never gets updated when you add a new scope.
-
23 sierpnia 2011 13:44
First what do you mean "against the UDT".
I checked the profiler report when the Synchronising part of the code was running and they seems to have executed. BTW i have saved the report when Synchronising parrt of the code was running (i have the report for when ServerDB and ClientDB were getting provisioned also). Do you want to review it.
Please let me know how can i upload a file into this thread.
-
23 sierpnia 2011 13:58Moderator
UDT = User Defined Type.
each table in your scope has a corresponding UDT. you can check it under using SSMS under programmability-> types-> user-defined table types
so the rows are getting inserted/updated and your problem is just the download stats displaying zero?
-
23 sierpnia 2011 14:16
For deprovising the previous scope i am droping the tables (4 tables) which were created during the provisioning of the DB and then only creating the new one. Do i need to do anyother thing?
NOTE: I was doing the same thing when i was synching the table where column names were same and there in that case it was working fine.
For the other question "so the rows are getting inserted/updated and your problem is just the download stats displaying zero?" --> No the rows are not getting inserted/updated in the Database also.
-
23 sierpnia 2011 14:31Moderator
i suggest you use the deprovisioning api when cleaning up scopes. search for deprovisioning in the docs to get an idea how to deprovision.
when you provision, sync framework creates stored procedures, triggers, and UDTs in addition to the tracking tables. sometimes these doesnt get cleaned up when you manually drop the tables and they get picked up by the new scope on the same tables when you reprovision.
- Oznaczony jako odpowiedź przez Shals1031 23 sierpnia 2011 15:34
-
23 sierpnia 2011 15:34
Thanks it done, by using the Deprovisioning i was bale to achieve my task.
Thnaks a ton June for your constant help now no more questions :)