locked
Realtime? RRS feed

  • Question

  •  

    Does this enable realtime peer to peer synchronization? Or is it only an on-demand synchronization.
    Tuesday, November 6, 2007 12:02 AM

Answers

  • There are no specific capabilities built into the Sync Framework to enable realtime synchronization and for the most part it is on-demand.  However, there is nothing limiting you from enabling near realtime synchronization programatically.  For example, through triggers in databases or event / file watchers. 

     

    For example, a demo that we are showing at TechEd in Barcelona and DevConnections in Las Vegas uses the Microsoft Sync Framework with file and database "watchers" that detect changes in the database and file systems and automatically synchronize information.

     

    Liam

     

    Tuesday, November 6, 2007 12:21 AM
  •  

    Hi Graham,

    Let me first start out by stating, although we cannot give you absolute real time synchronization I believe we can get close to real time and that I believe what you have outlined should be possible.  I do however want to be upfront and outline what we do and do not have today, some of the issues I think you will want to consider as well as some of the concepts I think you will need to know to implement your solution.

    When doing Peer to Peer (P2P) Synchronization using the Microsoft Sync Framework you can (at a high level) split it up into two main components:

    ·         Core Microsoft Sync Framework:  This is the piece of the Microsoft Sync Framework which handles all of the common things important to synchronization.  For example, it detects and resolves the conflicts, it ensures guaranteed data exchange and it also works with the Metadata store.  Metadata and Knowledge are extremely important concepts in P2P synchronization and I highly recommend you read more on this as well as the concept of synchronizing custom data stores in our developer center library (http://msdn2.microsoft.com/en-us/library/bb902847(SQL.100).aspx).  Luckily the metadata can be stored in SQL Server Compact (although this is not required).

    ·         Sync Service Providers:  This is the piece of the Microsoft Sync Framework which sits on top of the Core Microsoft Sync Framework and is the piece which interacts directly with the data store (SQL Server Compact in your case).   We have created a number of Sync Service Providers (such as one for SQL Server, NTFS file sync and SSE (rss and atom feeds).  We have not yet, created one for SQL Server Compact.  This means if you wanted to implement this now, you would be required to create a provider to SQL Server Compact on your own.  As an aside, I really think the ability to create your own providers is one of the coolest pieces of the Microsoft Sync Framework because it allows you to extend synchronization to any data store in which you can programmatically get access to the data.

    I do also want to point out that the Microsoft Sync Framework has not been built to be “real time”.  In most real time environments, people will assume changes will go through as they happen.  With the Microsoft Sync Framework, the speed for exchange will depend on how quickly you can detect the changes, how quickly you can launch synchronization and how quickly those changes can be propagated to each of the peers.  This is important in your case since SQL Server Compact does not have triggers which could launch synchronization automatically.  You will need some way of programmatically detecting the change and then launching sync.  The method in which you populate the metadata store is also important because as you state, you want to ensure synchronization is not affected as your database grows.  As such, you need to have an efficient way of updating the metadata storage tables with your changes.

    I would also like to recommend that you take a look at the following whitepapers which I think will help you get an idea of what is required to implement your solution.

    ·         Introduction to the Microsoft Sync Framework Runtime: Overview of the core of the Microsoft Sync Framework

    ·         Introducing Microsoft Sync Framework: Sync Services for File Systems:  Although this talks about synchronizing files (rather than data) the concepts should apply to your ability to sync from SQL Server Compact.

    I guess in summary what I am saying is that I do believe you should be able to get “near” real time P2P synchronization with the Microsoft Sync Framework and I hope what I have outlined helps clear up some of your questions.

    Please let me know if I can help further.

    Liam
    Tuesday, November 6, 2007 7:59 PM
  • I am sorry that I can not give you a firm date on this other than to say that it is one of the items that we consider important to implement in the near future.  I am hopeful we will have something in the first half of 2008. 

     

    I do want to also stress that creating a provider is really not that difficult once you understand the metadata and knowledge concepts of the Microsoft Sync Framework and I do not expect many stumbling block.  So realistically I don't think you are losing much if you swap out your provider with ours (once we have it).  I will also check with some of the engineers to make sure there are no additional stumbling blocks that I am not aware of.

     

    Perhaps you could email me directly (liam.cavanagh@NO_SPAM_microsoft.com) so I can update you with our progress as it happens.

     

    Liam

    Tuesday, November 6, 2007 9:03 PM

All replies

  • There are no specific capabilities built into the Sync Framework to enable realtime synchronization and for the most part it is on-demand.  However, there is nothing limiting you from enabling near realtime synchronization programatically.  For example, through triggers in databases or event / file watchers. 

     

    For example, a demo that we are showing at TechEd in Barcelona and DevConnections in Las Vegas uses the Microsoft Sync Framework with file and database "watchers" that detect changes in the database and file systems and automatically synchronize information.

     

    Liam

     

    Tuesday, November 6, 2007 12:21 AM
  • Liam,

    I am especially interested in trying to use this system to enable realtime or near-realtime synchronization of a SQL server CE DB over a peer to peer network. (Maybe enabled by peer channel or the windows grouping api). Is the synchronization mechanism efficient enought that I could say, perform a sync with all the peers every time I detect an update. We aren't talking about massive amounts of peers here, but I wouldnt want the performance of the system to dip as these databases grew. I haven't done enough analysis of the MSF architecture to know how well the syncing scales with the amount of data that needs to be kept synchronized.

     

    -Graham

     

    Tuesday, November 6, 2007 6:59 PM
  •  

    Hi Graham,

    Let me first start out by stating, although we cannot give you absolute real time synchronization I believe we can get close to real time and that I believe what you have outlined should be possible.  I do however want to be upfront and outline what we do and do not have today, some of the issues I think you will want to consider as well as some of the concepts I think you will need to know to implement your solution.

    When doing Peer to Peer (P2P) Synchronization using the Microsoft Sync Framework you can (at a high level) split it up into two main components:

    ·         Core Microsoft Sync Framework:  This is the piece of the Microsoft Sync Framework which handles all of the common things important to synchronization.  For example, it detects and resolves the conflicts, it ensures guaranteed data exchange and it also works with the Metadata store.  Metadata and Knowledge are extremely important concepts in P2P synchronization and I highly recommend you read more on this as well as the concept of synchronizing custom data stores in our developer center library (http://msdn2.microsoft.com/en-us/library/bb902847(SQL.100).aspx).  Luckily the metadata can be stored in SQL Server Compact (although this is not required).

    ·         Sync Service Providers:  This is the piece of the Microsoft Sync Framework which sits on top of the Core Microsoft Sync Framework and is the piece which interacts directly with the data store (SQL Server Compact in your case).   We have created a number of Sync Service Providers (such as one for SQL Server, NTFS file sync and SSE (rss and atom feeds).  We have not yet, created one for SQL Server Compact.  This means if you wanted to implement this now, you would be required to create a provider to SQL Server Compact on your own.  As an aside, I really think the ability to create your own providers is one of the coolest pieces of the Microsoft Sync Framework because it allows you to extend synchronization to any data store in which you can programmatically get access to the data.

    I do also want to point out that the Microsoft Sync Framework has not been built to be “real time”.  In most real time environments, people will assume changes will go through as they happen.  With the Microsoft Sync Framework, the speed for exchange will depend on how quickly you can detect the changes, how quickly you can launch synchronization and how quickly those changes can be propagated to each of the peers.  This is important in your case since SQL Server Compact does not have triggers which could launch synchronization automatically.  You will need some way of programmatically detecting the change and then launching sync.  The method in which you populate the metadata store is also important because as you state, you want to ensure synchronization is not affected as your database grows.  As such, you need to have an efficient way of updating the metadata storage tables with your changes.

    I would also like to recommend that you take a look at the following whitepapers which I think will help you get an idea of what is required to implement your solution.

    ·         Introduction to the Microsoft Sync Framework Runtime: Overview of the core of the Microsoft Sync Framework

    ·         Introducing Microsoft Sync Framework: Sync Services for File Systems:  Although this talks about synchronizing files (rather than data) the concepts should apply to your ability to sync from SQL Server Compact.

    I guess in summary what I am saying is that I do believe you should be able to get “near” real time P2P synchronization with the Microsoft Sync Framework and I hope what I have outlined helps clear up some of your questions.

    Please let me know if I can help further.

    Liam
    Tuesday, November 6, 2007 7:59 PM
  • Hmm... I guess I assumed that because MSF was providing an implementation of the metadata store on top of SQL Server CE that they would have also provided a Sync Service Provider for SQL Server CE. I don't mind trying to put this provider together, but I guess I would want to know if microsoft was planning on doing this already in the near future. The notifications wouldnt really be an issue, as I plan on tiering this out, and could just generate the notifications in an appropriate layer. Are there any particular stumbling blocks to trying to implement a Sync Service Provider for SQL Server CE? The idea here is to put together an easy way for a bunch of p2p client applications to work off a common data store. I had originally hoped that the Windows Grouping API would fit the bill, but it seems that its not really built for replicating data thats at all permanent.

     

    Tuesday, November 6, 2007 8:39 PM
  • I am sorry that I can not give you a firm date on this other than to say that it is one of the items that we consider important to implement in the near future.  I am hopeful we will have something in the first half of 2008. 

     

    I do want to also stress that creating a provider is really not that difficult once you understand the metadata and knowledge concepts of the Microsoft Sync Framework and I do not expect many stumbling block.  So realistically I don't think you are losing much if you swap out your provider with ours (once we have it).  I will also check with some of the engineers to make sure there are no additional stumbling blocks that I am not aware of.

     

    Perhaps you could email me directly (liam.cavanagh@NO_SPAM_microsoft.com) so I can update you with our progress as it happens.

     

    Liam

    Tuesday, November 6, 2007 9:03 PM