none
Is it possible to have 1 upload scope and multiple download scopes? RRS feed

  • Question

  • Hi All

     

    Currently I am using Sync Framework 2.1 to synchronize one server database with more than 100 client databases.

    The server database contains around 5GB of data.

    The Client is using SQL Server Express (datalimit of 10 GB)

     

    In the feature this will be upscaled to 2500 users (2500 client databases) so reducing amount of data is very important.

    The server database will be in the feature more than 20GB which requires to use filtering because of the 10GB limit on the client database.

    It is possible to filter data om Regions where client users are doing his job.

    We have calculated that we can reduce the data per client database to 4GB. This is perfect.

     

    But we have a problem:

    Whe were trying to use the following configuration

    1. ReferencialDataScope
      - This scope will be used to synchronize all referencial data tables (aka master data tables or primary key tables) (for example UserTypes, ProductType, ArticleType, OrderStatus, ...)
      - This scope will be Download only because the client is not allowed to change the data in one of these tables
      - This scope contians all tables which cannot be filtered on Region.
    2. DownloadDataTemplate
      - This template contains all tables which can be filtered on Region
      - This template contains a filter clause per table to filter on the region where the user belong to.
      - This template is Download only
    3. UploadDataScope
      - This scope contains all tables which can be manipulated by a client.
      - This scope is Upload only

     

    I know this is not working because scope 2 and 3 contains overlapping tables.
    But in theory they are not conflicting because:
    - The DownloadDataTemplate will only write SyncKnowledge to the client scope_config table and it is only manipulating the client tracking tables.
    - The UploadDataScope will only write SyncKnowledge to the server scope_config table and it is only manipulating the server tracking tables.

     

    Also if you use UploadAndDownload or DownloadAndUpload the SyncOrchestrator is starting two sessions (one for upload and one for download)

    this is not causing problems.

     

    So my question is.

    - How does Sync Framework detects that you are trying to upload data with another scope than you have downloaded this data (and visa versa)?

    - Is there a possibility to make this configuration possible?

     

    Thanks in advance

    Thursday, June 16, 2011 12:07 PM

All replies

  • Sync Framework works at the scope level and maintains "knowledge" at the scope level. it won't be able to detect overlapping scopes or overlapping data.

    in your case, i would create 3 scopes: ReferentialDataScope-DownloadOnly, DownloadTemplate-Filtered/Download only (filtered but client will not update), and change the 3rd one to be a Bidirectional scope (Upload/Download or Download/Upload) for filtered but updateable rows by client.

    Thursday, June 16, 2011 12:30 PM
    Moderator
  • Hi JuneT,

     

    First your commend about the "knowledge". This is correct but in case of a Download session this knowledge will be saved on the clients and in case of a Upload session the knowledge will be saved server-side.

    So this will never conflict. But the tracking table is saving the last scope_info.scope_local_id value into the <tablename>_tracking.update_scope_local_id column for each record that is synchronize with certain scope.

    The select changes stored procedure is checking on this column and thats the reason why a record will be selected if it need to be synchronized with another scope.

    Maybe it is possible to change these integers to let Sync Framework think these two scopes aren't different.

     

    About your scope suggestion.

    I whish I could do that but then I have another problem:

    It is possible that a client moves (for example a Article) record out of the region the user belongs to.

    In this case the record will never be uploaded to the server.

     

    For example: If all tables contains an IsActive column. Then you can create a filter so that all clients only downloading active records.

    but if the client changing  a record to inactive, it will never be uploaded to the server because it is out of the sight.

    In this case you can create another filter which only uploads inactive records. but thats not an option in my case.

     

    The restriction that you cannot have overlapping scopes is a big problem for me.

    Our database (120 tables) need to use filters on individuals, groups, regions and more, which all contains a lot of overlap with other filters.

    First I want to realize the filtering on regions. Then (if it is possible) try to implement other filters.

     

    So hopefully there is a way to make overlapping scopes possible.

     

    Maybe you provide me with anoter great post about sync framework on your blog :)

    So far your idle thoughts are very usefull.

    Thursday, June 16, 2011 1:20 PM
  • if your client will only contain rows for its own region, then you dont have to filter the client scope. in your example, the server scope can have a filter for Active rows only, and on the client scope, dont put a filter for Active rows. so a client can change a row from Active to InActive and it will still upload.

    however, other clients who has downloaded that row when it is still Active will never get the Inactive row since it's already "out of scope" in the server. sync framework doesnt support partition realignment, so that will be an issue.

    the problem with overlapping scopes is that you will have multiple scopes detect the same change and potentially result to rows being selected everytime.

    for example, if you have scope A and scope B and assuming you have bidirectional sync on both. if a row is updated on the client and uploaded to the server, when you sync Scope B, the same row will  be detected as changed and will be downloaded again. Now when you sync Scope A, the same row will be detected again and the loop goes on.

    glad to hear the blog posts are helping.

    i have a lot of catching up on my blogging, hopefully i find time soon to write again. 


    Thursday, June 16, 2011 1:51 PM
    Moderator
  • if your client will only contain rows for its own region, then you dont have to filter the client scope. in your example, the server scope can have a filter for Active rows only, and on the client scope, dont put a filter for Active rows. so a client can change a row from Active to InActive and it will still upload.

     

    This is where I am looking for but my configuration is not working becuause of your explaination below:

    for example, if you have scope A and scope B and assuming you have bidirectional sync on both. if a row is updated on the client and uploaded to the server, when you sync Scope B, the same row will be detected as changed and will be downloaded again. Now when you sync Scope A, the same row will be detected again and the loop goes on.

     

    If you change a record client side (without put the row "out of scope") the server will detect this as a change to send again. and the client will detect this downloaded record again and upload it to the server. Only put the record "out of scope" will end the loop. But then other users will have problems because it is moved in their scope.

     

    So probably I want to do something which can't be realized.

    In that's true I need to create a lot of scopes to avoid overlap.

    that's a difficult puzzle and probably it will cause performance problems because of a lot of synchronization sessions.

     

     

     

    Thursday, June 16, 2011 3:10 PM