Sync Services Failing on a Large Table

All replies

• 

  

  0

  Sign in to vote

  Hi Matthew,
  
  have you tried turning on the tracing or inspecting the inner exception if it contains any further information aside from the "Unable to enumerate changes..."?
  
  As for your second question, it's not uncommon to encounter memory issues when serializing large datasets. The dataset has two copies for each row (a before and after if i remember correctly). So for every row you download/upload you actually have two copies in the serialize message. You can turn on WCF Message logging to see for yourself.
  
  As a workaround to serializing a dataset, you may want to look up post on using a DatasetSurrogate (you convert the dataset to a custom object with rows contain in a list). In my previous project, I ended up converting the dataset to a datasetsurrogate then converting the datasetsurrogate to a byte array and passing the byte array to WCF. Then on the receiving side, you simply do the opposite.
  
  Another option you have is to move to a Collaboration scenario in SyncFx v2 and use the built-in memory-size based batching.
  
  Cheers,
  
  JuneT 
  

  • Proposed as answer by Patrick S. Lee Friday, March 5, 2010 12:16 AM
  • Marked as answer by MattMJB Friday, March 5, 2010 4:52 PM

  Thursday, March 4, 2010 6:23 PM
• 

  

  0

  Sign in to vote

  Thanks June. 
  
  I have tried adding additional tracing to see the inner exception but wasn't able to find any. Below are the additional traces I added to the WCF service. Is there anything else that I could add that might be helpful in terms of debugging?
  
  <sources>
      <source name="System.ServiceModel"
        switchValue="Information, ActivityTracing"
        propagateActivity="true">
       <listeners>
        <add name="xml" />
       </listeners>
      </source>
      <source name="CardSpace">
       <listeners>
        <add name="xml" />
       </listeners>
      </source>
      <source name="System.IO.Log">
       <listeners>
        <add name="xml" />
       </listeners>
      </source>
      <source name="System.Runtime.Serialization">
       <listeners>
        <add name="xml" />
       </listeners>
      </source>
      <source name="System.IdentityModel">
       <listeners>
        <add name="xml" />
       </listeners>
      </source>
     </sources>
  
  
  Thanks for your work around suggestions as well. I will look into those futher as wel.
  
  Matthew
  

  Thursday, March 4, 2010 10:36 PM
• 

  

  0

  Sign in to vote

  here's some link for enabling tracing : http://msdn.microsoft.com/en-us/library/ee617387(SQL.105).aspx & http://msdn.microsoft.com/en-us/library/cc807160(SQL.105).aspx
  
  it also helps if you subscribe to the different events such as SelectingChanges, ApplyChanges, etc...so you get to see where it's exactly failing.
  
  cheers,
  
  junet

  Friday, March 5, 2010 1:27 AM
• 

  

  0

  Sign in to vote

  Thanks, I will check those out.
  
  I figured out my initial enumeration problem, and the root cause appears to be the style of testing I was doing. I would load the table with a large amount of records, based on a date range, and then test the synch. If it didn't work, then I would then delete all the records, resynch, and then try loading a slightly smaller subset. I think the net effect of this was that I was generating way too much change tracking, so that even when I tried to synch against the table with a very small record set after reseting the anchor, the sheer volume of previous change tracking data was crashing it.
  
  It makes sense now that the only scenario that would work was synching when the problem table was empty. If I then reloaded the data and resynched, synch services was able to properly set my initial min_valid_version and not have to go back through all the enumerations...
  
  To resolve this in my test environment, I simply turned off change tracking on the table, and then turned it back on. Now I'm down to just my memory issue, which I will try to resolve thanks to junet's suggestions.
  
  Cheers,
  Matthew

  • Marked as answer by MattMJB Friday, March 5, 2010 4:53 PM

  Friday, March 5, 2010 4:52 PM
• 

  

  0

  Sign in to vote

  JuneT I know we've talked about this before, but do you have an example of where you used dataset surragates?

  ---

  Travich

  Friday, March 5, 2010 8:23 PM
• 

  

  0

  Sign in to vote

  travich,
  
  let me write one and post later.
  
  junet

  Saturday, March 6, 2010 1:47 AM
• 

  

  0

  Sign in to vote

  travich,
  
  here you go: http://jtabadero.spaces.live.com/blog/cns!BF49A449953D0591!1187.entry

  Monday, March 8, 2010 3:16 PM