You question about making the service thread safe... is an excellent question and something I am working through as well. I can explain my approach and how I am attempting this. I'm not 100% sure my approach is correct, but I am working through
it. It would be wonderful to find others doing similar things and share what we have found that works well and what is problematic.
For me, my web service just talks to a service class and 99% of my logic is in the service class, which is just a C# class library. That helps me with my unit testing and I think the design is much cleaner. In that service class library
I have a classes that contain what I am calling my threaded classes. So my web service nor my service class library are being called multiple times, but only once. Well, actually 8 times per hour, once per primary entity
that I am syncing. But the calling of web service is not where the threading comes into play.
The web service calls the service class library to execute a process that pulls a list of keys of entities that have been recently updated in the legacy system (updates after last sync cycle), which is typically a few thousand or
more, and sometimes as many as ten-thousand records. I then divide those records into logical sets based on the count, the branch (we have about 50 across the US), and I have a CRM configuration entity where I set the number of threads per
branch and others things. So I can adjust and throttle how this works. Once the number of records and number of threads are determined, the logic creates a thread and passes each set of keys to the thread and executes it. I'm playing
with just how to throttle the number of records per thread, thread pooling, and using tasks instead of threads, and the number of threads, etc. as I have a LOT of records to sync and I suspect that I can easily overload CRM and our servers.
I have overloaded the servers in one of our DEV environments, as it is a much smaller server. But so far everything is running well in production.
It is the service class library that spawns threads of the threaded classes. The threaded classes are doing the low-level sync between the legacy application and CRM. So by the time I spawn a thread, it is only reading from the legacy database
and making a call out to CRM through the SDK. What I found is that CRM SDK seems to handle many threads all updating creating/updating records in CRM. Although each thread seems to run pretty slow, I've been able to spawn
off as many as 100-threads at a time without it having any issues. However, I've pulled that back a great deal, as I don't want to push it to its limits.
In regards to thread-safe, I been working directly with non-pool threads in my sync classes, but I also have other smaller classes that I am spawning as tasks. I suspect that my non-pooled design has issues in this regards. To work around
that I have spent a lot of time setting up IDisposable in all of my classes and I perform a .Dispose cascade through all of my classes to handle my own garbage collection... and I suppress the garbage collector. My goal here is to free up memory resources
as fast as possible and not depend on .Net GC, but I am still working through the design. Currently when I monitor the memory profile, it seems to run well and memory seems well managed. But I am not 100% sure about
the design yet. I suspect I have many issues that I have yet to resolve.
Jon Gregory Rothlander
