I've created a web service in C# that I am calling from a SQL Agent scheduled job and that integrates data into CRM 2011 On-Premise. The web server is load-balanced with two servers in the farm at this time. On the CRM App Servers, they are also load-balanced and
the oddity is that on the CRM App Services, it runs about 95% of the processes on one and 5% on the other. The reason for this is that there are four unique process that are importing data into a database, as follows.
- 15K records per hour
- 100 records per hour
- 20K records per hour
- 1K records per hour
Note that the difference between the volumes ran by each are significant. What we find is that #2/#4 runs on the same server and #1/#3 run on the same server. Because #2 and #4 take up about 95% of the workload, server-2 is being underutilized.
What I would like to do is figure out how to break apart the two larger jobs into small subsets of data. This is running for about 60 offices/branches we have across the country. So I think that if I can adjust the design a bit, so that I can
fire off separate sessions/jobs per branch, the load-balancer would be better able to balance the workload. However, I am not sure how to go about designing/architecting this, as I am not sure exactly what the CRM App Servers are doing and how I
can go about getting them to correctly load-balance updates firing off from my web-service.
I might need the web service methods to fire off in such a way as to create multiple sessions that the load-balancer will balance for me. Currently, I am created unique threads per branch. So I think the best option here is to adjust the
design to create unique sessions where I am currently creating unique threads.
So my question is, what does the CRM App Server load-balancer need in order to consider something as a separate session or job? Would I need to create say a unique method for each session, or could I go into the global.asax for the web-service
and create muiltiple sessions in such a way that the load-balancer could pick them up and balance them correctly?
I just have no idea where to start looking at this, but I am sure it's a common issue to run into. So I am hoping someone that has done this can offer some suggestions or point me in the right direction.
Best regards,
Jon
Jon Gregory Rothlander
