Monday, October 22, 2012 10:38 PM
Our CRM Email router has been running fine for over the last year and recently stopped working. We are using CRM 2011 with UR 8 installed. We are using A single Outgoing configuration profile, 1 deployment, and the users queues and forward mailboxes tab works with load data. If I select a user and hit test access it takes a while and then errors with the message below:
This happened in both my Prod and QA envoirnment. In prod I tried to uninstall / reinstall and the errors were the same. I ended up building a new server and installing from scratch, which fixed everything. In QA my CRM app server and Email Router are on the same server and will be a bit more work to rebuild. I would prefer to find a fix in QA so if this happens again I can fix with out a rebuild. Anyone have some suggestions? I have rebooted the servers, reinstalled the router, and I have renamed the Microsoft.Crm.Tools.EmailAgent.systemstate.xml file to "old" and removed the .bak on the file name Microsoft.Crm.Tools.EmailAgent.systemstate.xml.bak
Server: smtp host name
Outgoing Status: Failure - An error occurred while checking the connection to the Microsoft Dynamics CRM server http://hostname/org name. Metadata contains a reference that cannot be resolved: 'http://hostname/org name/XrmServices/2011/Organization.svc?wsdl'. The operation has timed out
Event Log on the server: Source: MSCRMEmail, Evetn ID 26234 Level: Error
#26234 - The E-mail Router service could not process a provider work item using assembly: Microsoft.Crm.Tools.EmailProviders.dll and class: Microsoft.Crm.Tools.Email.Providers.SmtpPollingSendEmailProvider. System.InvalidOperationException: Metadata contains a reference that cannot be resolved: 'http://hostname/org name/XrmServices/2011/Organization.svc?wsdl'. ---> System.Net.WebException: The operation has timed out
at System.ServiceModel.Description.MetadataExchangeClient.MetadataLocationRetriever.DownloadMetadata(TimeoutHelper timeoutHelper)
at System.ServiceModel.Description.MetadataExchangeClient.MetadataRetriever.Retrieve(TimeoutHelper timeoutHelper)
--- End of inner exception stack trace ---
at Microsoft.Crm.ServiceProxyCache`1.GetNewServiceProxy(Uri serviceUrl, Credential credentials, Uri homeRealmUrl, String passportEnvironment, String onlineServiceEnvironment)
at Microsoft.Crm.Tools.Email.Providers.Utility.BuildOrganizationService(Uri organizationServiceUrl, Credential credentials)
at Microsoft.Crm.Tools.Email.Providers.Utility.GetCrmService(Uri discoveryUri, String authMode, String userName, String password)
at Microsoft.Crm.Tools.Email.Agent.ServiceCore.ExecuteProviderWork(Object providerQueueRequestObject)
- Moved by JLattimerMicrosoft Community Contributor, Moderator Tuesday, October 23, 2012 1:41 AM (From:CRM)
Wednesday, October 24, 2012 11:47 PM
If anyone can tell me how to fully uninstall the router that may help. On the prod server a uninstall / reinstall of the router didnt work. I had to reinstall windows and install the router with the same settings and it solved my issues.
It appears the uninstall is leaving somthing behind. I would assume a registry entry. In QA I uninstalled the email router and deleted 2 things left over in the registry that were obvious to CRM email router and reinstalled to have the same error. There has got to be more reg keys somewhere that are not obvious that need to be deleted. I really should not have to reinstall windows and then the router to fix this problem.
Wednesday, December 12, 2012 7:04 PMThis problem went away for an unknown reason and has now come back. Any ideas?
Wednesday, December 19, 2012 10:44 PM
More than likely an uninstall/re-install won’t help in this situation. It seems like there could be a performance issue in the environment somewhere.
First I would see if you can get to 'http://hostname/orgname/XrmServices/2011/Organization.svc?wsdl' in IE from the machine that hosts the email router. You should see an .xml page get returned
Then I would look at using KB 968520 to cleanup the Async table if it’s getting to big, usually anything over 500,000 records in this table will start causing perf issues.
Then I would look at the CRM 20011 Performance white paper.
You could also try decreasing the Maximum Messages per cycle down to 500 from the default 1000 on the Outgoing profile
You could also trying increasing the polling period from 60 to 90 or 120 on the Outgoing profile.
How long does it take before it times out? If its 30 seconds it could be a SQL timeout. You could try adding the OLEDBTimeout Reg key and give it a value of around 300 to see if that helps.
Are you experiencing any other perf issues on the servers?
Do you have any firewalls or any other security software that could be causing issues?
Do you have any proxy servers?
Do you have IFD/ADFS configured?
Do you have https configured on the CRM website or in Deployment manager?
- Edited by ChrisDodgson1 Wednesday, December 19, 2012 10:45 PM
Friday, December 21, 2012 12:54 AM
Thank you for taking the time to respond. I have been fighting with this for a while. I had 2 environments both start doing this after working perfectly for almost 2 years. They both did it and the problem went away on both on its own. It came back again on both, and now only one does it. On the current broken enviornment I have uninstalled / reinstalled and the problem is still there. However I built a new seperate server and it works fine with the same app and sql servers, so there is somthing wrong with the server itself.
http://hostname/orgname/XrmServices/2011/Organization.svc?wsdl from the error message resolved to a XML file
I dont think the KB will apply here as the seperate "fresh built server" works fine using the same settings.
I decreased from 1000 to 500 and it threw the same error
It takes about 5 minutes to timeout and throw the error in the route test access window. I will try setting the timeout it cant hurt!
I dont see any performance issues and this is a test enviornement so there is only a few users on the system at a time if any.
No firewalls and windows firewalls are disabled
we have a Internet Proxy server used as an explict proxy. Local servers are bypassed in the browser configurations.
IFD is not enabled
We are not using HTTPS
Friday, December 21, 2012 1:14 AM
I have also installed UR 11 since this post, that didnt help. I tried the reg keys in http://support.microsoft.com/kb/918609, rebooted and still got the error so I removed them. I also tried http://support.microsoft.com/kb/968520 as you suggested just for fun and that didnt fix it either =(
What else do you suggest I try. Again thanks for your help and input.
Saturday, January 19, 2013 3:47 AM
I had a similar issue after when my IIS bindings got messed up. The IIS site wasn't listening on loopback anymore so the email router which was installed on the same server wasn't able to connect to the CRM web services. I set the bindings for the IIS site that CRM was using to "All Unassigned", and everything took off again. You can also check out my blog post on it if that helps http://nerdsknowbest.blogspot.com/2013/01/mscrmemail-e-mail-router-service-could.html
Also, if you are running the router service under a service account, make sure it isn't having authentication issues (e.g. locked out, expired, ADFS issues, etc.)
I hope that helps your issue.
- Edited by G. Kjono Saturday, January 19, 2013 3:56 AM
Tuesday, January 22, 2013 7:56 PMBindings look ok, and the service is running under local system. Thanks for the post and ideas! Got anymore?
Tuesday, January 22, 2013 8:47 PM
If you go into the Email Router Configuration Manager, on the deployments tab you should have a deployment specified. Select it and click modify. Are there any credentials specified to connect to the web service? If so, I would check that account.
You could try adding the hostname that is specified in that tab to the host file with the IP of 127.0.0.1 as well. It could be having a loopback issue getting to the IP that's specified in DNS as well.
Tuesday, January 22, 2013 9:12 PM
The deployment is setup with "local System account" for access creds. If you go to the User/Queues/ Forward Mailboxes and click load it populates the users, when I select one of the users and click test access it says testing and hangs for ever and eventually fails.
I would think if the router can not talk to the server it would not be able to load the users?!?! I tried the NetBios name and FQDN with the host file and still have the error.
Tuesday, January 22, 2013 11:20 PM
For fun I have changed the router service to run as my account and the router started to work again. I put the service back to run under local system and the error came back. To verify what rights were needed and the proper setup I pulled down the MS doc for the Email Router installation. The computer account is part of the PrivUserGroup and has been since installation and the service is running as the local system when the error occurs.
- For E-mail Router, the following conditions must be met:
- The account that is running the E-mail Router must be LocalSystemAccount.
- The computer where the E-mail Router service is running must be added to the PrivUserGroup group.
For the Rule Deployment Wizard, the following conditions must be met:
- The user must have logged on as a Microsoft Dynamics CRM user with a security role.
- The user must be a local administrator on the computer where the wizard is running.
- The user must have administrative permissions on the Exchange Server.
Monday, February 18, 2013 11:00 AM
I have the exact same problem, CRM e-mail router was working just fine and suddenly I discovered today the same errors as TechGuy18 describes.
The only thing that got it to work is to change the service account from Local System to a domain account.
Nothing changed in the mean time, not the bindings, not the AD, nothing.
Do we have a solution to this problem?
Friday, May 03, 2013 6:56 PM
I finally was able to fix this, however this was listed as a fix for CRM 4.0 and both of the circumstances are not true. My deployment has the Org Web Service and Discovery Web Service on the same computer. Either way my problem was resolved with the resolution in this link. I still don't know why my original setup worked for years and randomly stopped working and why this was needed to fix it.........
- Marked As Answer by nrodriEditor Wednesday, May 15, 2013 1:51 PM