locked
Asynchronous operation problems and errors RRS feed

  • Question

  • Here are the errors I am getting in the administrative events screen on my CRM server.

     

    I get this error after restarting MSCRMAsyncService

    Host CRM: failed while monitoring asynchronous operations queue. Exception: System.InvalidOperationException: There is an error in XML document (1, 983). ---> System.Xml.XmlException: '', hexadecimal value 0x03, is an invalid character. Line 1, position 983.
       at System.Xml.XmlTextReaderImpl.Throw(Exception e)
       at System.Xml.XmlTextReaderImpl.Throw(String res, String[] args)
       at System.Xml.XmlTextReaderImpl.Throw(Int32 pos, String res, String[] args)
       at System.Xml.XmlTextReaderImpl.ThrowInvalidChar(Int32 pos, Char invChar)
       at System.Xml.XmlTextReaderImpl.ParseNumericCharRefInline(Int32 startPos, Boolean expand, BufferBuilder internalSubsetBuilder, Int32& charCount, EntityType& entityType)
       at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars)
       at System.Xml.XmlTextReaderImpl.ParseText()
       at System.Xml.XmlTextReaderImpl.ParseElementContent()
       at System.Xml.XmlTextReaderImpl.Read()
       at System.Xml.XmlTextReader.Read()
       at System.Xml.XmlReader.ReadElementString()
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read148_StringProperty(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read158_Property(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read134_DynamicEntity(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read133_BusinessEntity(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read168_BusinessEntityCollection(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read1505_GetUnprocessedRecordsResponse(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read1611_Response(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read1621_ExecuteResponse()
       at Microsoft.Xml.Serialization.GeneratedAssembly.ArrayOfObjectSerializer1.Deserialize(XmlSerializationReader reader)
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
       --- End of inner exception stack trace ---
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle)
       at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
       at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
       at Microsoft.Crm.SdkTypeProxy.CrmService.Execute(Request Request)
       at Microsoft.Crm.Asynchronous.SdkTypeProxyCrmServiceWrapper.Execute(Object request)
       at Microsoft.Crm.Asynchronous.PublishDuplicateRuleOperation.GetUnpublishedRecords(String entityName, String[] colSet, Guid ruleId, Int32 maxRecords, String[]& versionNumbers, ICrmService crmService)
       at Microsoft.Crm.Asynchronous.PublishDuplicateRuleOperation.PublishWithPaging(ShareableDuplicateRule rule, Boolean isMainEntity, AsyncEvent asyncEvent, DuplicateRuleDataAccess data, ICrmService crmService, IConfiguration configuration)
       at Microsoft.Crm.Asynchronous.PersistMatchCodeOperation.InternalExecute(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.AsyncOperationCommand.Execute(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.AsyncHostHandler.Handle(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.QueueManager.PoolHandler.ProcessAsyncEvent(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.QueueManager.PoolHandler.InvokeHandlerInPool(Object state)
       at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
       at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)

     

    I then get this error

    Host CRM: failed while monitoring asynchronous operations queue. Exception: System.InvalidOperationException: There is an error in XML document (1, 983). ---> System.Xml.XmlException: '', hexadecimal value 0x03, is an invalid character. Line 1, position 983.
       at System.Xml.XmlTextReaderImpl.Throw(Exception e)
       at System.Xml.XmlTextReaderImpl.Throw(String res, String[] args)
       at System.Xml.XmlTextReaderImpl.Throw(Int32 pos, String res, String[] args)
       at System.Xml.XmlTextReaderImpl.ThrowInvalidChar(Int32 pos, Char invChar)
       at System.Xml.XmlTextReaderImpl.ParseNumericCharRefInline(Int32 startPos, Boolean expand, BufferBuilder internalSubsetBuilder, Int32& charCount, EntityType& entityType)
       at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars)
       at System.Xml.XmlTextReaderImpl.ParseText()
       at System.Xml.XmlTextReaderImpl.ParseElementContent()
       at System.Xml.XmlTextReaderImpl.Read()
       at System.Xml.XmlTextReader.Read()
       at System.Xml.XmlReader.ReadElementString()
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read148_StringProperty(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read158_Property(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read134_DynamicEntity(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read133_BusinessEntity(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read168_BusinessEntityCollection(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read1505_GetUnprocessedRecordsResponse(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read1611_Response(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read1621_ExecuteResponse()
       at Microsoft.Xml.Serialization.GeneratedAssembly.ArrayOfObjectSerializer1.Deserialize(XmlSerializationReader reader)
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
       --- End of inner exception stack trace ---
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle)
       at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
       at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
       at Microsoft.Crm.SdkTypeProxy.CrmService.Execute(Request Request)
       at Microsoft.Crm.Asynchronous.SdkTypeProxyCrmServiceWrapper.Execute(Object request)
       at Microsoft.Crm.Asynchronous.PublishDuplicateRuleOperation.GetUnpublishedRecords(String entityName, String[] colSet, Guid ruleId, Int32 maxRecords, String[]& versionNumbers, ICrmService crmService)
       at Microsoft.Crm.Asynchronous.PublishDuplicateRuleOperation.PublishWithPaging(ShareableDuplicateRule rule, Boolean isMainEntity, AsyncEvent asyncEvent, DuplicateRuleDataAccess data, ICrmService crmService, IConfiguration configuration)
       at Microsoft.Crm.Asynchronous.PersistMatchCodeOperation.InternalExecute(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.AsyncOperationCommand.Execute(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.AsyncHostHandler.Handle(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.QueueManager.PoolHandler.ProcessAsyncEvent(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.QueueManager.PoolHandler.InvokeHandlerInPool(Object state)
       at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
       at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)

     

    then This one

    The Microsoft CRM Asynchronous Processing Service service terminated unexpectedly.  It has done this 1 time(s).  The following corrective action will be taken in 60000 milliseconds: Restart the service.

     

    and then this one

    #26090 - An error occurred while opening mailbox Lisa@NorthAmericaTraffic.com. Microsoft.Crm.Tools.Email.Providers.EmailException: An error occurred while executing POP3 command "<Command removed for security reasons>". The server response was: "-ERR Invalid command.".
       at Microsoft.Crm.Tools.Email.Providers.Pop3Client.ValidateLine(String receivedLine, String command, Boolean hideCommand)
       at Microsoft.Crm.Tools.Email.Providers.Pop3Client.ReadDataWithEncoding(StreamReader streamReader, String command, Boolean singleCommand, Boolean skipValidation, Boolean hideCommand)
       at Microsoft.Crm.Tools.Email.Providers.Pop3Client.ReceiveData(String command, Boolean singleCommand, Boolean skipValidation, Boolean hideCommand, String& encodingCharset)
       at Microsoft.Crm.Tools.Email.Providers.Pop3Client.SendCommand(String command, Boolean singleCommand, Boolean skipValidation, Boolean hideCommand, String& encodingCharset)
       at Microsoft.Crm.Tools.Email.Providers.Pop3Client.SendCommand(String command, Boolean singleCommand, Boolean skipValidation, Boolean hideCommand)
       at Microsoft.Crm.Tools.Email.Providers.Pop3Client.TryAuthenticateNtlm(String userName, String password)
       at Microsoft.Crm.Tools.Email.Providers.Pop3Client.AuthenticateNtlm(String userName, String password)
       at Microsoft.Crm.Tools.Email.Providers.Pop3PollingMailboxProvider.OpenMailbox()
       at Microsoft.Crm.Tools.Email.Providers.CrmPollingMailboxProvider.Run()

     

    then this

    Host CRM: failed while monitoring asynchronous operations queue. Exception: System.InvalidOperationException: There is an error in XML document (1, 983). ---> System.Xml.XmlException: '', hexadecimal value 0x03, is an invalid character. Line 1, position 983.
       at System.Xml.XmlTextReaderImpl.Throw(Exception e)
       at System.Xml.XmlTextReaderImpl.Throw(String res, String[] args)
       at System.Xml.XmlTextReaderImpl.Throw(Int32 pos, String res, String[] args)
       at System.Xml.XmlTextReaderImpl.ThrowInvalidChar(Int32 pos, Char invChar)
       at System.Xml.XmlTextReaderImpl.ParseNumericCharRefInline(Int32 startPos, Boolean expand, BufferBuilder internalSubsetBuilder, Int32& charCount, EntityType& entityType)
       at System.Xml.XmlTextReaderImpl.ParseText(Int32& startPos, Int32& endPos, Int32& outOrChars)
       at System.Xml.XmlTextReaderImpl.ParseText()
       at System.Xml.XmlTextReaderImpl.ParseElementContent()
       at System.Xml.XmlTextReaderImpl.Read()
       at System.Xml.XmlTextReader.Read()
       at System.Xml.XmlReader.ReadElementString()
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read148_StringProperty(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read158_Property(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read134_DynamicEntity(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read133_BusinessEntity(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read168_BusinessEntityCollection(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read1505_GetUnprocessedRecordsResponse(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read1611_Response(Boolean isNullable, Boolean checkType)
       at Microsoft.Xml.Serialization.GeneratedAssembly.XmlSerializationReader1.Read1621_ExecuteResponse()
       at Microsoft.Xml.Serialization.GeneratedAssembly.ArrayOfObjectSerializer1.Deserialize(XmlSerializationReader reader)
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
       --- End of inner exception stack trace ---
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle, XmlDeserializationEvents events)
       at System.Xml.Serialization.XmlSerializer.Deserialize(XmlReader xmlReader, String encodingStyle)
       at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
       at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
       at Microsoft.Crm.SdkTypeProxy.CrmService.Execute(Request Request)
       at Microsoft.Crm.Asynchronous.SdkTypeProxyCrmServiceWrapper.Execute(Object request)
       at Microsoft.Crm.Asynchronous.PublishDuplicateRuleOperation.GetUnpublishedRecords(String entityName, String[] colSet, Guid ruleId, Int32 maxRecords, String[]& versionNumbers, ICrmService crmService)
       at Microsoft.Crm.Asynchronous.PublishDuplicateRuleOperation.PublishWithPaging(ShareableDuplicateRule rule, Boolean isMainEntity, AsyncEvent asyncEvent, DuplicateRuleDataAccess data, ICrmService crmService, IConfiguration configuration)
       at Microsoft.Crm.Asynchronous.PersistMatchCodeOperation.InternalExecute(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.AsyncOperationCommand.Execute(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.AsyncHostHandler.Handle(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.QueueManager.PoolHandler.ProcessAsyncEvent(AsyncEvent asyncEvent)
       at Microsoft.Crm.Asynchronous.QueueManager.PoolHandler.InvokeHandlerInPool(Object state)
       at System.Threading._ThreadPoolWaitCallback.WaitCallback_Context(Object state)
       at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
       at System.Threading._ThreadPoolWaitCallback.PerformWaitCallbackInternal(_ThreadPoolWaitCallback tpWaitCallBack)
       at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback(Object state)

     

    then finally this one

    The Microsoft CRM Asynchronous Processing Service service terminated unexpectedly.  It has done this 2 time(s).

     

     

    Any thoughts? sounds like we started having problems last Wednesday while I was away. Some workflows from last week are stating "waiting for resources" in the status reason column in system jobs.

     

     

    thanks

     


    Thanks, Scott Coates North America Traffic
    Monday, November 1, 2010 2:41 PM

Answers