locked
workflows refusing to start RRS feed

  • Question

  • Hi

    i have a workflow that starts and sends mails  when a record is created and has been working fine all this while. Today i just noticed that the workflows have refused to start. All the services have started and the email router is working. i also tested other system processes like a bulk delete and that also didnt start. please what could be the issue

    Help as this is very important 


    System integrator

    Tuesday, January 29, 2013 9:17 AM

All replies

  • I'd start by enabling tracing and checking the AsyncService log file

    Microsoft CRM MVP - http://mscrmuk.blogspot.com/ http://www.excitation.co.uk

    Tuesday, January 29, 2013 9:31 AM
    Moderator
  • Hi David

    This is a part of the result of the AsyncService log file. I noticed that the Data source value(in BOLD font) is referencing an instance of MSSQL that is not in use.This could be the problem

    [2013-01-29 11:31:58.331] Process:CrmAsyncService |Organization:eb064e0e-3618-e211-a01e-001a4b4f00f5 |Thread:   24 |Category: Platform.Async |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DataAccessBase.ExecuteSqlCommandAndProcessRecords
    >DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b: Exception while trying to open the database connection Workstation ID=DYNAMICS.CrmAsyncService;Data Source=DYNAMICS\SQL2;Initial Catalog=CustomerContactSolutions_MSCRM;Integrated Security=True;Application Name=MSCRMAsyncService : System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
       at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity)
       at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, SqlConnection owningObject)
       at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, TimeoutTimer timeout)
       at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, TimeoutTimer timeout, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance)
       at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance)
       at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection)
       at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options)
       at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
       at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
       at System.Data.SqlClient.SqlConnection.Open()
       at Microsoft.Crm.CrmDbConnection.Open()
       at Microsoft.Crm.Asynchronous.DataAccessBase.ExecuteSqlCommandAndProcessRecords(IDbCommand command, RecordProcessor recordProcessor)


    System integrator

    Tuesday, January 29, 2013 10:50 AM
  • Hi,

    Look at the registry key:

    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSCRM

    name: configdb
    Stop the async services, and change the datasource to the correct server and start again the async services.


    Visit my blog for CRM material, improving performance, kerberos, IFD, development tips, etc. :) http://quantusdynamics.blogspot.com

    Tuesday, January 29, 2013 10:52 AM
    Answerer
  • Hi

    The registry key configdb value is pointing to the correct datasource but the trace log is reporting another value


    System integrator

    Tuesday, January 29, 2013 11:11 AM
  • in that case it could be in your MSCRM_CONFIG database the wrong datasource.

    if you look at:

    select * from Organization
    on the CONFIG database, is it also correct in there?


    Visit my blog for CRM material, improving performance, kerberos, IFD, development tips, etc. :) http://quantusdynamics.blogspot.com

    Tuesday, January 29, 2013 11:41 AM
    Answerer
  • hi

    Ran the query and the data source is wrong. here is the result it shows.

    Provider=SQLOLEDB;Data Source=DYNAMICS\SQL2;Initial Catalog=CustomerContactSolutions_MSCRM;Integrated Security=SSPI

    how do i change it to the right data source?


    System integrator

    Tuesday, January 29, 2013 12:08 PM
  • This means you have probably restored the databases without importing it, on deployment manager, disable the organization CustomerContactSolutions_MSCRM and then click EDIT, update the sql server (the sql server here should be incorrect), then re-enable, it's possible that this may give you an error, which could be related with the OptimizeOrgImport, you may need to set this to 1.

    Check again the same CONFIG on the database and if still has not updated the data source, you could do a manual update.

    update organization set ConnectionString = 'Copy and paste and change the data source'
    Where DatabaseName = 'CustomerContactSolutions_MSCRM'

    It's possible that the SQL server is correct, if that's the case and everything else is working you could do the SQL update.


    Visit my blog for CRM material, improving performance, kerberos, IFD, development tips, etc. :) http://quantusdynamics.blogspot.com



    Tuesday, January 29, 2013 12:21 PM
    Answerer
  • tried it and its still giving me the same error.

    [2013-01-29 14:02:05.030] Process:CrmAsyncService |Organization:e0a9a038-6923-e211-92f5-001a4b4f00f5 |Thread:   35 |Category: Platform.Async |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DataAccessSharedBase.ExecuteSqlCommand
    >DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b: Exception while trying to execute query:
    SET DEADLOCK_PRIORITY LOW
    BEGIN TRANSACTION
        DECLARE @res INT
        EXEC @res = sp_getapplock                
                    @Resource = 'AsyncService_app_lock',
                    @LockMode = 'Exclusive'
        IF @res NOT IN (0, 1)
        BEGIN
            RAISERROR ( 'Unable to acquire App Lock', 16, 1 )
        END
        ELSE
        BEGIN

    With TopSuspendedAsyncOperations(AsyncOperationId)
    AS
    (
    Select Top(@numberOfSuspendedOperationsToSelect) AsyncOperationId
    from AsyncOperationBase o1
    where
     StateCode = @suspendedState
    and ((PostponeUntil is NULL) or (PostponeUntil <= @modifiedOn))
    and ((DependencyToken is NULL) or (not exists (
     select AsyncOperationId
     from AsyncOperationBase o2
     where
      o2.Sequence < o1.Sequence
     and o2.DependencyToken = o1.DependencyToken
     and o2.RecurrencePattern is null
     and o2.StateCode in (@readyState,@lockedState,@suspendedState))))
    )
    update AsyncOperationBase
    set
     StateCode = @readyState,
     StatusCode = @waitingStatus,
     ModifiedOn = @modifiedOn,
     ModifiedBy = CreatedBy
    From AsyncOperationBase join TopSuspendedAsyncOperations on
    AsyncOperationBase.AsyncOperationId = TopSuspendedAsyncOperations.AsyncOperationId

    EXEC @res = sp_releaseapplock
                                @Resource = 'AsyncService_app_lock'
        END   
    COMMIT - System.Data.SqlClient.SqlException (0x80131904): Internal Query Processor Error: The query processor encountered an unexpected error during execution.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.Crm.CrmDbConnection.InternalExecuteWithRetry[TResult](Func`1 ExecuteMethod, IDbCommand command)
       at Microsoft.Crm.CrmDbConnection.InternalExecuteNonQuery(IDbCommand command, Boolean capturePerfTrace)
       at Microsoft.Crm.CrmDbConnection.ExecuteNonQuery(IDbCommand command, Boolean impersonate, Boolean capturePerfTrace)
       at Microsoft.Crm.Asynchronous.DataAccessSharedBase.ExecuteSqlCommand(IDbCommand command, Nullable`1 expectedAffectedRecords)
    [2013-01-29 14:02:09.421] Process:CrmAsyncService |Organization:d89be16a-5405-4d66-9729-3f7d40886c62 |Thread:    5 |Category: Platform.Async |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DataAccessBase.ExecuteSqlCommandAndProcessRecords
    >DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b: Exception while trying to execute query:
    SET DEADLOCK_PRIORITY LOW
    BEGIN TRANSACTION
        DECLARE @res INT
        EXEC @res = sp_getapplock                
                    @Resource = 'AsyncService_app_lock',
                    @LockMode = 'Exclusive'
        IF @res NOT IN (0, 1)
        BEGIN
            RAISERROR ( 'Unable to acquire App Lock', 16, 1 )
        END
        ELSE
        BEGIN

    update AsyncOperationBase
    set
     HostId = 'DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b',
     StateCode = 2,
     StatusCode = 20,
     StartedOn = '01/29/2013 13:02:07',
     ModifiedOn = '01/29/2013 13:02:07',
     ModifiedBy = CreatedBy,
     RetryCount = case coalesce(ErrorCode, 0)
         when 0 then coalesce(RetryCount, 0)
         else coalesce(RetryCount, 0)+1
         end
    output
     INSERTED.AsyncOperationId,
     INSERTED.OperationType,
     INSERTED.Data,
     INSERTED.DependencyToken,
     INSERTED.RegardingObjectId,
     INSERTED.RegardingObjectIdName,
     INSERTED.RegardingObjectTypeCode,
     INSERTED.OwningBusinessUnit,
     INSERTED.OwningExtensionId,
     INSERTED.OwningExtensionIdName,
     INSERTED.OwningExtensionTypeCode,
     INSERTED.OwnerId,
     INSERTED.RetryCount,
     INSERTED.MessageName,
     INSERTED.RecurrencePattern,
     INSERTED.RecurrenceStartTime,
     INSERTED.CorrelationId,
     INSERTED.CorrelationUpdatedTime,
     INSERTED.Depth,
     INSERTED.PrimaryEntityType,
     INSERTED.RequestId,
     INSERTED.CreatedOn,
     INSERTED.ExecutionTimeSpan
    where
    AsyncOperationId in
     (select top(1) AsyncOperationId from AsyncOperationBase WITH (READPAST, READCOMMITTEDLOCK, UPDLOCK)
      where StateCode = 0
      and OperationType IS NOT null
      order by Sequence asc)
    EXEC @res = sp_releaseapplock
                                @Resource = 'AsyncService_app_lock'
        END   
    COMMIT - System.Data.SqlClient.SqlException (0x80131904): Internal error. The string routine in file e:\sql10_main_t\sql\ntdbms\storeng\dfs\startup\dstream.cpp, line 792 failed with HRESULT 0x8007007a.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
       at System.Data.SqlClient.SqlDataReader.HasMoreRows()
       at System.Data.SqlClient.SqlDataReader.ReadInternal(Boolean setTimeout)
       at Microsoft.Crm.CrmDataReader.Read()
       at Microsoft.Crm.Asynchronous.DataAccessBase.ExecuteSqlCommandAndProcessRecords(IDbCommand command, RecordProcessor recordProcessor)
    [2013-01-29 14:02:09.421] Process:CrmAsyncService |Organization:d89be16a-5405-4d66-9729-3f7d40886c62 |Thread:    5 |Category: Platform.Workflow |User: 00000000-0000-0000-0000-000000000000 |Level: Error | AsyncEventLogHelper.LogOrganizationFailure
    >Received SQL Exception accessing organization {D89BE16A-5405-4D66-9729-3F7D40886C62}. Exception: System.Data.SqlClient.SqlException (0x80131904): Internal error. The string routine in file e:\sql10_main_t\sql\ntdbms\storeng\dfs\startup\dstream.cpp, line 792 failed with HRESULT 0x8007007a.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
       at System.Data.SqlClient.SqlDataReader.HasMoreRows()
       at System.Data.SqlClient.SqlDataReader.ReadInternal(Boolean setTimeout)
       at Microsoft.Crm.CrmDataReader.Read()
       at Microsoft.Crm.Asynchronous.DataAccessBase.ExecuteSqlCommandAndProcessRecords(IDbCommand command, RecordProcessor recordProcessor)
       at Microsoft.Crm.Asynchronous.QueueDataAccess.SelectEvents(Int32 numberOfEvents)
    [2013-01-29 14:02:19.577] Process:CrmAsyncService |Organization:eb064e0e-3618-e211-a01e-001a4b4f00f5 |Thread:   35 |Category: Platform.Async |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DataAccessSharedBase.ExecuteSqlCommand
    >DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b: Exception while trying to open the database connection Workstation ID=DYNAMICS.CrmAsyncService;Data Source=DYNAMICS\SQL2;Initial Catalog=CustomerContactSolutions_MSCRM;Integrated Security=True;Application Name=MSCRMAsyncService : System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
       at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity)
       at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, SqlConnection owningObject)
       at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, TimeoutTimer timeout)
       at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, TimeoutTimer timeout, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance)
       at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance)
       at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection)
       at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options)
       at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
       at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
       at System.Data.SqlClient.SqlConnection.Open()
       at Microsoft.Crm.CrmDbConnection.Open()
       at Microsoft.Crm.Asynchronous.DataAccessSharedBase.ExecuteSqlCommand(IDbCommand command, Nullable`1 expectedAffectedRecords)
    [2013-01-29 14:02:19.577] Process:CrmAsyncService |Organization:eb064e0e-3618-e211-a01e-001a4b4f00f5 |Thread:   38 |Category: Platform.Async |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DataAccessBase.ExecuteSqlCommandAndProcessRecords
    >DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b: Exception while trying to open the database connection Workstation ID=DYNAMICS.CrmAsyncService;Data Source=DYNAMICS\SQL2;Initial Catalog=CustomerContactSolutions_MSCRM;Integrated Security=True;Application Name=MSCRMAsyncService : System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
       at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
       at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
       at System.Data.SqlClient.SqlConnection.Open()
       at Microsoft.Crm.CrmDbConnection.Open()
       at Microsoft.Crm.Asynchronous.DataAccessBase.ExecuteSqlCommandAndProcessRecords(IDbCommand command, RecordProcessor recordProcessor)
    [2013-01-29 14:02:25.233] Process:CrmAsyncService |Organization:e0a9a038-6923-e211-92f5-001a4b4f00f5 |Thread:   38 |Category: Platform.Async |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DataAccessSharedBase.ExecuteSqlCommand
    >DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b: Exception while trying to execute query:
    SET DEADLOCK_PRIORITY LOW
    BEGIN TRANSACTION
        DECLARE @res INT
        EXEC @res = sp_getapplock                
                    @Resource = 'AsyncService_app_lock',
                    @LockMode = 'Exclusive'
        IF @res NOT IN (0, 1)
        BEGIN
            RAISERROR ( 'Unable to acquire App Lock', 16, 1 )
        END
        ELSE
        BEGIN

    With TopSuspendedAsyncOperations(AsyncOperationId)
    AS
    (
    Select Top(@numberOfSuspendedOperationsToSelect) AsyncOperationId
    from AsyncOperationBase o1
    where
     StateCode = @suspendedState
    and ((PostponeUntil is NULL) or (PostponeUntil <= @modifiedOn))
    and ((DependencyToken is NULL) or (not exists (
     select AsyncOperationId
     from AsyncOperationBase o2
     where
      o2.Sequence < o1.Sequence
     and o2.DependencyToken = o1.DependencyToken
     and o2.RecurrencePattern is null
     and o2.StateCode in (@readyState,@lockedState,@suspendedState))))
    )
    update AsyncOperationBase
    set
     StateCode = @readyState,
     StatusCode = @waitingStatus,
     ModifiedOn = @modifiedOn,
     ModifiedBy = CreatedBy
    From AsyncOperationBase join TopSuspendedAsyncOperations on
    AsyncOperationBase.AsyncOperationId = TopSuspendedAsyncOperations.AsyncOperationId

    EXEC @res = sp_releaseapplock
                                @Resource = 'AsyncService_app_lock'
        END   
    COMMIT - System.Data.SqlClient.SqlException (0x80131904): Internal Query Processor Error: The query processor encountered an unexpected error during execution.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.Crm.CrmDbConnection.InternalExecuteWithRetry[TResult](Func`1 ExecuteMethod, IDbCommand command)
       at Microsoft.Crm.CrmDbConnection.InternalExecuteNonQuery(IDbCommand command, Boolean capturePerfTrace)
       at Microsoft.Crm.CrmDbConnection.ExecuteNonQuery(IDbCommand command, Boolean impersonate, Boolean capturePerfTrace)
       at Microsoft.Crm.Asynchronous.DataAccessSharedBase.ExecuteSqlCommand(IDbCommand command, Nullable`1 expectedAffectedRecords)
    [2013-01-29 14:02:25.233] Process:CrmAsyncService |Organization:e0a9a038-6923-e211-92f5-001a4b4f00f5 |Thread:   38 |Category: Platform.Workflow |User: 00000000-0000-0000-0000-000000000000 |Level: Error | AsyncEventLogHelper.LogOrganizationFailure
    >Received SQL Exception accessing organization {E0A9A038-6923-E211-92F5-001A4B4F00F5}. Exception: System.Data.SqlClient.SqlException (0x80131904): Internal Query Processor Error: The query processor encountered an unexpected error during execution.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.Crm.CrmDbConnection.InternalExecuteWithRetry[TResult](Func`1 ExecuteMethod, IDbCommand command)
       at Microsoft.Crm.CrmDbConnection.InternalExecuteNonQuery(IDbCommand command, Boolean capturePerfTrace)
       at Microsoft.Crm.CrmDbConnection.ExecuteNonQuery(IDbCommand command, Boolean impersonate, Boolean capturePerfTrace)
       at Microsoft.Crm.Asynchronous.DataAccessSharedBase.ExecuteSqlCommand(IDbCommand command, Nullable`1 expectedAffectedRecords)
       at Microsoft.Crm.Asynchronous.QueueDataAccess.MoveSuspendedToReady()
       at Microsoft.Crm.Asynchronous.QueueManager.<>c__DisplayClassf.<OnMoveSuspendedToReadyTimerEvent>b__e(IQueueDataAccess queueDataAccess, IOrganizationConfiguration orgConfig)
       at Microsoft.Crm.Asynchronous.QueueManager.ExecuteForAllOrganizations(OrganizationQueueDataAccessHandler handler)
    [2013-01-29 14:02:33.420] Process:CrmAsyncService |Organization:eb064e0e-3618-e211-a01e-001a4b4f00f5 |Thread:   23 |Category: Platform.Async |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DataAccessSharedBase.ExecuteSqlCommand
    >DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b: Exception while trying to open the database connection Workstation ID=DYNAMICS.CrmAsyncService;Data Source=DYNAMICS\SQL2;Initial Catalog=CustomerContactSolutions_MSCRM;Integrated Security=True;Application Name=MSCRMAsyncService : System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
       at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Connect(ServerInfo serverInfo, SqlInternalConnectionTds connHandler, Boolean ignoreSniOpenTimeout, Int64 timerExpire, Boolean encrypt, Boolean trustServerCert, Boolean integratedSecurity)
       at System.Data.SqlClient.SqlInternalConnectionTds.AttemptOneLogin(ServerInfo serverInfo, String newPassword, Boolean ignoreSniOpenTimeout, TimeoutTimer timeout, SqlConnection owningObject)
       at System.Data.SqlClient.SqlInternalConnectionTds.LoginNoFailover(ServerInfo serverInfo, String newPassword, Boolean redirectedUserInstance, SqlConnection owningObject, SqlConnectionString connectionOptions, TimeoutTimer timeout)
       at System.Data.SqlClient.SqlInternalConnectionTds.OpenLoginEnlist(SqlConnection owningObject, TimeoutTimer timeout, SqlConnectionString connectionOptions, String newPassword, Boolean redirectedUserInstance)
       at System.Data.SqlClient.SqlInternalConnectionTds..ctor(DbConnectionPoolIdentity identity, SqlConnectionString connectionOptions, Object providerInfo, String newPassword, SqlConnection owningObject, Boolean redirectedUserInstance)
       at System.Data.SqlClient.SqlConnectionFactory.CreateConnection(DbConnectionOptions options, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection)
       at System.Data.ProviderBase.DbConnectionFactory.CreatePooledConnection(DbConnection owningConnection, DbConnectionPool pool, DbConnectionOptions options)
       at System.Data.ProviderBase.DbConnectionPool.CreateObject(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionPool.UserCreateRequest(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
       at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
       at System.Data.SqlClient.SqlConnection.Open()
       at Microsoft.Crm.CrmDbConnection.Open()
       at Microsoft.Crm.Asynchronous.DataAccessSharedBase.ExecuteSqlCommand(IDbCommand command, Nullable`1 expectedAffectedRecords)
    [2013-01-29 14:02:33.420] Process:CrmAsyncService |Organization:eb064e0e-3618-e211-a01e-001a4b4f00f5 |Thread:   38 |Category: Platform.Async |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DataAccessSharedBase.ExecuteSqlCommand
    >DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b: Exception while trying to open the database connection Workstation ID=DYNAMICS.CrmAsyncService;Data Source=DYNAMICS\SQL2;Initial Catalog=CustomerContactSolutions_MSCRM;Integrated Security=True;Application Name=MSCRMAsyncService : System.Data.SqlClient.SqlException (0x80131904): A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
       at System.Data.ProviderBase.DbConnectionPool.GetConnection(DbConnection owningObject)
       at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
       at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
       at System.Data.SqlClient.SqlConnection.Open()
       at Microsoft.Crm.CrmDbConnection.Open()
       at Microsoft.Crm.Asynchronous.DataAccessSharedBase.ExecuteSqlCommand(IDbCommand command, Nullable`1 expectedAffectedRecords)
    [2013-01-29 14:02:45.076] Process:CrmAsyncService |Organization:e0a9a038-6923-e211-92f5-001a4b4f00f5 |Thread:   45 |Category: Platform.Async |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DataAccessSharedBase.ExecuteSqlCommand
    >DYNAMICS.MSCRMAsyncService.4b90fd0d-f8df-4a1d-be2b-39208f14578b: Exception while trying to execute query:
    SET DEADLOCK_PRIORITY LOW
    BEGIN TRANSACTION
        DECLARE @res INT
        EXEC @res = sp_getapplock                
                    @Resource = 'AsyncService_app_lock',
                    @LockMode = 'Exclusive'
        IF @res NOT IN (0, 1)
        BEGIN
            RAISERROR ( 'Unable to acquire App Lock', 16, 1 )
        END
        ELSE
        BEGIN

    With TopSuspendedAsyncOperations(AsyncOperationId)
    AS
    (
    Select Top(@numberOfSuspendedOperationsToSelect) AsyncOperationId
    from AsyncOperationBase o1
    where
     StateCode = @suspendedState
    and ((PostponeUntil is NULL) or (PostponeUntil <= @modifiedOn))
    and ((DependencyToken is NULL) or (not exists (
     select AsyncOperationId
     from AsyncOperationBase o2
     where
      o2.Sequence < o1.Sequence
     and o2.DependencyToken = o1.DependencyToken
     and o2.RecurrencePattern is null
     and o2.StateCode in (@readyState,@lockedState,@suspendedState))))
    )
    update AsyncOperationBase
    set
     StateCode = @readyState,
     StatusCode = @waitingStatus,
     ModifiedOn = @modifiedOn,
     ModifiedBy = CreatedBy
    From AsyncOperationBase join TopSuspendedAsyncOperations on
    AsyncOperationBase.AsyncOperationId = TopSuspendedAsyncOperations.AsyncOperationId

    EXEC @res = sp_releaseapplock
                                @Resource = 'AsyncService_app_lock'
        END   
    COMMIT - System.Data.SqlClient.SqlException (0x80131904): Internal Query Processor Error: The query processor encountered an unexpected error during execution.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.Crm.CrmDbConnection.InternalExecuteWithRetry[TResult](Func`1 ExecuteMethod, IDbCommand command)
       at Microsoft.Crm.CrmDbConnection.InternalExecuteNonQuery(IDbCommand command, Boolean capturePerfTrace)
       at Microsoft.Crm.CrmDbConnection.ExecuteNonQuery(IDbCommand command, Boolean impersonate, Boolean capturePerfTrace)
       at Microsoft.Crm.Asynchronous.DataAccessSharedBase.ExecuteSqlCommand(IDbCommand command, Nullable`1 expectedAffectedRecords)


    System integrator

    Tuesday, January 29, 2013 1:14 PM
  • Are you trying to resume the same workflow or this are new ones? if is the same one, then you may want to cancel that and run another one, as workflows hold static information when they are run, but not sure if the ConnectionString would be one of those.

    Which updated did you do? sql udpate or deployment manager or both? btw the configdb on the server should also point at the same ConnectionString.


    Visit my blog for CRM material, improving performance, kerberos, IFD, development tips, etc. :) http://quantusdynamics.blogspot.com

    Tuesday, January 29, 2013 1:35 PM
    Answerer
  • i am trying to run new ones as non has ran before. also i did both updates.

    System integrator

    Tuesday, January 29, 2013 2:03 PM
  • Hi i am still stuck with this issue on workflows not been able to start. Here is a the details from the event log file.

    Log Name:      Application
    Source:        MSCRMAsyncService
    Date:          1/30/2013 12:12:29 PM
    Event ID:      17415
    Task Category: None
    Level:         Error
    Keywords:      Classic
    User:          N/A
    Computer:      xxxxxxxxxxxxxx
    Description:
    Host DYNAMICS: error while processing organization e0a9a038-6923-e211-92f5-001a4b4f00f5. Exception: System.Data.SqlClient.SqlException (0x80131904): Internal Query Processor Error: The query processor encountered an unexpected error during execution.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.Crm.CrmDbConnection.InternalExecuteWithRetry[TResult](Func`1 ExecuteMethod, IDbCommand command)
       at Microsoft.Crm.CrmDbConnection.InternalExecuteNonQuery(IDbCommand command, Boolean capturePerfTrace)
       at Microsoft.Crm.CrmDbConnection.ExecuteNonQuery(IDbCommand command, Boolean impersonate, Boolean capturePerfTrace)
       at Microsoft.Crm.Asynchronous.DataAccessSharedBase.ExecuteSqlCommand(IDbCommand command, Nullable`1 expectedAffectedRecords)
       at Microsoft.Crm.Asynchronous.QueueDataAccess.MoveSuspendedToReady()
       at Microsoft.Crm.Asynchronous.QueueManager.<>c__DisplayClassf.<OnMoveSuspendedToReadyTimerEvent>b__e(IQueueDataAccess queueDataAccess, IOrganizationConfiguration orgConfig)
       at Microsoft.Crm.Asynchronous.QueueManager.ExecuteForAllOrganizations(OrganizationQueueDataAccessHandler handler)
    Event Xml:
    <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
      <System>
        <Provider Name="MSCRMAsyncService" />
        <EventID Qualifiers="49152">17415</EventID>
        <Level>2</Level>
        <Task>0</Task>
        <Keywords>0x80000000000000</Keywords>
        <TimeCreated SystemTime="2013-01-30T11:12:29.000000000Z" />
        <EventRecordID>3058534</EventRecordID>
        <Channel>Application</Channel>
        <Computer>xxxxxxxxxxx</Computer>
        <Security />
      </System>
      <EventData>
        <Data>DYNAMICS</Data>
        <Data>e0a9a038-6923-e211-92f5-001a4b4f00f5</Data>
        <Data>System.Data.SqlClient.SqlException (0x80131904): Internal Query Processor Error: The query processor encountered an unexpected error during execution.
       at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
       at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
       at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
       at System.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString)
       at System.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean async)
       at System.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, String method, DbAsyncResult result)
       at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
       at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
       at Microsoft.Crm.CrmDbConnection.InternalExecuteWithRetry[TResult](Func`1 ExecuteMethod, IDbCommand command)
       at Microsoft.Crm.CrmDbConnection.InternalExecuteNonQuery(IDbCommand command, Boolean capturePerfTrace)
       at Microsoft.Crm.CrmDbConnection.ExecuteNonQuery(IDbCommand command, Boolean impersonate, Boolean capturePerfTrace)
       at Microsoft.Crm.Asynchronous.DataAccessSharedBase.ExecuteSqlCommand(IDbCommand command, Nullable`1 expectedAffectedRecords)
       at Microsoft.Crm.Asynchronous.QueueDataAccess.MoveSuspendedToReady()
       at Microsoft.Crm.Asynchronous.QueueManager.&lt;&gt;c__DisplayClassf.&lt;OnMoveSuspendedToReadyTimerEvent&gt;b__e(IQueueDataAccess queueDataAccess, IOrganizationConfiguration orgConfig)
       at Microsoft.Crm.Asynchronous.QueueManager.ExecuteForAllOrganizations(OrganizationQueueDataAccessHandler handler)</Data>
      </EventData>
    </Event>


    System integrator

    Wednesday, January 30, 2013 12:32 PM