locked
Intermittent error on IScheduler.SubmitJob - The operation could not be completed because the affected object is already in use by the scheduler RRS feed

  • Question

  • I get the following exception intermittently every few days when submitting jobs.

    Microsoft.Hpc.Scheduler.Properties.SchedulerException: The operation could not be completed because the affected object is already in use by the scheduler.  Please try again later.
       at Microsoft.Hpc.Scheduler.Store.CallResult.Throw()
       at Microsoft.Hpc.Scheduler.Store.JobEx.BeginSubmit(StoreProperty[] props, AsyncCallback callback, Object param)
       at Microsoft.Hpc.Scheduler.Store.JobEx.Submit(StoreProperty[] submitProps)
       at Microsoft.Hpc.Scheduler.Scheduler.SubmitJob(ISchedulerJob job, String username, String password)

    HPC event logs at the time show this:

    The scheduler was unable to commit a transaction.

    Exception detail: Microsoft.Hpc.Scheduler.Store.OptimisticLockViolationException: Expected to update 1 rows, but actually updated 0, for SQL command:

    An unexpected exception occurred. For more information about this exception, see the Details tab. 

     Additional data:
     Expected to update 1 rows, but actually updated 0, for SQL command:
    SET NOCOUNT ON;
    SET NOCOUNT OFF;
    SET NOCOUNT OFF;
    UPDATE Job SET
    SubmitTime = @Param0,ChangeTime = @Param1,State = @Param2,PrevState = @Param3,CredentialId = @Param4,IsAuthenticated = @Param5
    WHERE ID = 106472 AND timestamp <= 0x000000000ACCF4ED
    SET NOCOUNT ON;

    HPC Pack 2012 R2 Server version 4.2.4400.0

    Should I catch and retry when I see this exception?

    Friday, April 22, 2016 8:21 AM

Answers

  • Thanks for reporting the issue.

    This exception usually means the object you've created was updated by the scheduler service between the client calls.

    You can just catch this exception and retry the operation.

    Let me know if you have further questions.

    • Marked as answer by TimJRoberts1 Tuesday, May 17, 2016 3:10 PM
    Monday, April 25, 2016 9:30 AM

All replies

  • Thanks for reporting the issue.

    This exception usually means the object you've created was updated by the scheduler service between the client calls.

    You can just catch this exception and retry the operation.

    Let me know if you have further questions.

    • Marked as answer by TimJRoberts1 Tuesday, May 17, 2016 3:10 PM
    Monday, April 25, 2016 9:30 AM
  • OK thanks do you know a specific error code within SchedulerException that is safe to retry?

    If it is an internal exception that can be retried, can it be fixed in HPC itself?

    Tuesday, April 26, 2016 9:00 AM
  • Hi Jim,

        I've logged a work item for this issue. We will investigate and try to fix it in our side.


    Qiufang Shi

    Thursday, April 28, 2016 11:28 AM