none
Task Dependancies RRS feed

  • Question

  • Can anyone clarify the constraints around task dependancies?

    When running the following code:

    Scheduler

    scheduler = new Scheduler();

    scheduler.Connect(

    "HEADNODE");

     

    ISchedulerJob job = scheduler.CreateJob();

    job.MinimumNumberOfCores = 1;

    job.MaximumNumberOfCores = 20;

    job.Name =

    "Test a Job with task dependancies";

    job.Project =

    "Test Project";

    job.IsExclusive =

    false;

    job.RunUntilCanceled =

    false;

    job.Runtime = (

    int)TimeSpan.FromMinutes(5).TotalSeconds;

    job.Priority =

    JobPriority.Highest;

     

    IStringCollection names = scheduler.CreateStringCollection();

     

    for (int i = 0; i < 16; i++)

    {

     

    ISchedulerTask task = job.CreateTask();

    task.Name =

    "Long Running Task Name" + i.ToString();

    task.WorkDirectory =

    "C:\\Program Files\\Richard";

    task.CommandLine =

    "DeployedApplication.exe 1000";

    job.AddTask(task);

    names.Add(task.Name);

    }

     

    // Now create a load of dependant tasks that depend on the above set.

     

    for (int i = 0; i < 16; i++)

    {

     

    ISchedulerTask task = job.CreateTask();

    task.Name =

    "Short Running Task Name" + i.ToString();

    task.DependsOn = names;

    task.WorkDirectory =

    "C:\\Program Files\\Richard";

    task.CommandLine =

    "DeployedApplication.exe 100";

    job.AddTask(task);

    }

    scheduler.SubmitJob(job,

    "username", "password");

    we are gettingt the following exception:

    Microsoft.Hpc.Scheduler.Properties.SchedulerException was unhandled

    Message="The value for 'DependsOn' is too large for the database. The size should be smaller than 320."

    Source="Microsoft.Hpc.Scheduler.Store"

    Code=-2147220939

    Params="DependsOn|||320"

    StackTrace:

    at Microsoft.Hpc.Scheduler.Store.CallResult.Throw()

    at Microsoft.Hpc.Scheduler.Store.StoreServer.PostRemoteCall(CallResult cr)

    at Microsoft.Hpc.Scheduler.Store.StoreServer.Task_AddTaskToJob(Int32 jobId, Int32& taskId, StoreProperty[] taskProps)

    at Microsoft.Hpc.Scheduler.Store.JobEx.CreateTask(StoreProperty[] taskProperties)

    at Microsoft.Hpc.Scheduler.SchedulerTask.CreateTask(IClusterJob job, Int32 rootGroupId, Dictionary`2 taskGroupIdMapping)

    at Microsoft.Hpc.Scheduler.SchedulerJob.CreateJob()

    at Microsoft.Hpc.Scheduler.SchedulerJob.Submit(ISchedulerStore store, String username, String password)

    at Microsoft.Hpc.Scheduler.Scheduler.SubmitJob(ISchedulerJob job, String username, String password)

    at InitialTestOfHPc.Program.TestTaskDependancies() in C:\Development\InitialTestOfHPc\InitialTestOfHPc\Program.cs:line 66

    at InitialTestOfHPc.Program.Main(String[] args) in C:\Development\InitialTestOfHPc\InitialTestOfHPc\Program.cs:line 15

    at System.AppDomain._nExecuteAssembly(Assembly assembly, String[] args)

    at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)

    at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()

    at System.Threading.ThreadHelper.ThreadStart_Context(Object state)

    at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)

    at System.Threading.ThreadHelper.ThreadStart()

    Any ideas?

    Thanks

    Richard

    Tuesday, December 22, 2009 12:05 PM

Answers

  • Hi Richard,

    Yes, you've hit a known limitation in task dependencies for the current version (V2) of HPC Server. i.e. The list of task names in the DependsOn value cannot be longer than 320 chars. This is fixed in the latest V3 Beta1. For V2, there are a couple of possible of workarounds:

    (i) The dependencies created via the GUI (HpcCLusterManager) do not have this limitation because it ues groups of tasks, rather than task names. Therefore, you can use the GUI to create the dependencies. If you need to re-use this set of tasks and dependencies, you could export the job to XML and then re-import it when you need it.

    -or-

    (ii) When you need to make a task dependent on a large set of tasks, then you can give each of the tasks in the large set the same task name. The value of DependsOn will then just be a single task name, and this will make the task dependent on all tasks with that name.

    Regards,

    Patrick
    Tuesday, December 22, 2009 6:57 PM

All replies

  • Hi Richard,

    Yes, you've hit a known limitation in task dependencies for the current version (V2) of HPC Server. i.e. The list of task names in the DependsOn value cannot be longer than 320 chars. This is fixed in the latest V3 Beta1. For V2, there are a couple of possible of workarounds:

    (i) The dependencies created via the GUI (HpcCLusterManager) do not have this limitation because it ues groups of tasks, rather than task names. Therefore, you can use the GUI to create the dependencies. If you need to re-use this set of tasks and dependencies, you could export the job to XML and then re-import it when you need it.

    -or-

    (ii) When you need to make a task dependent on a large set of tasks, then you can give each of the tasks in the large set the same task name. The value of DependsOn will then just be a single task name, and this will make the task dependent on all tasks with that name.

    Regards,

    Patrick
    Tuesday, December 22, 2009 6:57 PM
  • This does not appear to be fixed (or at least the message is the same) in 2008 R2 (I assume that is the reference to V3 above).

    I have also tried your option (ii) by naming all the tasks the same - that did not work for me either (same error), I will continue to try various approaches to that method, but it has not looked good so far.

    What are my other options? I am creating a dynamic job based on the iteration of a list (call it a parametric sweap, but with filenames, not numbers) - so taking a previously exported Job, and modifying it is not real attractive.  I am working via code (CScript, not C#, but so far that has not been a big hinderance), so is there a way for me to Add the Job (scheduler.AddJob(Job)) and THEN export it (via code) to XML so that I could take advantage of the GroupID attributes in the XML?  Or are those facilities only (XML Export) only relevant to the GUI interface?

    Since I am effectively trying to get One task (a Summary task) to run when the other 1400 tasks are done - I COULD make a seperate job for it - but is there a way for me to make Job 2 dependant on the completion of Job 1???

     

    Wednesday, October 27, 2010 9:42 PM