Sign in
Microsoft.com
 
Microsoft
 
 
 

locked
Job Priorities RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote
    I am testing the behaviour of different job priorities but I am not getting the expected results. The scenario is:

    2 Compute Nodes, each with 8 cores.
    2 Jobs, each with 10 tasks. First job submitted as "Lowest" priority and second as "Highest" priority. MaximumNumberOfCores = 4.

    I am expecting the first four tasks of the "Lowest" priority job to start executing because there are 16 cores available = 4 tasks * 4 cores. Given that the "Highest" priority job is submitted directly after the "Lowest" but there are no available cores, all of its tasks stay as queued as expected - all good so far.

    Next, I would expected the scheduler to let those first four "Lowest" priority tasks complete because the "Pre-emption" configuration is set to "Graceful". However once complete, I would expect all the "Highest" priority tasks to run to completion before any more "Lowest" priority tasks are run. But this is not what I see. Only one of the "Highest" priority tasks runs (for whatever reason) and the rest of the "Lowest" priority tasks are run instead.

    Can anyone explain this behaviour please?

    Monday, January 4, 2010 9:53 AM

Answers

  • Question
    Sign in to vote
    0
    Sign in to vote
    This is the expected behavior. The highest priority job will start as soon as possible with it's minimum resource request. Once it has started it the lower priority jobs that are running will continue to run with the resources they currently are using.

    What I think you are expecting is "shrink to grow" which we don't support. Once the higher priority job starts the currently running jobs will be allowed to continue to run with the resources they have allocated when the higher priority job starts. They will not shrink further.
    • Proposed as answer by Steve Chilcoat Thursday, January 7, 2010 9:04 PM
    • Marked as answer by Simon Place Friday, January 8, 2010 10:52 AM
    Thursday, January 7, 2010 8:57 PM

All replies

  • Question
    Sign in to vote
    0
    Sign in to vote
    Hi Simon,

    can you provide what's the minimum number of cores for each of your jobs?

    liwei
    Monday, January 4, 2010 7:13 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    Hi Simon,

    You've mentioned that the MaximumNumberOfCores=4, but I'm not sure you're specifying that for each Task or for each Job. It would be great if you could provide some more details on your jobs and tasks, such as the job files (.xml) for each job, and then I'm sure we can better explain what is happening.

    Regards,

    Patrick
    Monday, January 4, 2010 7:24 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi

    Sorry about the lack of detail. The jobs/tasks are submitted via the API thus:

    The job is created first:

    m_Scheduler = new Scheduler();
    m_Scheduler.Connect(headnode);
    m_Scheduler.SetInterfaceMode(false, (IntPtr)null);

    m_Job = m_Scheduler.CreateJob();
    m_Job.Name = jobName;
    m_Job.UnitType = JobUnitType.Core;
    m_Job.SetJobTemplate(simulationType.ToString());

    Then a bunch of tasks are created in a loop:

    ISchedulerTask task = m_Job.CreateTask();
    task.Name = simulationJob.Name;
    task.CommandLine = "{commandline}";
    task.WorkDirectory = @"C:\Program Files (x86)\Simon\BECMConsoleApp";
    task.MaximumNumberOfCores = 4;
    m_Job.AddTask(task);

    And finally the job is submitted:

    m_Scheduler.SubmitJob(m_Job, null, null);

    So very few properties are actually set.

    Thanks for your help.

    Simon

    Tuesday, January 5, 2010 9:21 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    I should probably also make clear the fact that the priority is set via the job template which is specified in code above. The priority is the only property which is currently set within the job templates.

    Thanks
    Simon
    Tuesday, January 5, 2010 9:32 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    The MaximumNumberOfCores setting is actually there as a bit of a fudge. The data that I had was actually constraining me to a maximum of 10 tasks and so I was trying to ensure that not all of the tasks from the "Lowest" priority job could start at once. I have now increased the number of tasks on each job to 30 and have removed the MaximumNumberOfCores setting. Hence the defaults result in Auto allocation at the job level and Min-Max Cores of 1-1 at the task level. This matches my actual intent. However I am still seeing the same result where the first 16 "Lowest" priority tasks run first but then the final 14 "Lowest" priority tasks seem to get priority over the queued "Highest" priority tasks.

    Thanks
    Simon
    Tuesday, January 5, 2010 10:23 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    Hi Simon,

    OK, thanks. If the second job does get a higher priority than the first, then you should see (as you are expecting) the second job to be scheduled while there are still queued tasks in the first job. However, if both jobs are created with the same template and the same same priority, then what you are seeing is the expected behavior, where the first job will run to completion before the second job begins.

    It really help us understand exactly what's happening if we could see the job xml for each job. After you've run the two jobs, could you please export each job to a file and post the two xml fles here for us to look at?

    Regards,

    Patrick
    Tuesday, January 5, 2010 5:42 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi Patrick

    It would seem that the prioritisation algorithm is not giving much weighting to the priority property. I have now changed the job templates (Highest and Lowest) so that the MinCores-MaxCores settings are both 1-16 and only the priority differs. In this scenario only, the Highest priority job only gets one core whilst the Lowest priority job merrily consumes the rest. It seems as though submission time is carrying more weighting here.

    If I change the core allocation on the Highest priority template to 16-16 then this will draw all resources leaving the Lowest priority tasks queued until such time as the Highest priority ones have all completed. A MinCores setting somewhere between 1 and 16 will allow the Lowest priority job to be serviced to a lesser or greater extent. However achieving the desired outcome via resource requirement seems to negate the whole point of actual job priority which now plays little or no part. It also introduces an extra maintenance overhead since the templates now have to be changed in line with the cluster profile. Allowing Min/Max to be specified increases flexibility but I was expecting submission time and priority alone to produce my expected behaviour. I am also worried that having to maintain many template properties within a cluster supporting several hetereogenous applications may prove difficult.

    Is Job Priority really a poor cousin to other settings or am I still missing something?

    Thanks
    Simon

    P.S I cannot paste all the tasks here since the thread will not support this message size. Hopefully those that remain are sufficient for what you need.

    Lowest:

    <?xml version="1.0" encoding="utf-8"?>
<Job Version="2.000" Id="544" Name="Simulation run -1 for configuration 648" SubmitTime="07/01/2010 15:50:50" CreateTime="07/01/2010 15:50:49" StartTime="07/01/2010 15:50:50" EndTime="07/01/2010 15:53:03" ChangeTime="07/01/2010 15:50:49" UnitType="Core" MinCores="1" MaxCores="16" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" RunUntilCanceled="false" IsExclusive="false" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" UserName="TFS\pt00834" JobType="Batch" Priority="Lowest" RequiredNodes="" IsBackfill="false" NextTaskNiceID="31" HasGrown="false" HasShrunk="true" OrderBy="" TaskLevelUpdateTime="07/01/2010 15:52:50" MinMaxUpdateTime="07/01/2010 15:52:50" ComputedMinCores="1" ComputedMaxCores="14" RequestCancel="None" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" AutoCalculateMax="false" AutoCalculateMin="false" ParentJobId="0" ChildJobId="0" NumberOfCalls="0" NumberOfOutstandingCalls="0" CallDuration="0" CallsPerSecond="0" FailOnTaskFailure="false" Preemptable="true" ProjectId="1" JobTemplateId="2" OwnerId="3" ClientSourceId="5" Project="" JobTemplate="Adhoc" DefaultTaskGroupId="1112" Owner="TFS\pt00834" ClientSource="Client" xmlns="http://schemas.microsoft.com/HPCS2008/scheduler/">
    <Dependencies />
    <Tasks>
        <Task Version="2.000" Id="40685" SubmitTime="07/01/2010 15:50:50" CreateTime="07/01/2010 15:50:49" StartTime="07/01/2010 15:50:50" EndTime="07/01/2010 15:51:52" ChangeTime="07/01/2010 15:50:50" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="544" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40180" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="1" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1112" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
        <Task Version="2.000" Id="40686" SubmitTime="07/01/2010 15:50:50" CreateTime="07/01/2010 15:50:49" StartTime="07/01/2010 15:50:50" EndTime="07/01/2010 15:51:52" ChangeTime="07/01/2010 15:50:50" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="544" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40181" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="2" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1112" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
        <Task Version="2.000" Id="40687" SubmitTime="07/01/2010 15:50:50" CreateTime="07/01/2010 15:50:49" StartTime="07/01/2010 15:50:50" EndTime="07/01/2010 15:51:51" ChangeTime="07/01/2010 15:50:50" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="544" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40182" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="3" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1112" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
        <Task Version="2.000" Id="40688" SubmitTime="07/01/2010 15:50:50" CreateTime="07/01/2010 15:50:49" StartTime="07/01/2010 15:50:50" EndTime="07/01/2010 15:51:53" ChangeTime="07/01/2010 15:50:50" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="544" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40183" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="4" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1112" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
        <Task Version="2.000" Id="40689" SubmitTime="07/01/2010 15:50:50" CreateTime="07/01/2010 15:50:49" StartTime="07/01/2010 15:50:50" EndTime="07/01/2010 15:51:51" ChangeTime="07/01/2010 15:50:50" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="544" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40184" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="5" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1112" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
        <Task Version="2.000" Id="40690" SubmitTime="07/01/2010 15:50:50" CreateTime="07/01/2010 15:50:49" StartTime="07/01/2010 15:50:50" EndTime="07/01/2010 15:51:52" ChangeTime="07/01/2010 15:50:50" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="544" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40185" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="6" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1112" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
    </Tasks>
</Job>


    Highest:

    <?xml version="1.0" encoding="utf-8"?>
<Job Version="2.000" Id="545" Name="Simulation run -1 for configuration 648" SubmitTime="07/01/2010 15:50:55" CreateTime="07/01/2010 15:50:54" StartTime="07/01/2010 15:51:51" EndTime="07/01/2010 15:55:11" ChangeTime="07/01/2010 15:50:54" UnitType="Core" MinCores="1" MaxCores="16" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" RunUntilCanceled="false" IsExclusive="false" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" UserName="TFS\pt00834" JobType="Batch" Priority="Highest" RequiredNodes="" IsBackfill="false" NextTaskNiceID="31" HasGrown="false" HasShrunk="true" OrderBy="" TaskLevelUpdateTime="07/01/2010 15:54:52" MinMaxUpdateTime="07/01/2010 15:54:52" ComputedMinCores="1" ComputedMaxCores="12" RequestCancel="None" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" AutoCalculateMax="false" AutoCalculateMin="false" ParentJobId="0" ChildJobId="0" NumberOfCalls="0" NumberOfOutstandingCalls="0" CallDuration="0" CallsPerSecond="0" FailOnTaskFailure="false" Preemptable="true" ProjectId="1" JobTemplateId="3" OwnerId="3" ClientSourceId="5" Project="" JobTemplate="Baseline" DefaultTaskGroupId="1113" Owner="TFS\pt00834" ClientSource="Client" xmlns="http://schemas.microsoft.com/HPCS2008/scheduler/">
    <Dependencies />
    <Tasks>
        <Task Version="2.000" Id="40715" SubmitTime="07/01/2010 15:50:55" CreateTime="07/01/2010 15:50:54" StartTime="07/01/2010 15:51:51" EndTime="07/01/2010 15:53:01" ChangeTime="07/01/2010 15:50:55" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="545" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40210" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="1" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1113" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
        <Task Version="2.000" Id="40716" SubmitTime="07/01/2010 15:50:55" CreateTime="07/01/2010 15:50:54" StartTime="07/01/2010 15:52:21" EndTime="07/01/2010 15:53:17" ChangeTime="07/01/2010 15:50:55" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="545" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40211" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="2" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1113" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
        <Task Version="2.000" Id="40717" SubmitTime="07/01/2010 15:50:55" CreateTime="07/01/2010 15:50:55" StartTime="07/01/2010 15:53:01" EndTime="07/01/2010 15:54:04" ChangeTime="07/01/2010 15:50:55" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="545" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40212" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="3" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1113" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
        <Task Version="2.000" Id="40718" SubmitTime="07/01/2010 15:50:55" CreateTime="07/01/2010 15:50:55" StartTime="07/01/2010 15:53:03" EndTime="07/01/2010 15:54:06" ChangeTime="07/01/2010 15:50:55" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="545" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40213" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="4" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1113" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
        <Task Version="2.000" Id="40719" SubmitTime="07/01/2010 15:50:55" CreateTime="07/01/2010 15:50:55" StartTime="07/01/2010 15:53:03" EndTime="07/01/2010 15:54:06" ChangeTime="07/01/2010 15:50:55" ErrorCode="0" ErrorParams="" State="Finished" PreviousState="Running" ParentJobId="545" ExitCode="0" RequestCancel="None" Closed="false" RequeueCount="0" AutoRequeueCount="0" FailureReason="None" PendingReason="None" InstanceId="0" Output="" RecordId="40214" Name="Calculation" MinCores="1" MaxCores="1" MinSockets="1" MaxSockets="1" MinNodes="1" MaxNodes="1" NiceId="5" CommandLine="BECMConsoleApp.exe configurationid=648 simulationruninstanceid=-1 jobtype=calc firsttrackid=1 lasttrackid=1 " WorkDirectory="C:\Program Files (x86)\Simon\BECMConsoleApp" HasCustomProps="false" IsParametric="false" GroupId="1113" ParentJobState="Finished" UnitType="Core" ParametricRunningCount="0" ParametricCanceledCount="0" ParametricFailedCount="0" ParametricQueuedCount="0" />
    </Tasks>
</Job>
    Thursday, January 7, 2010 4:18 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    This is the expected behavior. The highest priority job will start as soon as possible with it's minimum resource request. Once it has started it the lower priority jobs that are running will continue to run with the resources they currently are using.

    What I think you are expecting is "shrink to grow" which we don't support. Once the higher priority job starts the currently running jobs will be allowed to continue to run with the resources they have allocated when the higher priority job starts. They will not shrink further.
    • Proposed as answer by Steve Chilcoat Thursday, January 7, 2010 9:04 PM
    • Marked as answer by Simon Place Friday, January 8, 2010 10:52 AM
    Thursday, January 7, 2010 8:57 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    Many thanks to everyone that has helped with this. It feels as though we've bottomed this out even though the answer isn't the one that I was hoping for ;-)

    Simon
    Friday, January 8, 2010 10:52 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    I really do not think that this is the expected behavior of much of the user base. It is the currently implemented behavior. If graceful pre-emption, job growing and job shrinking all are enabled, a resource that is released by a task should go to the job at the top of the pecking order still looking for resources.

    I ran into this problem, too and I don't have a solution for it still. Our workload regularly has long running low priority jobs with a huge number of relatively short tasks. In the interest of maximizing the cluster throughput, these low priority jobs grow to use up any unused resource. However, they fail to shrink for new higher priority tasks but for their minimum allocation.

    I really hope that this "priority only gets you the minimum resources to start the job" behavior changes in future versions as it is a big detriment to high cluster throughput in heterogeneous job environments (forcing us to artificially limit the maximum number of resources the "long running, low priority jobs" can request even though the cluster might be idle otherwise).
    • Edited by Marc W de Friday, January 15, 2010 8:38 AM clarification why this hinders throughput
    Friday, January 15, 2010 8:29 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    Marc

    That's very interesting input and it's comforting to note that I am not the only person perplexed by the implemented behaviour.

    On your original thread, Josh commented that a possible change to this behaviour is under discussion. Perhaps someone on the HPC product team can comment on whether this change is likely to happen e.g. in R2.
    Friday, January 15, 2010 9:43 AM
  • Question
    Sign in to vote
    0
    Sign in to vote
    There is a new feature in HPC Server 2008 R2, which is currently available in the Beta1. It is a new scheduling policy called "Service Balanced Scheduling". If the cluster is set up to use this policy then jobs start with their minimum resource request and then potentially grow as the unused resources are rebalanced among the running jobs. Periodically the cluster resources are rebalanced, if possible, using a formula that is a function of priority and a cluster wide weight. Depending on the weight the higher priority jobs will get more resources than lower priority jobs.

    More info at:
    http://technet.microsoft.com/en-us/library/ee783547(WS.10).aspx
    Friday, January 15, 2010 6:13 PM