Sign in
Microsoft.com
 
Microsoft
 
 
 
Microsoft >
> 
> 
> 
Question on constraints and timing in Job Submission and Management

Answered Question on constraints and timing in Job Submission and Management

  • Wednesday, August 27, 2008 2:43 AM
     
     
    Sign In to Vote
    0
    Dear Community,
    I'd like to use the Windows Server 2008 Scheduler to control access to a (commercial) program that runs its own cluster-control algorithms. This commercial program can break a cluster of nodes into to N queues (and the number N  is configurable internally).

    Is it possible to put a constrain when submitting a job that constrains the cluster run at most N  executables, but then queue any more job requests until a job has finished?

    For example. If N=3, I'd like to run at most 3 executables of the main program on the headnode (which would in turn send jobs to its own cluster), and then have a fourth program wait until one of the 3 executables has finished the job.

    Also, is it possible to submit a job with a constraint that it will run at some future time? For example, if a lab machine will not generate some data files until midnight, I'd like to submit a job that won't start running until midnight. I can not see any option to do this on either the job submit command line, or the Cluster Manager window tool.

    Much thanks for any advice
    regards
    Simon Michnowicz
    Australian Proteomics Computation FacilityL
    udwig Institute For Cancer Research
    Royal Melbourne Hospital,
    Victoria

     

     

All Replies

  • Wednesday, August 27, 2008 5:53 PM
    Moderator
     
     Answered
    Sign In to Vote
    0
    You may be able to do this by doing the following:
    Create a job which requests 3 - 3 cores
    Add all of your tasks, each of which requests 1 core

    Your job will be assigned 3 cores and run up to 3 tasks at a time.

    As for running at a time in the future . . . we don't have that feature right now but at looking into it for a later version.

    Thanks,
    Josh
    -Josh
     
  • Thursday, August 28, 2008 7:32 AM
     
     
    Sign In to Vote
    0
    Josh
    thanks for the response.

    1) If I request a single core, is my program constrained to running only on that core? The program that runs on the headnode may well use all the cores. Does the allocation algorithm look at the usage of the destination resource? For example, if a non-queued executable is consuming the resources of a node, will the queuing algorithm delay a job submission until it is free?

    I had considered using the core solution, but I was also interested in running a general solution. For example, suppose I want to implement a M/M/N queue, i.e. a single FIFO queue, with N servers. (The servers here executables running , say MPI, across dedicated  nodes.  Is there a simple way to do this with Windows Queuing system? (The reason you may want N servers is that we have found that the law of diminished returns applies to some of our algorithms when using many nodes to process the job.)
     
    2) I can solve this problem by using the at command, i.e. at <time> job sumbit <job>. It just feels strange to use two different job scheduling tools.

    Regards
    Simon Michnowicz
     
  • Monday, September 08, 2008 5:48 PM
    Moderator
     
     Answered
    Sign In to Vote
    0

    1. Your job will be constrained to the number of cores that you request.  You can overcome this by using the Task's Exclusive option, which will assign your task all processors on the node.

    I'm not sure I understand the solution that you're talking about with an M/M/N queue . . . can you explain a bit further?  If you're looking to start up a bunch of servers and FIFO queue jobs to them, I'd recommend checking out the new (in v2) SOA workload programming model, as this provides just that sort of functionality.


    2. That command will SUBMIT a job at the time you specify, but doesn't guarantee it will run at the time that you specify.  However it's a good mechanism for now.  We are looking into including a feature to allow "time scheduled" jobs in the future.

    -Josh