locked
How to stop a SOA job from growing - stop more tasks being added when the job is running? RRS feed

  • Question

  • When I create a job to run a SOA service, a number of tasks are created as part of the job depending on various factors. Is it possible, once the tasks have all been added to the job, that no more are added? I don't care how many tasks are initially created for the job, only that no more are added when the job is running. Also, if possible I would like to set this option only for a job, and not for the whole cluster, so I'm interested in some way to do this preferably on the job level.

    Is there some way to stop a job from growing (having more tasks assigned)? 

    Thanks.

     

     



    Tuesday, August 16, 2011 4:17 PM

Answers

  • Hi Mingqing,

    I actually found a working solution, and thought I should document it here for posterity. I could only get the job to not grow if, after the job had been created, I set ISchedulerJob.TargetResourceCount to be the number of tasks that were created for that job (ISchedulerJob.GetTaskList().Count). I also set the allocationAdjustInterval in the configuration file to -1. Only both of these setting applied together would stop the job from growing.

    • Marked as answer by krolley Monday, October 3, 2011 2:01 PM
    Monday, October 3, 2011 2:01 PM

All replies

  • Hi,

    There're two ways to disabling grow for a SOA job:

    1. Set AutomaticGrowthEnabled=false in scheduler setting, but this is a cluster wide setting and would impact normal job as well
    2. Set AllocationAdjustInterval = -1 in service configuration file so that broker will not do resource adjust based on number of outstanding SOA requests, therefore grow is disabled, but this also means SOA won’t do shrink (removing tasks). This is per service level setting.
      <microsoft.Hpc.Broker>
        <!--configuration to control broker's monitoring behavior-->
        <monitor messageThrottleStartThreshold="4096"
                 messageThrottleStopThreshold="3072"
                 loadSamplingInterval="1000"
                 allocationAdjustInterval="-1"

    Unfortunately, we don't support this type of setting in job level right now.


    • Marked as answer by krolley Wednesday, August 24, 2011 7:49 AM
    • Unmarked as answer by krolley Wednesday, September 21, 2011 4:07 PM
    Wednesday, August 17, 2011 4:08 AM
  • Hi Mingqing,

    I have been testing this today, and found that setting allocationAdjustInterval is having no effect. In my case, I have two compute nodes and when I start the job one compute node is offline. The job starts with a single task for the single online compute node. After some time, I bring the other compute node online. Literally one second after I bring the second compute node online, a second task is added to the running job, a task that uses the second compute node. It doesn't matter whether I set the allocationAdjustInterval to -1, or 300000, a second job is created almost instantaneously after I bring the second compute node online.

    Is there some setting I can check to see that this setting should be working? Should the setting appear as an environment variable? Are the allocation adjustment passes logged to the SOA infrastructure trace? I am setting the SessionStartInfo.RequestedNodesList property to explicitly use the two compute nodes (e.g. setting it to "computenode1|computenode2"), does this have an effect? Also, the job has no other special settings - no min and max resources, and auto calculate resources is set.

    Any help is appreciated.

    Wednesday, September 21, 2011 4:07 PM
  • Hi krolley,

    In that case you could try set AutomaticGrowthEnabled=false by "cluscfg setparams"

    Tuesday, September 27, 2011 3:19 AM
  • Hi Mingqing,

    I actually found a working solution, and thought I should document it here for posterity. I could only get the job to not grow if, after the job had been created, I set ISchedulerJob.TargetResourceCount to be the number of tasks that were created for that job (ISchedulerJob.GetTaskList().Count). I also set the allocationAdjustInterval in the configuration file to -1. Only both of these setting applied together would stop the job from growing.

    • Marked as answer by krolley Monday, October 3, 2011 2:01 PM
    Monday, October 3, 2011 2:01 PM