Hello,<br/> We run stochastic simulations on our cluster; this kind of simulations require the submission of a lot of runs with several different parameters, passed at the command line. So, we generate and save an XML job file, load it, and submit it. With CCS 2003 loading was as fast as possible (just 1/2 seconds for a job with 30k tasks). On HPC 2008 it takes almost half an hour, on the default Job template (no additional validation required). <br/> Is that expected or is there something wrong with our deployment? <br/> Which are the exact steps going on during submission from an XML file? (does it do schema validation? other validations? Is it possible to turn them off in some config file/reg key?)<br/> Thanks,<br/> <br/> Lorenzo<br/>© 2009 Microsoft Corporation. All rights reserved.Fri, 29 May 2009 07:12:19 Zcccee2b8-03af-4d23-8ce1-6e1fca2dc34dhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#cccee2b8-03af-4d23-8ce1-6e1fca2dc34dhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#cccee2b8-03af-4d23-8ce1-6e1fca2dc34dLorenzohttp://social.microsoft.com/Profile/en-US/?user=LorenzoHello,<br/> We run stochastic simulations on our cluster; this kind of simulations require the submission of a lot of runs with several different parameters, passed at the command line. So, we generate and save an XML job file, load it, and submit it. With CCS 2003 loading was as fast as possible (just 1/2 seconds for a job with 30k tasks). On HPC 2008 it takes almost half an hour, on the default Job template (no additional validation required). <br/> Is that expected or is there something wrong with our deployment? <br/> Which are the exact steps going on during submission from an XML file? (does it do schema validation? other validations? Is it possible to turn them off in some config file/reg key?)<br/> Thanks,<br/> <br/> Lorenzo<br/>Mon, 30 Mar 2009 07:26:01 Z2009-03-30T07:26:01Zhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#73f15b3b-10bb-4f6e-948d-e624cb0adf1fhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#73f15b3b-10bb-4f6e-948d-e624cb0adf1fSteve Chilcoat - Microsofthttp://social.microsoft.com/Profile/en-US/?user=Steve%20Chilcoat%20-%20MicrosoftLorenzo,<br/><br/>Are the 30,000 tasks individual tasks or 30,000 parametric tasks? <br/><br/>ThanksWed, 01 Apr 2009 13:56:05 Z2009-04-01T13:56:05Zhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#d924c86d-fb27-4330-96f0-4550490fec2bhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#d924c86d-fb27-4330-96f0-4550490fec2bLorenzohttp://social.microsoft.com/Profile/en-US/?user=LorenzoThey are 30000 individual tasks. We cannot use parametric tasks for this particular application (it acts on different files and parameter combinations, which can not be summarized by a single integer value).<br/> <br/> Lorenzo<br/>Wed, 01 Apr 2009 14:55:54 Z2009-04-01T14:55:54Zhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#9ed6a09c-47e1-408a-86b3-1fe22d0ab43ahttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#9ed6a09c-47e1-408a-86b3-1fe22d0ab43aJosh Barnardhttp://social.microsoft.com/Profile/en-US/?user=Josh%20Barnard<p>Slow performance when adding a large number of tasks to a job is a known issue that we are looking into and hope to address soon.  In the meantime I suggest you use Parametric Sweeps as they will be much more performant to create.<br/><br/>You can work around the problem you mentoined above by wrapping you task in a batch (or PSH, or ...) script that starts your application with the correct set of parameters.  You can use the * in your sweep to represent the index into the array of parameter sets, for example.<br/><br/>Thanks!<br/>Josh</p><hr class="sig">-JoshThu, 02 Apr 2009 22:03:16 Z2009-04-02T22:03:16Zhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#2509bb27-52cb-4d77-be9e-a5d6c3d2af95http://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#2509bb27-52cb-4d77-be9e-a5d6c3d2af95Lorenzohttp://social.microsoft.com/Profile/en-US/?user=LorenzoHello,<br/> <br/> I'd like to avoid the creation of batch; it's cumbersome and error prone. <br/> I suppose that we'll either write our own submission application (so that at least the user has to not wait minutes to press the &quot;submit&quot; button) or go back to CCS 2003 till the issue is fixed.<br/> <br/> Thanks!<br/> Lorenzo<br/>Fri, 03 Apr 2009 07:15:58 Z2009-04-03T07:15:58Zhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#556bf724-49b9-44aa-a0c1-6afd8974095bhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#556bf724-49b9-44aa-a0c1-6afd8974095bJosh Barnardhttp://social.microsoft.com/Profile/en-US/?user=Josh%20BarnardLorenzo,<br/>We are working on this issue right now.  Could you please send me your XML file?   It would be helpful for us to see how it is constructed/what your job contains.<br/><br/>My e-mail: JBarnard AT Microsoft DOT com<br/><br/>Thanks!<br/>Josh<hr class="sig">-JoshThu, 09 Apr 2009 00:03:23 Z2009-04-09T00:03:23Zhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#895626a6-598c-4a50-8044-2e8a0be7fb35http://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#895626a6-598c-4a50-8044-2e8a0be7fb35Josh Barnardhttp://social.microsoft.com/Profile/en-US/?user=Josh%20BarnardLorenzo,<br/>We have some fixes around this that should be included in our first Serivce Pack; please install that SP on your client and head node when it comes out and hopefully times will be significantly reduced!<br/><br/>Thanks,<br/>Josh<hr class="sig">-JoshThu, 28 May 2009 21:32:38 Z2009-05-28T21:32:38Zhttp://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#9c6ec40a-7933-4192-8e6a-b194e26a34b6http://social.microsoft.com/Forums/en-US/windowshpcsched/thread/cccee2b8-03af-4d23-8ce1-6e1fca2dc34d#9c6ec40a-7933-4192-8e6a-b194e26a34b6Lorenzohttp://social.microsoft.com/Profile/en-US/?user=LorenzoWill try it asap!<br/> Thank you!<br/> <br/> Lorenzo<br/>Fri, 29 May 2009 07:12:14 Z2009-05-29T07:12:14Z