I work with a finite element solver that will create a mesh, and then sweep a parameter from A to B using a step size of C. The licensing for the software will only allow the meshing to be completed on one node using all the processors available on that node. Once the meshing is completed the license will allow the sweep to be completed on multiple nodes. Each node takes one of the steps (C) and runs it concurrently with other steps.
Presently, I set up Job1 in HPC to run the meshing on node Z and then stop. I then have to create Job2 in HPC that takes the meshing result and uses it to run the sweeps on a separate set of nodes (Q thru Y). The resoning for doing this is that if I reserved the resources of Q thru Z up front I would have Q thru Y sitting there for hours while the meshing was being performed on Z instead of having someone else utilize these resources.
What I would like to do is run Job1, and instead of running Job2 just take the command from Job2 and set it up as the last task in Job1 so that the meshing and sweeps could be run from one job. The catch is the sweeps that I currently run in Job 2 would need to utilize a different set of resources (Q thru Y). Remember I only reserved resource Z for Job1. Therefore, I not only want to fire off the sweeps as the last task in Job1, but also use a new set of nodes to finish out Job1. I would even settle for setting up 2 different jobs like I currently do, and then have the last task in Job1 fire off Job2 if this can be accomplished easier due to allocating different resources for each job.
It would probably be really easy to keep what I currently have and just fire off Job2 as the last task from Job1. Is this possible? How do I do it?
Thanks for Choosing HPC, it appears that your question can be better solved by using Activation Filter
You can set up your filter for Job2 to query the status of your 1st Job, then Job2 will only run if Job1 finishes.
Please let me know if this works for you.