Adding tasks to running jobs RRS feed

  • Question

  • I have a Linux-Node running some job and want to add dynamically tasks to this job.

    I access HPC via Rest-API.

    First problem is, I have to provide a password. How can I use the credentials of the current user?

    Next problem is, the newly created tasks stay in "Configuring"-state. How can I submit these tasks?

    Thursday, March 16, 2017 8:40 AM

All replies

  • Hi Daniel,

      You can check our python samples (from the SDK) to add and submit the task in a job (We are working on a python SDK to help this type of work in near future). And you have to provide the username password in plain text.

      For your second question, the task need to be submited after it is adding to the job

    Qiufang Shi

    Thursday, March 16, 2017 10:11 AM
  • Hi Qiufang,

    submitting the task is exactly the problem. There is no SubmitTask-command for the Rest-API, only SubmitJob, which does not work for already running jobs.

    Thursday, March 16, 2017 12:17 PM
  • Hi Daniel, have you tried requeue task?

    Qiufang Shi

    Friday, March 17, 2017 12:48 AM
  • requeue-task only requeues failed or canceled tasks, and put them back into "configuring" state. But my newly created tasks are already in "configuring" state.

    There exisits ISchedulerJob.SubmitTaskById, but not for the REST-API.

    I'll give up, because I cannot create jobs which use multiple nodes, so I'm forced to create a job for each task.

    HPC is not suitable for the way I'd like to work:

    1. creating a job with I initial task

    2. inside this task add dynamically more tasks to do work in parallel over many nodes

    3. have some finalization

    Friday, March 17, 2017 12:33 PM
  • Hi Daniel,

      Sorry for the trouble you've all met. I'll make sure I've tried before a reply the workable solution for you.

      And what timeline are you looking for? Will you try again if we put the "Submit Task" in the REST API? Do you need depdencies between the tasks you submitted dynamically?

    Qiufang Shi

    Monday, March 20, 2017 2:53 PM
  • Daniel,

      As we tested, the "Requeue" API does able to submit the configuring task in the running job. So it shall just work without a Submit Task API.

    Qiufang Shi

    Tuesday, March 21, 2017 6:37 AM