locked
Python Example of using the HPC Basic Profile Web Service RRS feed

  • Question

  • Hello All,

    I have a python application that I would like to use to drive a parallel application on the Windows HPC Server 2008.  I took a look at the C# API provided for communicating with the HPC Basic Profile (HPCBP) Web service, which can run on the head node of a cluster.  In the long run, I think that I may choose to go with a .NET implementation, but right now I would like to continue to use python to drive my parallel application.

    I know that as a comprimise I could use Iron Python, but right now going that route has it's own set of limitations. So my question is does anyone know of an example, sample, tutorial, or outline for communicating with the HPC Server 2008 Job Scheduler via HPCBP using Python?

    Regards
    Wednesday, August 26, 2009 11:18 PM

Answers

  • Hi,
    These pages include COM examples:
    http://msdn.microsoft.com/en-us/library/cc907080(VS.85).aspx

    I believe those examples are all using C++ though.  This website covers how to call COM from Python code:
    http://www.devshed.com/c/a/Python/Windows-Programming-in-Python/2/

    Perhaps with those two links combined, you can put something together?  I will ask around internally if anyone has Python code using COM to submit a job.

    Thanks,
    Josh


    -Josh
    Thursday, August 27, 2009 6:13 PM
    Moderator
  • I have IronPython code that'll talk to HPC Server 2008 job scheduler. It is not generic python and is not talking to HpcBasicProfile. But hope it helps.

    # This is a IronPython program. Run it under IronPython like: ipy.exe [file.py]
    #
    import sys
    import clr
    sys.path.append("C:\Program Files\Microsoft HPC Pack 2008 SDK\Bin")
    clr.AddReferenceToFile("Microsoft.Hpc.Scheduler.dll")
    from Microsoft.Hpc.Scheduler import *

    schedulername="localhost"
    jobid=0
    scheduler = Scheduler()

    def Usage():
            print "runjob.py [-s scheduler] [-view jobid]\n"       

    def ParseCmdLine():
            global schedulername, jobid
           
            if len(sys.argv) == 1:
                    Usage()
                    return 1

            i = 1
            while (i < len(sys.argv)):
                    if sys.argv[i] == "-s":
                            schedulername = sys.argv[i+1]
                            i = i + 2
                    elif sys.argv[i] == "-view":
                            jobid = int(sys.argv[i+1])
                            i = i + 2
                    else:
                            Usage()
                            return 1
                                  
            return 0             
                   

    def CreateJob():
            job = scheduler.CreateJob()
            job.Name = "pythonTest"
            task = job.CreateTask()
            task.CommandLine="hostname.exe"
            job.AddTask(task)
            scheduler.SubmitJob(job, None, None)
            print "Job has been submitted. ID: ", job.Id
            return job.Id

    def ViewJob(jobid):
            job = scheduler.OpenJob(jobid)
            print "JobId=", job.Id
            print "JobName=", job.Name
            print "JobState=", job.State
            print "JobOwner=", job.Owner

    def main():
            if ParseCmdLine() != 0:
                    sys.exit(1)

            scheduler.Connect(schedulername)

            if jobid == 0:
                    CreateJob()
            else:
                    ViewJob(jobid)

            sys.exit(0)
           
    if __name__ == "__main__":
        main()

           

    Thursday, August 27, 2009 6:28 PM

All replies

  • On page 6 of the White Paper, Windows HPC Server 2008: Using Windows HPC Server 2008 Job Scheduler the statement:

    A variety of additional scripting languages are supported in the COM interface, including Perl, Python, Microsoft JScript, and Microsoft Visual Basic Scripting Addition.

    in reference to Job Submission.  However, I haven't been able to find a useful example.  My thought was that if I could find a useful example, I might be able to use the Python win32 Extension to access a COM object that could handle job submission for me.  Does anyone have suggestions for a reference?

    Regards
    Thursday, August 27, 2009 3:59 PM
  • Hi,
    These pages include COM examples:
    http://msdn.microsoft.com/en-us/library/cc907080(VS.85).aspx

    I believe those examples are all using C++ though.  This website covers how to call COM from Python code:
    http://www.devshed.com/c/a/Python/Windows-Programming-in-Python/2/

    Perhaps with those two links combined, you can put something together?  I will ask around internally if anyone has Python code using COM to submit a job.

    Thanks,
    Josh


    -Josh
    Thursday, August 27, 2009 6:13 PM
    Moderator
  • I have IronPython code that'll talk to HPC Server 2008 job scheduler. It is not generic python and is not talking to HpcBasicProfile. But hope it helps.

    # This is a IronPython program. Run it under IronPython like: ipy.exe [file.py]
    #
    import sys
    import clr
    sys.path.append("C:\Program Files\Microsoft HPC Pack 2008 SDK\Bin")
    clr.AddReferenceToFile("Microsoft.Hpc.Scheduler.dll")
    from Microsoft.Hpc.Scheduler import *

    schedulername="localhost"
    jobid=0
    scheduler = Scheduler()

    def Usage():
            print "runjob.py [-s scheduler] [-view jobid]\n"       

    def ParseCmdLine():
            global schedulername, jobid
           
            if len(sys.argv) == 1:
                    Usage()
                    return 1

            i = 1
            while (i < len(sys.argv)):
                    if sys.argv[i] == "-s":
                            schedulername = sys.argv[i+1]
                            i = i + 2
                    elif sys.argv[i] == "-view":
                            jobid = int(sys.argv[i+1])
                            i = i + 2
                    else:
                            Usage()
                            return 1
                                  
            return 0             
                   

    def CreateJob():
            job = scheduler.CreateJob()
            job.Name = "pythonTest"
            task = job.CreateTask()
            task.CommandLine="hostname.exe"
            job.AddTask(task)
            scheduler.SubmitJob(job, None, None)
            print "Job has been submitted. ID: ", job.Id
            return job.Id

    def ViewJob(jobid):
            job = scheduler.OpenJob(jobid)
            print "JobId=", job.Id
            print "JobName=", job.Name
            print "JobState=", job.State
            print "JobOwner=", job.Owner

    def main():
            if ParseCmdLine() != 0:
                    sys.exit(1)

            scheduler.Connect(schedulername)

            if jobid == 0:
                    CreateJob()
            else:
                    ViewJob(jobid)

            sys.exit(0)
           
    if __name__ == "__main__":
        main()

           

    Thursday, August 27, 2009 6:28 PM