locked
Submitting job outside of headnode RRS feed

  • Question

  • I have an HPC Cluster setup entirely on Azure IAAS VM's. Client programs run fine as long as they are run from the headnode. When I attemt to run the same client program from outside of the headnode, the CreateSession throws an exception with the error: "Fail to get the cluster name". My code is as follows:

    SessionStartInfo info = new SessionStartInfo("myheadnode.cloudapp.net", "HelloWorld");
    info.Username = @"DOMAIN\testhpcuser";
    info.Password = @"MyP@$$w0rd";
    info.Secure = false;
    info.TransportScheme = TransportScheme.WebAPI;      // Local Client needs WebAPI or it has a login problem.
    using (Session session = Session.CreateSession(info))

    I found this thread on the forum that states I cannot use the .Net API but must use the Rest API.

    http://social.microsoft.com/Forums/en-US/018ea855-5970-4cce-9ab1-525bbed7e40d/job-submission-with-scheduler-on-azure

    This answer was given on 11/30/2011, which is a lifetime in Azure years. Is this statement still true?  The error I am getting is pretty non-descriptive.

    Other information that may be of value:

         Head node and Compute nodes are Windows Server 2012

         The problem occur on both HPC Pack 2012 and HPC Pack 2012 SP1

         Azure HPC Scheduler SDK is 1.8

         I have imported the Azure HPC Default Certificate from the Headnode to both my Azure account and the client machine.

    I would appreciate any help.

    Monday, September 23, 2013 2:20 PM

All replies

  • The Scheduler On Azure article is really only relevant to the Scheduler On Azure install topology... which is no longer available starting with v4sp1.  The article does not apply to a full install... which is what you have in IAAS.

    I would expect your .net code to work fine from inside the IAAS network (with corrected hn wins name not cloud fqdn) but to fail "from the internet".  Is this what you are seeing?

    Monday, September 23, 2013 11:25 PM
  • Yes, I need to access the HPC Scheduler from the internet.  Some users will not be logged on to the domain of the HPC Cluster.  I was hoping by adding the Username and Password of the HPC domain would handle the authentication.

    Is there another way I can make this work?  I really just want to access the HPC Scheduler in the same way a Bing/Google Search would handle it.  (an anonymous user).


    Tuesday, September 24, 2013 1:17 PM
  • The Scheduler REST API and Web Portal are options: HPC Web Components

    The Scheduler REST API will not address SOA features though... perhaps the Web Portal will be sufficient?

    I would proceed cautiously here.  The HPC stack is/was intended to be run on subnets behind suitable firewalls/security controls.  What you describe above would result in exposing the headnode as an attack surface.  Truly anonymous user access would likely expose your Azure account to excessive monetary charges as well as loss of security of any secrets associated with jobs and/or tasks.  There may be other risks as well. 

    d


    • Edited by DarylMsft Tuesday, September 24, 2013 7:05 PM wording too strong
    Tuesday, September 24, 2013 6:07 PM
  • Daryl, thanks for your replies.  According to the Web API Reference, you can create a SOA session when the Web Service is hosted in Azure.  I think I will try going down this path.

    I agree with your security concerns.  Right now I am only in a "Proof of Concept" stage.  If our project gets approved, then security for our entire application, including HPC, will need to be addressed. 

    I'll reply to this thread with my results with the Web API.

    Tony

    Tuesday, September 24, 2013 7:48 PM
  • I am not an authority on SOA but I _think_ the SOA REST API was only available in "Scheduler On Azure"...

    In that context, the two REST APIs (SOA and Scheduler) were "normally" run separately from the Scheduler behind a load balancer... thus the (OS's of the) listeners were the attack surfaces and not the Scheduler process/OS.   This separation design might be helpful if/when you exit PoC.

    d

    Tuesday, September 24, 2013 8:02 PM