none
Setup HPC Headnode and Cluster on Azure

    質問

  • After I surf around the Internet, the only method of setting up HPC server is using AzureSampleService sample code - Deployment Application.

    I could not find any resources online about automation setup of the headnode and cluster (compute nodes).

     

    I would like to know how to setup HPC Server and Cluster on Windows Azure by creating new project?

     

    P.S. all of the nodes are worker role here.

    And configuration should be in command line. (because of automation)

     

    I hope someone could give me a start point of setting up the whole cluster and server. (head node and compute node)

     

    thanks

    Ken


    Ken Hui Application Developer
    2012年1月18日 7:30

すべての返信

  • After I google around, I used the sample program to generate the config file (ServiceConfiguration.Cloud.cscfg) and deploy to Azure.

    However the headnode does not function correctly. I used RDP and remote into the headnode. I always get the same error of "No such host is known" when I am using HPC command line.

    And I also find the database didn't update.

    Anyone know how does Azure HPC update / link up the DB ??

    I checked the SQL connection string is correct in config file.

    I started the Cluster Manager and Job Manager, but both of them ask me to select a head node to connect.

    thanks in advance.


    Ken Hui Application Developer
    2012年1月20日 8:40
  • There are two sample applications that I've found.  One from November, 2011 which I was not able to get running. Second one that seems to be a later version of the first and am able to get running.  The second is located at http://msdn.microsoft.com/en-us/hh670560.  I had problems with naming conventions (storage account name, site, name, db name, etc), certificates and db permissions to get it working.

    I too am struggling on how to find all the strings and required configuration to merge into our applicaion (we already use Hpc for on premisis).  Where did you find the filled out ServiceConfiguration.Cloud.cscfg?  My file contains many properties with no values and when looking at the AppConfigure code many of those properties are not referenced while others are filled out programatically that are not in the ServiceConfiguration.Cloud.csfg file. 

    We used a scripted approach for our early adoption of of the web side of our application but I don't currently see a way to use that approach to setup and Hpc set of nodes needed for the worker role side of things. 

    Is a scripted solution possible from a packaging and deployment persepective for Hpc in Azure?

     

    -Scott

    2012年1月20日 16:27
  • Try this out and see if it can help you

    http://blogs.msdn.com/b/hpctrekker/archive/2011/12/06/deploying-an-hpc-cluster-using-just-powershell-part-i.aspx

    It is still a little rough around the edges, expect a new version shortly! (I'm not sure where it'll be posted, most likely the link in the blog willbe updated)

    2012年1月20日 20:14
  • hi Scott

    According to this page, you will find that you need to use the ClusterConfig to generate those "no values" setting in your configuration files.

    What I did is that

    1. Download the Azure Sample Service source code (the only official sample code) from here

    http://code.msdn.microsoft.com/windowsazure/Windows-Azure-HPC-7d75eb26

     

    2. You need to create a console program and using ClusterConfig Class (Reference could be found in HPC Scheduler SDK ) to config your ServiceConfiguration.Cloud/Local.cscfg . Simple sample code of using ClusterConfig could be find in here

    http://msdn.microsoft.com/en-us/library/hh560251(v=vs.85).aspx

    Or the doc file included in the Azure Sample Service

     

    3. I copy the original ServiceConfiguration.Cloud.cscfg from my Azure solution to the Output folder of the console program (for mine: ConsoleApp/bin/Debug) as input.cscfg

     

    4. After the output.cscfg is generated, I use a notepad to open the output.cscfg and copy those config value to my original ServiceConfiguration.cscfg. Note that the Service Name is different from your original one, it should be "HPCAzureService" (I guess there is a parameter in the ClusterConfig class that could change it, however I can't even find the class reference of this class on msdn)

    5. At this time you should have the well configure cscfg file.

    Note : You need to generate pfx file from your Cert when you are using ConsoleApp for configure generation.

    ----------------------------------------

    I read through the 3 hands on lab about HPC Azure. The link you provided are one of them. I found they are using the same Deployment Application for HPC Deployment. However what I would like to do is deploying HeadNode and ComputeNode only (i.e. w/o Web Front End). The Azure Sample Service also include a initializeDB.exe and some SQL files which is important to build up your DB for HPC. You can find how to use the initializeDB.exe from PublishToAzure.cs in the AzureSampleService sample code in the Delopyment Application.

    I build up the HeadNode and Compute Node and the DB. But I still found that the HeadNode does not connect to the DB. the Node Table in the DB does not contain IP info of the nodes.

    Question : How do I fill in the missing info of the DB?? I just want to manually install the HPC on Azure. My Front End will be my custom MVC Web Role and linked with Transform Manager. I hope someone could help. Thanks

     

    ----------------------------------------

     

    Hi James

    I read the PS for Azure installation too, it is similar to AzureSampleService. But I still can't find the magic of linking up HPC headnode and the DB yet. I hope you could give me some direction.

    Thanks in advance.

     

    Ken


    Ken Hui Application Developer
    2012年1月23日 3:37
  • I am also facing the same problem. have you succeeded? can you tell me how do you make the head node running... Thanks

    Ashi

    2012年9月10日 9:02