locked
Windows 7 SP1 VHD - Azure Node template. RRS feed

  • Question

  • I have asked the following on 'Azure' HPC forum, to no avail:

    1) Is Family 4 OS (2012 R2) not supported yet for HPC Azure Node template?

    2) Can I specify a Windows 7 OS VHD in a HPC Azure template (I need to customize the VHD with Oracle client etc that is supported on Win 7), I can always 'upload' the customized VHD to my cloud storage account.. Is this possible at all?

    I am interested particularly in #2 above. If I can attach a VHD, should that VHD contain the HPC client utilities pre installed? (thats a prerequisitve for all of my "on-premise" workstation nodes)

    Thanks

    Wednesday, September 21, 2016 7:23 PM

All replies

  • 1. What version of HPC Pack are you using? If you are using HPC Pack 2012 R2, Server 2008 R2, Server 2012 and Server 2012 R2 are supported OS in windows azure node template

    2. Windows Azure nodes template is only for Azure Cloud Service which is PaaS node only, thus you can't specify customized image. But you can leverage our HPC Pack IaaS solution where you can specify customized image https://blogs.technet.microsoft.com/windowshpc/2014/12/02/automating-hpc-cluster-deployments-in-azure-iaas-part-i-create-a-hpc-pack-cluster-using-the-microsoft-hpc-pack-iaas-deployment-script/ ; If you want burst to azure IaaS nodes, this is currently not available. Or you can use ARM template to deploy the whole cluster in azure. (Please just search HPC in the azure marketplace)

    3. If you still want to continue to deploy HPC IaaS cluster, you can specify your own image with our without HPC Pack installed (In your case, it is not the client utilities, but install as workstation node) as you can specify below config:

      • ImageName: (optional) Specifies the name of the base image for broker nodes. You can use your private image as the base image if it meets the following requirements:
      1. The image must be in the specified Location. HPCPackInstalled: (Optional)Specify this attribute to true if the image you are using is already with HPC Pack installed, i.e. it was captured from a installed node. If this attribute is not specified or specified to false, the image must not have HPC Pack installed.
            ImageName cannot be specified if OSVersion is already specified


    Qiufang Shi

    Thursday, September 22, 2016 1:47 AM
  • Since I have an on-premise headnode and few workstationnodes on-premise, wanted to test out the on-demand burst mode by augmenting more azure nodes. 

    I followed the steps from https://azure.microsoft.com/en-us/documentation/articles/cloud-services-setup-hybrid-hpcpack-cluster/

    What I was not able to do is to customize the OS that is created by the above tutorial. (eg., installing 4.6.1 which a requirement for my SOA dll) and an installation of 'oracle client' that updates the on-premise database (on-premise workstation nodes work fine since i have oracle client installed as part of the image in our organization).

    the next challenge would be to 'update' the on-premise oracle database in simplest way possible (is hybrid connection from biztalk service the best and easiest of all options?)

    Thursday, September 22, 2016 3:07 AM
  • 1. First, the steps you're referring is to do HPC bursting to azure PaaS nodes which you can't use customized image. This image is maintained by Microsoft and will be kept updated with latest windows patches. If you keep going with this way, you can try:

    - Install your oracle client on these PaaS nodes. (Leverage "startup task" that's defined in the azure node template)

    2. There is one way currently supported for you to use your customized image. You need:

    - Set up azure site to site VPN or express route to extend your network to cloud, please check: https://azure.microsoft.com/en-us/documentation/articles/virtual-networks-overview/

    - Spin up virtual machines in the cloud through ARM template, and install HPC Pack and join your local domain;

    3. For updating on premise database from client machine, you may have to set up the site to site VPN or express route.


    Qiufang Shi

    Thursday, September 22, 2016 8:34 AM
  • 1) The default image I am getting is 2012 (when I create the Azure Default Template). How do I get 2012 R2?

    2) What's the best way to access the "files" or content created by SOA dlls on the azure nodes without having to go the express route/VPN? Can I 'push' files from there to the head node as part of node release task? or can i access the azure node's shared folder from the on-premise headnode after the job is completed (and the aZure node is still 'online')?

    Thursday, September 22, 2016 5:21 PM
  • 1. You can change the default image during template creation or edit the template as below:

    2. Yes, you can upload the data to azure storage in the job's release task, and create a job that depend on this job but running on on-premise node to pull the result back. And there are quite a few ways to do this, I can share some tools you can look into now:

    - hpcpack.exe this is a built-in tool in HPC pack

    - Leverage azure files ; when you submit your job, you can specify a pre-task and release-task to mount and unmounts the azure files to the azure nodes so that your application can write the out put the share without change. And then use azcopy in a job that running on on-premise node to download the output back to on premise

    If you have any problem, let us know


    Qiufang Shi

    Friday, September 23, 2016 2:26 AM
  • 1) I do not see "2012 R2" OS when creating template and

    2) It's not visible even when the template is edited; I have also verified that the OS is actually 2012 and not 2012 R2 in the 2 nodes that were created from the template by RDP-ing to them.

    (I am on *latest* HPC pack - 2012 R2 Update 3 with July QFE installed and I am the admin of cluster)

    (The certificate chosen is Default Azure management and I am trying this out on a MSDN VS Professional subscription)

    (I created the storage account/resource-group on "Classic" portal per tutorial on HPC azure site and all of it is on SouthCentralUS - could that be a reason ?)

    Please advise.

    

    • Edited by SRIRAM R Friday, September 23, 2016 3:59 PM
    Friday, September 23, 2016 3:24 PM
  • Hi, Sriram,

    Can you open HpcClusterManager, and click 'Help' menu and select 'About' to check the Server version and client version?

    HPC cluster support Windows server 2012 R2 from HPC 2012 R2

    Thanks,

    Yongjun

    Monday, September 26, 2016 2:24 AM
  • HPC Pack 2012 R2 with 

    Server Version: 4.5.5111.0

    Client Version: 4.5.5111.0

    Monday, September 26, 2016 2:33 AM
  • Hi, Sriram,

    we are trying to repro this issue on our local env.

    some more questions:

    1, how do you install/upgrade your HPC cluster,  what is the initial version of this HPC cluster?

    whether it is new installation of HPC 2012 R2 Update 3, then patch to latest QFE?

    Or you install some previous version, such as HPC 2012 at first, then patch one by one to latest version?

    2, can you send the %CCP_HOME%Conf\Microsoft.Ccp.ClusterModel.sdmDocument to me?

    Thanks,

    Yongjun

    Monday, September 26, 2016 2:51 AM
  • 1)  Install procedure:

    a) Installed HPC Pack 2012 R2 Update 3 on a brand new image of 2012 R2 OS

    b) Applied patches (July QFE)

    2) Requested file uploaded to https://1drv.ms/u/s!AnOkuFPvV8aAivJeEtXFVye_C_3SwQ

    Please advise.

    Monday, September 26, 2016 4:02 PM
  • Hi, Sriram,

    I just use the same steps to install one env, but cannot repro this issue, can you restart HpcManagement service on Head node and restart HpcClusterManager to try again?

    Tuesday, September 27, 2016 12:47 AM
  • No luck.

    Restarted the HPCManagement service and opened the existing Azure Node Template and the OS Family still has 2008 R2 and 2012; Do not see 2012 R2.


    Wednesday, September 28, 2016 2:45 PM
  • Hi, Sriram,

    we still cannot repro this issue, and the code logic is only check the server version, if server version larger than 2012 SP1, then include "2012 R2",

    So still need your help to investigate this issue:

    1, can you export your Azure node template and send to me, you can remove some security issue in the XML first

    2, can you send the HpcManagement log to me

    Thanks,

    Yongjun

    Thursday, September 29, 2016 1:08 AM
  • Here is the info you asked for:

    https://1drv.ms/f/s!AnOkuFPvV8aAivJfEtXFVye_C_3SwQ

    Thursday, September 29, 2016 1:51 AM
  • Any update on this.?
    Wednesday, October 19, 2016 7:38 PM
  • Hi, Sriram,

    I cannot repro this issue on our local environment after several try.

    Can you help to repro and collect log on head node?

    1, backup HpcClusterManager.exe under %CCP_HOME%bin

    2.download the private bits from http://yongjunc3.blob.core.windows.net/hpclogs/HpcClusterManager.exe and copy to %CCP_HOME%bin

    3, remove the existing log under %HOMEPATH%AppData\Local\Microsoft\Hpc\LogFiles\ClusterManager

    4, start HpcClusterManager.exe and open one Azure node template or create one new azure node template, check whether can see "2012 R2"

    5, send the log under %HOMEPATH%AppData\Local\Microsoft\Hpc\LogFiles\ClusterManager to me

    Thursday, October 20, 2016 4:06 AM
  • No luck. 

    Sending the files to hpcpack@microsoft.com

    Please advise.

    Thursday, October 20, 2016 4:02 PM
  • Hi, Sriram,

    From HpcClusterManager log, I can see that,  client connect to lco-tv-ctvengin, and the server model version is 4.0

    Can you ensure that lco-tv-ctvengin is the correct head node, and it is installed with HPC Pack 2012 R2 Update 3 and then with July QFE?

    Friday, October 21, 2016 1:08 AM
  • Yes, that is the correct headnode and here is the output from get-hpcpatchstatus.ps1:

    HeadNode role detected

    HPC Pack Version: 4.5
    Patch Status:

    ID        Status    Installed On Display Name
    --        ------    ------------ ------------
    KB3161422 Installed 2016/07/12   HotFix KB3161422 for Microsoft HPC Pack 2012 R2 Update 3

    Friday, October 21, 2016 1:57 AM
  • so what is your database configuration, you use local db or remote db, and after patch, do you do some manual operation on HPC database?

    Current, the problem is that some SDM document stored in db is old version, I don't know what is the cause?

    To recover this issue, you can do the following steps on head node:

    1, stop HpcManagement service

    2, go to C:\Program Files\Microsoft HPC Pack 2012\Conf, run the following command under cmd console (run as admin)

    sdml Microsoft.Ccp.ClusterModel.sdmDocument Microsoft.Ccp.ComputerModel.sdmDocument Microsoft.Ccp.TemplateModel.sdmDocument

    3, start HpcManagement service, then open HpcClusterManager, try again, it should resolve this issue, if the issue is still there

    please send the log under %HOMEPATH%AppData\Local\Microsoft\Hpc\LogFiles\ClusterManager to me, my email is yongtia@microsoft.com

    Friday, October 21, 2016 2:12 AM