none
Create Node Template Wizard not recognizing storage account RRS feed

  • Question

  • I have a cluster up and running that was created via 'Create HPC Pack cluster for Excel workloads'. I want to enable 'burst' capability to scale up compute nodes when needed.

    To create a Node Template for burst scenarios I've followed the steps in https://technet.microsoft.com/en-us/library/gg481758(v=ws.11).aspx and become stuck on step 6. Provide Subscription Information comes up with a messagebox 'No service was found under this subscription'. Searching the problem led me to https://technet.microsoft.com/library/jj899624.aspx?f=255&MSPPError=-2147217396 and the possible lack of a Cloud Service. Creation of a Cloud Service solved the first issue and now the error I have is 'No storage account was found under this subscription.' There are already storage accounts that were created when the cluster pack was deployed. How can I use the accounts that were created to create the node template?

    Tuesday, August 1, 2017 6:04 PM

Answers

  • Hi IvenBach,

    HPC Pack needs classic storage account for Azure bursting. And recently Azure changed the default resource group name convention when you create a new classic storage account, which may be the cause of this problem.
    Please try to deploy a new classic storage account in a resource group named Default-Storage-<Location>, e.g. Default-Storage-WestEurope, and try Azure burst again.Thanks,
    Zihao

    • Marked as answer by IvenBach Monday, August 7, 2017 7:09 PM
    Thursday, August 3, 2017 8:34 AM

All replies

  • Hi IvenBach,

    HPC Pack needs classic storage account for Azure bursting. And recently Azure changed the default resource group name convention when you create a new classic storage account, which may be the cause of this problem.
    Please try to deploy a new classic storage account in a resource group named Default-Storage-<Location>, e.g. Default-Storage-WestEurope, and try Azure burst again.Thanks,
    Zihao

    • Marked as answer by IvenBach Monday, August 7, 2017 7:09 PM
    Thursday, August 3, 2017 8:34 AM
  • I was able to create the node template. I added a new node to the cluster and brought it online. 

    I ran a test with a small sample run and encountered the error System.IO.FileNotFoundException(). Is there any way to have the Azure nodes default to the storage account created by the Azure Resouce Manager (ARM)? Would the resource group have to default to the naming convention you provided above?
    • Edited by IvenBach Monday, August 7, 2017 10:04 PM
    Monday, August 7, 2017 9:52 PM
  • Hi IvenBach,

    Can you provided the detail of FileNotFoundException?

    HPC Pack cannot use storage account created by ARM. Please use ASM and follow the naming convention.

    Thanks,
    Zihao

    Tuesday, August 8, 2017 2:36 AM
  • When the AzureNode wasn't online the offloading worked without issue. When the node was brought Online that was when the error occurred. he FileNotFoundException had the message and callstack as follows:

    Unable to find workbook in package <filename>, or in workbookpath \\<headNode>Runtime$\SOA\UserJobs\<domain>.<username>\7\<filenameWithExtension>. Make sure the workbook is correctly deployed to Windows Azure nodes.   File name: '<filename>'   at Microsoft.HPC.Excel.ExcelService.AzureRetryWithWorkbookPath(string workbookPath, Nullable`1 lastSaveDate, String packageWorkbookPath)    at    Microsoft.Hpc.Excel.ExcelService.OpenWorkbookAnAzure(String workbookPath, Nullable`1 lastSaveDate)     at   Microsoft.Hpc.Excel.ExcelService.Calculate(String macroName, Byte[] inputs, Nullable`1 lastSaveDate)

    A second error of "Experienced an exception while processing cluster responses.   Microsoft.HPC.Excel.ExcelClientException: Experienced an exception while processing cluster responses.   at   Microsoft.HPC.Excel.ExcelClient.Run(Boolean executeLocally)" also followed.

    ARM = https://portal.azure.com and ASM = https://manage.windowsazure.com/ as explained from https://blogs.technet.microsoft.com/meamcs/2016/12/22/difference-between-azure-service-manager-and-azure-resource-manager/. Double checking to ensure I'm going to the correct portal.


    • Edited by IvenBach Tuesday, August 8, 2017 7:05 PM
    Tuesday, August 8, 2017 7:00 PM
  • Hi IvenBach,

    As now the new Azure portal https://portal.azure.com can manage classic storage accounts as well, you don't need go back to the old portal https://manage.windowsazure.com/ any more. Just select "Classic" in the "Deployment model" option when creating new storage account will do the trick.

    For now we don't officially support run Excel workbook offloading in PaaS node. You can build an Azure IaaS cluster instead. Please read following instruction.
    https://docs.microsoft.com/en-us/azure/virtual-machines/windows/excel-cluster-hpcpack

    Thanks,
    Zihao



    Wednesday, August 9, 2017 1:50 AM
  • That article was the one I followed to get the cluster up and running. By creating a cluster in this manner there isn't the ability to enable burst nodes then?
    Wednesday, August 9, 2017 7:08 PM
  • Hi IvenBach,

    You can always do Azure Burst. The nodes you created by doing Azure Burst is so called "PaaS Nodes". And they can not do Excel workload offloading.

    If you followed this article, you would have already deployed Compute Nodes (also known as "IaaS Nodes") with image ComputeNodeWithExcel. Please use those nodes for Excel workload offfloading.

    Thanks,
    Zihao

    Thursday, August 10, 2017 1:32 AM
  • I see those nodes when I create the cluster. Offloading to perform the calculations works without issue.

    If, for example, I have a head node and 5 compute nodes in the cluster I can send requests to it. If the request is a small number 500 then the number of compute nodes is sufficient to calculate in a timely manner. However, if I were to send of a much larger request of 500,000 it would take 1,000 times as long. From your response adding burst nodes because they are 'Paas' nodes won't work. Can 'Iaas' nodes be added in a similar way to decrease the time needed for calculation?

    Friday, August 11, 2017 5:00 PM
  • Hi IvenBach,

    Bursting IaaS nodes is going to be a new feature in coming HPC Pack 2016 Update 1. For now, please consider using auto grow shrink

    https://docs.microsoft.com/en-us/azure/virtual-machines/windows/classic/hpcpack-cluster-node-autogrowshrink

    Thanks,
    Zihao

    Saturday, August 12, 2017 3:42 AM
  • Reading some more I came across https://social.microsoft.com/Forums/en-US/74eb01f2-cf57-4c70-a662-c5ea568132f7/azure-hpc-cluster-autogrowshrink-not-triggering?forum=windowshpcitpros.

    'Hpc auto grow shrink cannot auto add compute nodes to cluster, you need add the nodes to your HPC IaaS cluster first, then if you enable auto grow shrink, it can stop the nodes when no job, and start the nodes when new job is coming.' This sets the node(s) in a Stopped(Deallocated) state. Using the Cluster Manager to 'Take Offline' would be a Stopped state that leaves the node Allocated and still incurring a charge?

    • Edited by IvenBach Tuesday, August 15, 2017 9:49 PM
    Tuesday, August 15, 2017 7:39 PM
  • "Take offline" operation won't move the node to stopped state, this is for admin to do node maintaining operations like install applications, etc.

    If you apply this QFE(https://www.microsoft.com/en-us/download/details.aspx?WT.mc_id=rss_alldownloads_all&id=55650), you shall be able to use the built-in auto grow shrink we back ported from HPC Pack 2016.

    In HPC Pack 2016 Update 1, you are able to add IaaS nodes through node template instead of using ARM template to deploy.


    Qiufang Shi

    Wednesday, August 16, 2017 1:24 AM
  • Qiufang,

    Per 'Install Instructions' directions I checked the version of the Cluster Manager and it is already on 4.5.5168.0 for both the Server and Client version.

    On step 4, as described in https://social.microsoft.com/Forums/en-US/c9f4032f-fe10-43aa-b136-ec5be60a5f03/configarmautogrowshrinkcert-tenantid-parameter-not-found?forum=windowshpcitpros#c9f4032f-fe10-43aa-b136-ec5be60a5f03, 'TenantId' was omitted this time. I received the error 'Failed to assign 'Contributor' role for the service Principal, try again...'. Adding '$_' to the script and running it again resulted in 'You had configured the certificate before'.

    I tried to continue but 'Set-HpcClusterProperty' isn't recognized as a cmdlet. How should I proceed?



    • Edited by IvenBach Wednesday, August 16, 2017 9:42 PM
    Wednesday, August 16, 2017 9:31 PM
  • Hi Iven,

    to use "set-hpcClusterProperty" you have to "add-pssnapin Microsoft.hpc" first.

    And could you reach to "hpcpack@microsoft.com" on what you are trying to accomplish so that we can give you more appropriate suggestion?


    Qiufang Shi

    Friday, August 18, 2017 9:25 AM