none
Copying files from a compute node using IaaS script RRS feed

  • Question

  • I am using the IaaS script to set up a cluster.  I have created a VM image for the compute nodes; said image includes a file share which include a DLL and config file I wish to copy onto the head node.  I have a post-config script that I am trying to run to include this, but the files do not get copied over.  Note that if the commands are copied over manually to a PS window on the head node, the commands execute without issue.  The commands in question are as follows

    mkdir c:\TargetDir 

    $HPCPath = (Get-ChildItem Env:\CCP_HOME).Value + "ServiceRegistration"

    Copy-Item "\\ComputeNode\Share\*.dll" "C:\TargetDir"

    Copy-Item "\\ComputeNode\Share\*.config" $HPCPath

    I have commands both before and after the two copy-item lines that are run without issue.  I have tried running using the -PassThru and -Force flags, but I cannot catch an error.  It just simply fails silently.  Any thoughts?

    Monday, August 17, 2015 9:38 PM

Answers

  • Hi there are two reasons that stops you from copying the files.

    1. The compute node may be not ready yet when the post script got run.

    2. Even when the compute node is ready, you cannot directly copy from \\ComputeNode\Share, because the post script on the head node is triggered by a remote PowerShell call, it cannot directly access the netshare on the computer node because delegating authentication is forbidden by default. But you can work around it by explicitly specify the username and password with "net use" command.

    So please try the script as below:

    mkdir c:\TargetDir 
    $HPCPath = (Get-ChildItem Env:\CCP_HOME).Value + "ServiceRegistration"

    #Wait for the compute node to be ready
    while($true)
    {
        # Flush the DNS cache in case the cached compute node ip is wrong.
        Start-Process -FilePath ipconfig -ArgumentList "/flushdns" -Wait -NoNewWindow | Out-Null
        if(Test-Connection -ComputerName ComputeNode -Quiet)
        {
            break
        }
    }

    net use \\ComputeNode\Share adminPassword /User:domain\adminUsername
    Copy-Item "\\ComputeNode\Share\*.dll" "C:\TargetDir"
    Copy-Item "\\ComputeNode\Share\*.config" $HPCPath


    • Edited by Sunbin Zhu Tuesday, August 18, 2015 2:09 AM
    • Marked as answer by KWilliams1 Tuesday, August 18, 2015 5:44 PM
    Tuesday, August 18, 2015 2:06 AM

All replies

  • I think you try to deploy the SOA service configuration and assembly? If yes, I think you can use local folder on the compute node in the service registration file. Thus you only need:

    1. Make a service registration file that point to the local path on compute nodes, put the registration file in azure storage (Later time we will support azure files)

    2. Download the service registration file to service registration folder (In post configuration script).


    Qiufang Shi

    Tuesday, August 18, 2015 1:09 AM
  • Hi there are two reasons that stops you from copying the files.

    1. The compute node may be not ready yet when the post script got run.

    2. Even when the compute node is ready, you cannot directly copy from \\ComputeNode\Share, because the post script on the head node is triggered by a remote PowerShell call, it cannot directly access the netshare on the computer node because delegating authentication is forbidden by default. But you can work around it by explicitly specify the username and password with "net use" command.

    So please try the script as below:

    mkdir c:\TargetDir 
    $HPCPath = (Get-ChildItem Env:\CCP_HOME).Value + "ServiceRegistration"

    #Wait for the compute node to be ready
    while($true)
    {
        # Flush the DNS cache in case the cached compute node ip is wrong.
        Start-Process -FilePath ipconfig -ArgumentList "/flushdns" -Wait -NoNewWindow | Out-Null
        if(Test-Connection -ComputerName ComputeNode -Quiet)
        {
            break
        }
    }

    net use \\ComputeNode\Share adminPassword /User:domain\adminUsername
    Copy-Item "\\ComputeNode\Share\*.dll" "C:\TargetDir"
    Copy-Item "\\ComputeNode\Share\*.config" $HPCPath


    • Edited by Sunbin Zhu Tuesday, August 18, 2015 2:09 AM
    • Marked as answer by KWilliams1 Tuesday, August 18, 2015 5:44 PM
    Tuesday, August 18, 2015 2:06 AM
  • Net use was indeed the missing key in all of this.  Thank you! (I already have code to check that the node is up before starting, I simply edited it out)

    Thanks!

    Tuesday, August 18, 2015 5:45 PM