Upon investigating, it seems to be happening when the nodepreparecommandline fails. in my case, i get an exit code of 267 and the session/hpc puts the nodes instantly into exclusion list.
this is happening when i am using azcopy command to copy files from blob storage container to IaaS noded as preparnodecommandline. What's weird is i can RDP into IaaS node and run the same azcopy command and it works just fine. apparently, preparenodecommandline
cannot create a directory if it does not exist on the IaaS node using azcopy command. here is the command that i used:
prepareNodeCommandLine="AzCopy /Source:https://????.blob.core.windows.net/testcontainer/testsoa /Dest:C:\Users\Public\TestHPCService\TestSOA /SourceKey:<key> /S /MT /XO /Y"
Any pointers why this may be happening
I am on 2016 Update 2.