none
HPC Pack 2016 Update 2 Backward Compatibility RRS feed

  • Question

  • Hi

    It says here: https://docs.microsoft.com/en-us/powershell/high-performance-computing/what-s-new-in-hpc-pack-2016-update-1?view=hpc16-ps#backward-compatibility


    that HPC Pack Update 1 can connect to previous versions of an HPC Pack Cluster. 

    I'm trying to connect to an HPC Pack 2012 Cluster with HPC 2016 Update 2 Cluster Manager

    This fails with the message

    "Unable to connect to the head node"

    and the following detail:

    The connection to the management service failed. detail error: Microsoft.Hpc.RetryCountExhaustException: Retry Count of RetryManager is exhausted. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
       at System.Net.Sockets.Socket.EndReceive(IAsyncResult asyncResult)
       at System.Net.Sockets.NetworkStream.EndRead(IAsyncResult asyncResult)
       --- End of inner exception stack trace ---
       at System.Net.TlsStream.EndWrite(IAsyncResult asyncResult)
       at System.Net.ConnectStream.WriteHeadersCallback(IAsyncResult ar)
       --- End of inner exception stack trace ---
       at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
       at System.Net.Http.HttpClientHandler.GetResponseCallback(IAsyncResult ar)
       --- End of inner exception stack trace ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.HttpClientExtension.<>c__DisplayClass5_0.<<GetHttpApiCallAsync>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.RetryManager.<InvokeWithRetryAsync>d__33`1.MoveNext()
       --- End of inner exception stack trace ---
       at Microsoft.Hpc.RetryManager.<InvokeWithRetryAsync>d__33`1.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.HttpClientExtension.<GetHttpApiCallAsync>d__5.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.HpcFabricRestContext.<>c__DisplayClass14_1`1.<<GetFirstHttpApiCallResponseAsync>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.HpcFabricRestContext.<GetFirstHttpApiCallResponseAsync>d__14`1.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.HpcFabricRestContext.<ResolveSingletonServicePrimaryAsync>d__10.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.ServiceResolverExtension.<ResolveSdmNodeAsync>d__18.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.SystemDefinitionModel.Store.SdmRemoteStore.SdmRemoteStoreConnector.<ConnectAsync>d__2.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.SystemDefinitionModel.Store.SdmRemoteStore.<ConnectAsync>d__58.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.SystemDefinitionModel.Store.SdmRemoteStore.CheckConnection()
       at Microsoft.SystemDefinitionModel.Store.SdmRemoteStore.<>c__DisplayClass45_0.<QueryDocuments>b__0()
       at Microsoft.SystemDefinitionModel.Store.SdmRetry.<>c__DisplayClass4_0`1.<InvokeWithRetry>b__0()
       at Microsoft.Hpc.RetryManager.<InvokeWithRetryAsync>d__33`1.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.SystemDefinitionModel.Store.SdmRetry.InvokeWithRetry[T](Func`1 function, CancellationToken cancellationToken)
       at Microsoft.SystemDefinitionModel.ModelDocuments.LoadAllDocuments()
       at Microsoft.SystemDefinitionModel.DefinitionSpaceView.TryResolve(String simpleName)
       at Microsoft.SystemDefinitionModel.ModelDocuments..ctor(Model model)
       at Microsoft.SystemDefinitionModel.Model.InitializeModel()
       at Microsoft.ComputeCluster.Management.ClusterManager.ConnectCore(Model model)
       at System.Threading.Tasks.Task.Execute()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.RetryManager.<>c__DisplayClass34_0.<<InvokeWithRetryAsync>b__0>d.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.RetryManager.<InvokeWithRetryAsync>d__33`1.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.Hpc.RetryManager.<InvokeWithRetryAsync>d__34.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.ComputeCluster.Management.ClusterManager.<ConnectAsync>d__90.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.ComputeCluster.Management.ClusterManager.<ConnectAsyncWithModel>d__89.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.ComputeCluster.Management.ClusterManager.<ConnectAsync>d__87.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
       at Microsoft.ComputeCluster.Admin.ConnectionManager.<ConnectManagementService>b__10_0(Task`1 t)

    Is this because the Cluster Manager isn't backward compatible, only the SDK? (I can't manage to make the connection with code built against the dlls in the SDK either).

    Any ideas how I can make this work please?

    Thanks

    Monday, February 25, 2019 5:06 PM

All replies

  • Hi Bryan Hanson,

    The back compatibility support in HPC Pack 2016 for previous HPC Pack 2012/R2 is limited to job feature, not including resource management. So you may open HPC Job Manager GUI in HPC Pack 2016 connecting to HPC Pack 2012/R2 clusters, but not HPC Cluster Manager. SDK is also back compatibility supported.

    Regards,

    Yutong Sun

    Thursday, February 28, 2019 3:04 AM
  • Hello Yutong,

    A follow-up to the question above - I've been searching the forums, but can't seem to find the answer to it. I have a dead simple C# console application that's using Microsoft.HPC.SDK (5.2.6283) NuGet package and essentially does the following:

                var headnodeAddress = "<headnode IP address>";
                var valuationServiceName = "ServiceName";

                SessionStartInfo sessionStartInfo = new SessionStartInfo(headnodeAddress, valuationServiceName);

                using (Session session = Session.CreateSession(sessionStartInfo))
                {
                    Console.WriteLine("Connected to '{0}'", session.EndpointReference.ToString());
                }

    It attempts to connect to HPC 2012 R2 instance / head node (Server version: 4.4.4864.0) - the application fails with the following exception:

    > System.AggregateException: One or more errors occurred. ---> System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a send. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. --->

    The exact same code does not work even directly from the 2012 head node (manually copy the .exe / run it), but it does work fine from the 2016 Update 2 head node. 

    Hence, two questions:

    1. Is the SDK backward compatible? 

    2. If so, is there something special / different approach required to get this working against the 2012 R2 version?

    Thanks in advance!


    Thursday, March 7, 2019 8:43 PM
  • Hi,

      Currently SOA is not backward compatible to previous version while the JOB API part is. You need use V4 SDK for HPC Pack 2012 R2.


    Qiufang Shi

    Friday, March 8, 2019 7:59 AM
  • Hello Qiufang,

    Thanks for the prompt response. So it looks like we'll need to maintain two versions of our client codebase (we have 2012 R2 and now 2016 cluster in place as well) with two separate SDKs, doesn't look like there's a way around it at the moment.

    Is that the correct understanding? Is there a plan to have a single SDK compatible with both, SOA-wise, or is that not on the roadmap at the moment?

    Thanks!

    Friday, March 8, 2019 1:34 PM
  • Hi Phil,

      We will take a check on this for HPC Pack 2016 Update 3 but no promise yet. 


    Qiufang Shi

    Monday, March 11, 2019 8:22 AM
  • Thanks Qiufang - got it, all makes sense.
    Monday, March 11, 2019 2:50 PM
  • Changed the WCF service to just run as local system account instead of specifying it in the service panel.
    Sunday, November 10, 2019 6:18 PM