Remove From My Forums

How to catch the request for username/password?

Question
0

Sign in to vote

Hi,

We're experimenting with HPC (currently running the 2016 version).

Using a C# desktop application we're using DurableSession.CreateSession() to create a session for a user-task, which then sends some tasks to HPC in a specific order. This works, however every user must login to the headnode to run a single task job so that the prompt for username/password is shown and they can mark the 'Remember this password?' question.

This is not a viable situation however, because we don't want to give access to the headnode VM to all users.

I've read that DurableSession.CreateSession() should ask for the username/password (https://blogs.technet.microsoft.com/windowshpc/2013/03/14/hpc-pack-soa-tutorial-i-write-your-first-soa-service-and-client/), but this does not happen correctly I think.

If the call to CreateSession happens in the main thread the application just 'hangs' on a dispatched wait event (no pop-ups or anything appear),and if the call to CreateSession happens in another thread the following string appears in my output:

Enter the password for 'Domain\User' to connect to 'hpcheadnode': /N))Remember this password? (Y/N)member this password? (Y/N)mber this password? (Y/N)er this password? (Y/N) this password? (Y/N)his password?

and this string grows with several partial "Remember this password? (Y/N) per second.

In my opinion this means that the HPC cluster is set up correctly, but what are we doing wrong in that we do not actually see the prompt?

The basis for our code is:

var info = new SessionStartInfo(headNode, ServiceName)
   {
       ServiceJobName = jobName
   };
//info.UseWindowsClientCredential = true;
var jobTemplate = Options.JobTemplate;
if (!string.IsNullOrEmpty(jobTemplate))
{
   info.JobTemplate = jobTemplate;
}
return DurableSession.CreateSession(info);

Where the info.UseWindowsClientCredential is not used because it has been added in HPC 2016 but we cannot find any information about it, and it locked me out of my account in <1 second.

And we can test with the same user that works in our 'before' case by using 'hpccred delcreds /user:', so I know this user has the required permissions to run the job.

I sincerely hope anybody can help us get this authentication working, or at least point us in the right direction.

Thank you in advance,

Sjoerd

Tuesday, June 20, 2017 3:36 PM

Answers

0

Sign in to vote
Hi Sjoerd,

You may either use Set-HpcSoaCredential powershell cmdlet to save the client credential, or set startInfo.UserName and startInfo.Password in Session API client code each time creating a session.

startInfo.UseWindowsClientCredential is used when a non-domain joined client machine is talking to a domain joined head node with Net.Tcp binding. hpccred is used for submitting scheduler jobs not for creating SOA sessions.

Regards,

Yutong Sun
- Marked as answer by SEgmond Friday, July 7, 2017 9:27 AM
Wednesday, June 21, 2017 7:51 AM

All replies

0

Sign in to vote
Hi Sjoerd,

You may either use Set-HpcSoaCredential powershell cmdlet to save the client credential, or set startInfo.UserName and startInfo.Password in Session API client code each time creating a session.

startInfo.UseWindowsClientCredential is used when a non-domain joined client machine is talking to a domain joined head node with Net.Tcp binding. hpccred is used for submitting scheduler jobs not for creating SOA sessions.

Regards,

Yutong Sun
- Marked as answer by SEgmond Friday, July 7, 2017 9:27 AM
Wednesday, June 21, 2017 7:51 AM
0

Sign in to vote

Hi Yutong Sun,

Thank you for your fast response.

I've been trying out your solution with Set-HpcSoaCredential, because we don't want to ask the user for its credential every time. But I'm getting in trouble with it.

First I tried to run that command locally (on the client/application computer), this generates the following error:

Exception thrown: 'System.ServiceModel.EndpointNotFoundException' in Microsoft.Hpc.Scheduler.Session.dll

Additional information: Could not connect to net.tcp://hpcheadnode:9091/3224/NetTcp/GetResponse. The connection attempt lasted for a time span of 00:00:21.0033805. TCP error code 10060: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond <IP address of hpcheadnode>:9091.

where my debugging information gives the following information:

Information: 0 : Calling resolve/singleton on HPCHEADNODE:8939 with parameter SchedulerStatefulService.
Information: 0 : resolve/singleton on HPCHEADNODE:8939 with parameter SchedulerStatefulService returned RDTHPC04.
Information: 0 : [WcfProxy] Begin to create wcf proxy to net.tcp://hpcheadnode:5800/SchedulerStoreService
Information: 0 : [WcfProxy] End to create wcf proxy

is there something wrong with my port numbers in this case? Can I set the port numbers somewhere? I cannot find anything about that in the Set-HpcSoaCredential documentation at: https://technet.microsoft.com/en-us/library/dn887855.aspx

Secondly I tried to run the Set-HpcSoaCommand on the head node because it didn't work on the client, but now my entire cluster is unresponsive.

The cluster manager now gives the following exception: There has an exception happens: Value cannot be null.
Parameter name: dnsName

And both the head nodes and compute nodes have an error in Node Connectivity "HPC Management Service or Node Manager Service unreachable"

There are also exceptions in the server manager log:

Application: HpcSessionStateful.exe
Framework Version: v4.0.30319
Description: The application requested process termination through System.Environment.FailFast(string message).
Message: RunAsync failed due to an unhandled exception causing the host process to crash: System.ArgumentNullException: Value cannot be null.
Parameter name: dnsName
   at Microsoft.Hpc.Scheduler.Session.Internal.Common.CommonSchedulerHelper.<GetScheduler>d__0.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Hpc.Scheduler.Session.Internal.SessionLauncher.BrokerNodesManager..ctor()
   at Microsoft.Hpc.Scheduler.Session.Internal.LauncherHostService.LauncherHostService.<OpenService>d__9.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.Hpc.Scheduler.Session.Internal.SessionLauncher.SessionLauncherStatefulService.<RunAsync>d__2.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.ServiceFabric.Services.Runtime.StatefulServiceReplicaAdapter.<ExecuteRunAsync>d__f.MoveNext()
Stack:
   at System.Environment.FailFast(System.String)
   at Microsoft.ServiceFabric.Services.Runtime.ServiceHelper.HandleRunAsyncUnexpectedException(System.Fabric.IServicePartition, System.Exception)
   at Microsoft.ServiceFabric.Services.Runtime.StatefulServiceReplicaAdapter+<ExecuteRunAsync>d__f.MoveNext()
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action, Boolean, System.Threading.Tasks.Task ByRef)
   at System.Threading.Tasks.Task.FinishContinuations()
   at System.Threading.Tasks.Task`1[[System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].TrySetResult(Boolean)
   at System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1[[System.Boolean, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].SetResult(Boolean)
   at Microsoft.ServiceFabric.Services.Runtime.StatefulServiceReplicaAdapter+<WaitForWriteStatusAsync>d__18.MoveNext()
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Runtime.CompilerServices.AsyncMethodBuilderCore+MoveNextRunner.Run()
   at System.Threading.Tasks.AwaitTaskContinuation.RunOrScheduleAction(System.Action, Boolean, System.Threading.Tasks.Task ByRef)
   at System.Threading.Tasks.Task.FinishContinuations()
   at System.Threading.Tasks.Task`1[[System.Threading.Tasks.VoidTaskResult, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]].TrySetResult(System.Threading.Tasks.VoidTaskResult)
   at System.Threading.Tasks.Task+DelayPromise.Complete()
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.TimerQueueTimer.CallCallback()
   at System.Threading.TimerQueueTimer.Fire()
   at System.Threading.TimerQueue.FireNextTimers()
and
Faulting application name: HpcSessionStateful.exe, version: 5.0.5826.0, time stamp: 0x584e991c
Faulting module name: unknown, version: 0.0.0.0, time stamp: 0x00000000
Exception code: 0x80131623
Fault offset: 0x00007ff882811290
Faulting process id: 0xe5c
Faulting application start time: 0x01d2ea79a5bbc2b9
Faulting application path: C:\ProgramData\SF\_Node_0\Fabric\work\Applications\HpcApplicationType_App0\SessionLauncherStatefulServicePkg.Code.1.0.0\HpcSessionStateful.exe
Faulting module path: unknown
Report Id: 2593913c-8bce-4315-b975-ee93e45d6ce5
Faulting package full name:
Faulting package-relative application ID:

Even rebooting all head and compute nodes does not solve this problem, even after executing Remove-HpcSoaCredential.

Is there any way to revert these actions? Or is there some documentation somewhere on how to fix this?

Thank you in advance,

Sjoerd

Wednesday, June 21, 2017 10:44 AM
0

Sign in to vote

Hi Sjoerd,

What's the setup configuration of the HPC Pack 2016 cluster? Is it 3 headnode on-prem cluster with domain joined? I did not simply repro the SOA credential issue locally. After running Set-HpcSoaCredential -Credential <domain>\<username> either from a client machine or on one of the 3 head nodes, creating SOA sessions with the build-in EchoClient.exe would succeed without asking for the credential of the user.

With regard to the cluster unresponsive (to be specific what hangs?), suppose it may not be related to the Set-HpcSoaCredential cmdlet, we need more logs to investigate though. Could you send us the logs under %CCP_HOME%Data\LogFiles\SOA on all the head nodes via email yutongs@microsoft.com?

Regards,

Yutong Sun

Friday, June 23, 2017 9:37 AM
0

Sign in to vote

Hi Yutong Sun,

We have 3 VM's on-premise within our domain for testing. One VM is the head node and scheduler, the other two are compute nodes. I've asked my IT guy to look into the problem, but so far he hasn't been able to solve the problems. He did mention that there are new DNS servers since recently, but that he did update the HPC VM's to point to the correct DNS servers.

We've also requested new machines for a second test cluster. Once we get them (or our current cluster is fixed) I'll be able to continue with my original problem.

I'll send the logs to your e-mail.

Thank you for helping,

Sjoerd

Tuesday, June 27, 2017 12:38 PM
0

Sign in to vote

Hi Yutong Sun,

Thank you very much for all your help. In the end it seems that the problems I encountered after your first answer were caused by other factors. So I'll mark the usage of Set-HpcSoaCredential as the answer to my original question.

Furthermore for other people with a similar question I'd like to add that regretfully there is no Check-HpcSoaCredential or similar function that can verify if the user is already registered. The only way to check is to start a session and check if it proceeds or 'hangs' in a Configuring state (or have another custom application keep track of who is registered or not).

Thanks for everything,

Sjoerd

Friday, July 7, 2017 9:26 AM
0

Sign in to vote

Changed the WCF service to just run as local system account instead of specifying it in the service panel.

Sunday, November 10, 2019 6:17 PM

Answered by:

How to catch the request for username/password?

Question

Answers

All replies