none
HPC Pack SP2 - Soft Card Configuration

    Question

  • What is the complete method for setting up soft card jobs under service pack 2?  The documentation from http://technet.microsoft.com/en-us/library/hh184316(WS.10).aspx seems incomplete.  When I try the following I get an access denied.

     

    C:\Windows\System32>hpccred createcert /scheduler:headnode

    Enrolled in certificate with Thumbprint 57DA0C966677867D8F7A08FBEB0D22F542D93C56

    C:\Windows\System32>Job submit /scheduler:headnode echo hello

    Enter '1' to provide a password or '2' to provide an HPC SoftCard:

    2

    Cryptography error:Access denied.

    What am I missing?

    Thanks,

    Rob

    Friday, August 5, 2011 2:55 PM

Answers

  • Just to post the answer, in case someone else comes across this issue, we talked with chcrall and Lukasz, who were able to help us find the issue.  It turns out the System account needs Full Control to the following two folders, and should also be the Owner.  For some reason in our configuration, that wasn't the case, and it was breaking SoftCard logon.

    C:\ProgramData\Microsoft\Crypto\SystemKeys

    C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18

    • Marked as answer by Rob-HPC Tuesday, August 9, 2011 4:45 PM
    Tuesday, August 9, 2011 4:45 PM

All replies

  • Hi Rob,

    I have a few questions for you:

    1. Could you take a look at newly generated certificate (with certmgr for example) and tell us:
     - what's the value for 'Enhanced Key Usage' field?
     - does it have corresponding private key, which is exportable?

    2. Are you trying this on the client machine or directly on the headnode?

    3. On the headnode, do you see any related error events in Event Viewer -> Application and Services Logs -> Microsoft -> HPC -> Scheduler -> Operational ?

    Thanks,
    Łukasz

    Friday, August 5, 2011 6:39 PM
  • Lukasz pointed out the "exportable" attribute.  If the key cannot be exported on the client, I believe you will get this access denied error.  If the key isn't exportable, go back to the template on the CA you are using to generate the certificate and make sure it is exportable in the template.

    Right click on the template -> Manage

    Go to the extensions tab -> Request handling

    set "Allow private key to be exported"

     

    Chris

    Friday, August 5, 2011 9:12 PM
  • Hi Chris & Lukasz,

    Just wanted to let you know that Rob and I are working on the same problem. I also put in a question but in the Job Submission forum. I didn't realize Rob and already posed the question. I just posted some log entries there.

    The cert template is base on the Smartcard Logon template. The changes made were to allow the private key to be exported and authenticated users to enroll and auto-enroll. The CAPI logs on the server that we run the hpccred setcreds -softcard shows the the private keys were exported.

    We have tested this both from the head node directly and from one of our session host nodes. All running Windows 2008 R2 SP2. We get the same behavior in both cases.

    Thanks,

    Pat


    Pat Collins
    Friday, August 5, 2011 10:06 PM
  • Hi Lukasz,

     

    Here are the screen shots of the configuration that you asked for.

    1. The Enhanced Key Usage Field is as follows:

    This is a look from the headnode, the Private Key is available, and exportable.  I did a test export just to make sure, and that was configured in the Cert Template.

     

    2. We've tried this on both the Headnode, and a client machine.  We get the same results from both.  Here is a PS screenshot from the Headnode.  (ahh.. 2 image limitation per message.

     

    Monday, August 8, 2011 6:22 PM
  • 2. We've tried this on both the Headnode, and a client machine.  We get the same results from both.  Here is a PS screenshot from the Headnode.

    3. Yes, here are the messages related to the issue that I'm seeing in the Event log:

     

     

    - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    - <System>
     <Provider Name="Microsoft-HPC-Scheduler" Guid="{5B169E40-A3C7-4419-A919-87CD93F2964D}" /> 
     <EventID>8</EventID> 
     <Version>0</Version> 
     <Level>2</Level> 
     <Task>0</Task> 
     <Opcode>0</Opcode> 
     <Keywords>0x8000000000000000</Keywords> 
     <TimeCreated SystemTime="2011-08-08T18:05:55.418323500Z" /> 
     <EventRecordID>1138650</EventRecordID> 
     <Correlation /> 
     <Execution ProcessID="3788" ThreadID="8096" /> 
     <Channel>Microsoft-HPC-Scheduler/Operational</Channel> 
     <Computer>xxxxxxxxxxxxxxxx</Computer> 
     <Security UserID="S-1-5-18" /> 
     </System>
    - <EventData>
     <Data Name="Message">Access denied.</Data> 
     <Data Name="ExceptionString">Exception detail: System.Security.Cryptography.CryptographicException: Access denied. at System.Security.Cryptography.CryptographicException.ThrowCryptogaphicException(Int32 hr) at System.Security.Cryptography.X509Certificates.X509Utils._LoadCertFromBlob(Byte[] rawData, IntPtr password, UInt32 dwFlags, Boolean persistKeySet, SafeCertContextHandle& pCertCtx) at System.Security.Cryptography.X509Certificates.X509Certificate.LoadCertificateFromBlob(Byte[] rawData, Object password, X509KeyStorageFlags keyStorageFlags) at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password) at Microsoft.Hpc.Scheduler.Store.SchedulerStoreInternal.VerifyCertificate(String unencryptedPassword, Byte[] certificate, String userSid, WindowsIdentity ownerIdentity) Current stack: at Microsoft.Hpc.Scheduler.SchedulerTracing.TraceException(String facility, Exception exception) at Microsoft.Hpc.Scheduler.Store.SchedulerStoreInternal.VerifyCertificate(String unencryptedPassword, Byte[] certificate, String userSid, WindowsIdentity ownerIdentity) at Microsoft.Hpc.Scheduler.Store.SchedulerStoreInternal.SetUserNamePasswordCertificate(ConnectionToken token, String userName, String unencryptedPassword, Nullable`1 reusable, Byte[] certificate) at Microsoft.Hpc.Scheduler.Store.SchedulerStoreInternal.SaveCertificate(ConnectionToken token, String username, String password, Nullable`1 reusable, Byte[] certificate) at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs) at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg, Int32 methodPtr, Boolean fExecuteInContext) at System.Runtime.Remoting.Messaging.ServerObjectTerminatorSink.SyncProcessMessage(IMessage reqMsg) at System.Runtime.Remoting.Messaging.ServerContextTerminatorSink.SyncProcessMessage(IMessage reqMsg) at System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessageCallback(Object[] args) at System.Runtime.Remoting.Channels.ChannelServices.DispatchMessage(IServerChannelSinkStack sinkStack, IMessage msg, IMessage& replyMsg) at Microsoft.Hpc.Scheduler.Store.ServiceAsClientServerSink.ProcessMessage(IServerChannelSinkStack sinkStack, IMessage requestMsg, ITransportHeaders requestHeaders, Stream requestStream, IMessage& responseMsg, ITransportHeaders& responseHeaders, Stream& responseStream) at System.Runtime.Remoting.Channels.BinaryServerFormatterSink.ProcessMessage(IServerChannelSinkStack sinkStack, IMessage requestMsg, ITransportHeaders requestHeaders, Stream requestStream, IMessage& responseMsg, ITransportHeaders& responseHeaders, Stream& responseStream) at System.Runtime.Remoting.Channels.Tcp.TcpServerTransportSink.ServiceRequest(Object state) at System.Runtime.Remoting.Channels.SocketHandler.ProcessRequestNow() at System.Runtime.Remoting.Channels.SocketHandler.BeginReadMessageCallback(IAsyncResult ar) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken) at System.Net.Security.NegotiateStream.ProcessFrameBody(Int32 readBytes, Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.NegotiateStream.ReadCallback(AsyncProtocolRequest asyncRequest) at System.Net.FixedSizeReader.CheckCompletionBeforeNextRead(Int32 bytes) at System.Net.FixedSizeReader.ReadCallback(IAsyncResult transportResult) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at System.Threading.ExecutionContext.runTryCode(Object userData) at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Net.ContextAwareResult.Complete(IntPtr userToken) at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken) at System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped) at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)</Data> 
     </EventData>
     </Event>
    

     

    - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    - <System>
     <Provider Name="Microsoft-HPC-Scheduler" Guid="{5B169E40-A3C7-4419-A919-87CD93F2964D}" /> 
     <EventID>0</EventID> 
     <Version>0</Version> 
     <Level>4</Level> 
     <Task>1</Task> 
     <Opcode>0</Opcode> 
     <Keywords>0x8000000000000000</Keywords> 
     <TimeCreated SystemTime="2011-08-08T18:05:51.955101300Z" /> 
     <EventRecordID>1138649</EventRecordID> 
     <Correlation /> 
     <Execution ProcessID="3788" ThreadID="8120" /> 
     <Channel>Microsoft-HPC-Scheduler/Operational</Channel> 
     <Computer>xxxxxxxxxxxxxxxxxxxxxxxx</Computer> 
     <Security UserID="S-1-5-18" /> 
     </System>
    - <EventData>
     <Data Name="Id">1579</Data> 
     <Data Name="Name" /> 
     <Data Name="Owner">xxxxxxxxxxx</Data> 
     <Data Name="Template">Default</Data> 
     <Data Name="Priority">Normal</Data> 
     </EventData>
     </Event>
    



    Monday, August 8, 2011 6:26 PM
  • Just to post the answer, in case someone else comes across this issue, we talked with chcrall and Lukasz, who were able to help us find the issue.  It turns out the System account needs Full Control to the following two folders, and should also be the Owner.  For some reason in our configuration, that wasn't the case, and it was breaking SoftCard logon.

    C:\ProgramData\Microsoft\Crypto\SystemKeys

    C:\ProgramData\Microsoft\Crypto\RSA\S-1-5-18

    • Marked as answer by Rob-HPC Tuesday, August 9, 2011 4:45 PM
    Tuesday, August 9, 2011 4:45 PM