locked
Softcard authentication for HPC job submission is not working. RRS feed

  • Question

  • We are running Windows HPC Server 2008 R2 SP2 and are trying to get softcard authentication for job submission working. We followed the procedures outlined in http://technet.microsoft.com/en-us/library/hh184316(WS.10).aspx#BKMK_softcard but no luck yet. We can successfully create certificates with "hpccred createcert". The problem is when we try to cache the credentials by running "hpccred setcreds -softcard /scheduler:<name>" we get "Cryptography error:Access denied." We see this same error message in the HPC Scheduler logs in the event viewer on the head node. We also observe an "Audit Failure" in the security event logs on the head node for a "System Integrity" task associated with the Microsoft Key Storage Provider whenever we try to cache the credentials.

    One conjecture is that the system is trying to write to the user's credential store but cannot due to permissions, hence the "Access Denied." If anyone can shed some light on this we would greatly appreciate it.
    Friday, August 5, 2011 5:11 PM

All replies

  • Hi Pat,

    Could you share full content of the error event messages you're getting in the event viewer?

    Also could you take a look at created certificate and tell us what's the content of 'Enhanced Key Usage' field and if it contains a private key?

    Regards,
    Łukasz

    Friday, August 5, 2011 8:11 PM
  • Hi Lukasz,

    Thank you for the response. Here is the information you asked for. I did edit the system name and url for security reasons.

    HPC Scheduler log after running

    hpccred setcreds -softcard /scheduler:headnode
    Cryptography error:Access denied.

    HPC Scheduler log:

    - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    - <System>
      <Provider Name="Microsoft-HPC-Scheduler" Guid="{5B169E40-A3C7-4419-A919-87CD93F2964D}" />
      <EventID>8</EventID>
      <Version>0</Version>
      <Level>2</Level>
      <Task>0</Task>
      <Opcode>0</Opcode>
      <Keywords>0x8000000000000000</Keywords>
      <TimeCreated SystemTime="2011-08-05T21:09:02.141372300Z" />
      <EventRecordID>1138593</EventRecordID>
      <Correlation />
      <Execution ProcessID="3384" ThreadID="11980" />
      <Channel>Microsoft-HPC-Scheduler/Operational</Channel>
      <Computer>headnode.cluster.local</Computer>
      <Security UserID="S-1-5-18" />
      </System>
    - <EventData>
      <Data Name="Message">Access denied.</Data>
      <Data Name="ExceptionString">Exception detail: System.Security.Cryptography.CryptographicException: Access denied. at System.Security.Cryptography.CryptographicException.ThrowCryptogaphicException(Int32 hr) at System.Security.Cryptography.X509Certificates.X509Utils._LoadCertFromBlob(Byte[] rawData, IntPtr password, UInt32 dwFlags, Boolean persistKeySet, SafeCertContextHandle& pCertCtx) at System.Security.Cryptography.X509Certificates.X509Certificate.LoadCertificateFromBlob(Byte[] rawData, Object password, X509KeyStorageFlags keyStorageFlags) at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password) at Microsoft.Hpc.Scheduler.Store.SchedulerStoreInternal.VerifyCertificate(String unencryptedPassword, Byte[] certificate, String userSid, WindowsIdentity ownerIdentity) Current stack: at Microsoft.Hpc.Scheduler.SchedulerTracing.TraceException(String facility, Exception exception) at Microsoft.Hpc.Scheduler.Store.SchedulerStoreInternal.VerifyCertificate(String unencryptedPassword, Byte[] certificate, String userSid, WindowsIdentity ownerIdentity) at Microsoft.Hpc.Scheduler.Store.SchedulerStoreInternal.SetUserNamePasswordCertificate(ConnectionToken token, String userName, String unencryptedPassword, Nullable`1 reusable, Byte[] certificate) at Microsoft.Hpc.Scheduler.Store.SchedulerStoreInternal.SaveCertificate(ConnectionToken token, String username, String password, Nullable`1 reusable, Byte[] certificate) at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInContext, Object[]& outArgs) at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMessage msg, Int32 methodPtr, Boolean fExecuteInContext) at System.Runtime.Remoting.Messaging.ServerObjectTerminatorSink.SyncProcessMessage(IMessage reqMsg) at System.Runtime.Remoting.Messaging.ServerContextTerminatorSink.SyncProcessMessage(IMessage reqMsg) at System.Runtime.Remoting.Channels.CrossContextChannel.SyncProcessMessageCallback(Object[] args) at System.Runtime.Remoting.Channels.ChannelServices.DispatchMessage(IServerChannelSinkStack sinkStack, IMessage msg, IMessage& replyMsg) at Microsoft.Hpc.Scheduler.Store.ServiceAsClientServerSink.ProcessMessage(IServerChannelSinkStack sinkStack, IMessage requestMsg, ITransportHeaders requestHeaders, Stream requestStream, IMessage& responseMsg, ITransportHeaders& responseHeaders, Stream& responseStream) at System.Runtime.Remoting.Channels.BinaryServerFormatterSink.ProcessMessage(IServerChannelSinkStack sinkStack, IMessage requestMsg, ITransportHeaders requestHeaders, Stream requestStream, IMessage& responseMsg, ITransportHeaders& responseHeaders, Stream& responseStream) at System.Runtime.Remoting.Channels.Tcp.TcpServerTransportSink.ServiceRequest(Object state) at System.Runtime.Remoting.Channels.SocketHandler.ProcessRequestNow() at System.Runtime.Remoting.Channels.SocketHandler.BeginReadMessageCallback(IAsyncResult ar) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken) at System.Net.Security.NegotiateStream.ProcessFrameBody(Int32 readBytes, Byte[] buffer, Int32 offset, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.NegotiateStream.ReadCallback(AsyncProtocolRequest asyncRequest) at System.Net.FixedSizeReader.CheckCompletionBeforeNextRead(Int32 bytes) at System.Net.FixedSizeReader.ReadCallback(IAsyncResult transportResult) at System.Net.LazyAsyncResult.Complete(IntPtr userToken) at System.Threading.ExecutionContext.runTryCode(Object userData) at System.Runtime.CompilerServices.RuntimeHelpers.ExecuteCodeWithGuaranteedCleanup(TryCode code, CleanupCode backoutCode, Object userData) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state) at System.Net.ContextAwareResult.Complete(IntPtr userToken) at System.Net.LazyAsyncResult.ProtectedInvokeCallback(Object result, IntPtr userToken) at System.Net.Sockets.BaseOverlappedAsyncResult.CompletionPortCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped) at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)</Data>
      </EventData>
      </Event>



    Security Log of Audit Failures

    - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    - <System>
      <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
      <EventID>5061</EventID>
      <Version>0</Version>
      <Level>0</Level>
      <Task>12290</Task>
      <Opcode>0</Opcode>
      <Keywords>0x8010000000000000</Keywords>
      <TimeCreated SystemTime="2011-08-05T21:09:02.141372300Z" />
      <EventRecordID>5971641</EventRecordID>
      <Correlation />
      <Execution ProcessID="776" ThreadID="7908" />
      <Channel>Security</Channel>
      <Computer>headnode.cluster.local</Computer>
      <Security />
      </System>
    - <EventData>
      <Data Name="SubjectUserSid">S-1-5-18</Data>
      <Data Name="SubjectUserName">headnode$</Data>
      <Data Name="SubjectDomainName">PORTAL</Data>
      <Data Name="SubjectLogonId">0x3e7</Data>
      <Data Name="ProviderName">Microsoft Software Key Storage Provider</Data>
      <Data Name="AlgorithmName">RSA</Data>
      <Data Name="KeyName">{EC570324-B591-4831-AE71-2C1B54ED9DCF}</Data>
      <Data Name="KeyType">%%2500</Data>
      <Data Name="Operation">%%2481</Data>
      <Data Name="ReturnCode">0x80090010</Data>
      </EventData>
      </Event>


    - <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
    - <System>
      <Provider Name="Microsoft-Windows-Security-Auditing" Guid="{54849625-5478-4994-A5BA-3E3B0328C30D}" />
      <EventID>5061</EventID>
      <Version>0</Version>
      <Level>0</Level>
      <Task>12290</Task>
      <Opcode>0</Opcode>
      <Keywords>0x8010000000000000</Keywords>
      <TimeCreated SystemTime="2011-08-05T21:09:02.141372300Z" />
      <EventRecordID>5971640</EventRecordID>
      <Correlation />
      <Execution ProcessID="776" ThreadID="7908" />
      <Channel>Security</Channel>
      <Computer>headnode.cluster.local</Computer>
      <Security />
      </System>
    - <EventData>
      <Data Name="SubjectUserSid">S-1-5-18</Data>
      <Data Name="SubjectUserName">headnode$</Data>
      <Data Name="SubjectDomainName">PORTAL</Data>
      <Data Name="SubjectLogonId">0x3e7</Data>
      <Data Name="ProviderName">Microsoft Software Key Storage Provider</Data>
      <Data Name="AlgorithmName">%%2432</Data>
      <Data Name="KeyName">le-HPCSoftcardLogon-4c7ed12d-b050-4c56-b485-85a75c08ba3f</Data>
      <Data Name="KeyType">%%2500</Data>
      <Data Name="Operation">%%2480</Data>
      <Data Name="ReturnCode">0x5</Data>
      </EventData>
      </Event>


    Enhanced Key Usage of the Cert is:

    Smart Card Logon (1.3.6.1.4.1.311.20.2.2)
    Client Authentication (1.3.6.1.5.5.7.3.2)

     

    Regards,

    Pat

     

     


    Pat Collins
    Friday, August 5, 2011 9:30 PM
  • Wednesday, August 10, 2011 2:29 PM