System.Net.HttpWebRequest fails to logon to a website via com-interop in Windows 10 1809

All replies

• 

  

  0

  Sign in to vote

  Hi Stochastically,

  Since your question is more related to Excel and VBA, you could post a new thread in Microsoft Office for development forum for suitable support.

  The CLR Forum discuss and ask questions about .NET Framework Base Classes (BCL) such as Collections, I/O, Regigistry, Globalization, Reflection. Also discuss all the other Microsoft libraries that are built on or extend the .NET Framework, including Managed Extensibility Framework (MEF), Charting Controls, CardSpace, Windows Identity Foundation (WIF), Point of Sale (POS), Transactions.

  Best Regards,

  Wendy

  ---

  MSDN Community Support
  Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

  Monday, February 11, 2019 7:21 AM
• 

  

  0

  Sign in to vote

  Hi Wendy,

  Thank you for the response.  I am not sure that the forum you suggest is the right one, because it seems to me that this relates to the behaviour of low level Windows 10 and .Net DLLs.  Nonetheless, I have posted the question there.

  Is there a forum that relates to the Windows 10 and .Net DLLs that implement web security, namely protocols such as TLS?  The heart of my problem is that those low level DLLs choose a different implementation in the two cases that work versus the case that doesn't work.  My best guess is that I need a registry setting (or to delete a registry setting) to fix this.

  Thanks.

  Monday, February 11, 2019 7:48 AM
• 

  

  0

  Sign in to vote

  Seems you've hit by this problem.

  Try explicitly set the ServicePointManager.SecurityProtocol  and see if it does any help.
  
  

  • Edited by cheong00 Tuesday, February 12, 2019 3:09 AM

  Tuesday, February 12, 2019 2:46 AM
• 

  

  0

  Sign in to vote

  Thanks for the suggestion, cheong00, but in fact I've already tried setting ServicePointManager.SecurityProtocol  explicitly, and also checked that everything is x64.  Whether I set ServicePointManager.SecurityProtocol explicitly or not, when I use Fiddler to look at the internet traffic being generated, all cases were use security protocol "Version: 3.3 (TLS/1.2)".

  Thanks too for the link to Avoid sending TLS_EMPTY_RENEGOTIATION_INFO_SCSV cipher ... .  But I'm not sure it helps.  On the same computer, the correct internet traffic is generated when the code is called directly, versus the incorrect internet traffic (that uses TLS_EMPTY_RENEGOTIATION_INFO_SCSV) when called from Excel via com-interop.  As I said to Wendy yesterday,  the heart of my problem is that low level windows/.Net DLLs choose to generate different internet traffic in the different cases.  My best guess is that I need a registry setting (or to delete a registry setting) to fix this.

  Please let me know if you have any other ideas, or whether you can recommend a better forum.

  Thanks, Paul

  Tuesday, February 12, 2019 7:46 AM
• 

  

  0

  Sign in to vote

  Since I'm unable to setup test for your issue, can you try to add try..catch block to tell us the exact line exception is thrown?

  Also, check ComputerB and see if there exists another code running in Excel with the same certificate. .NET will reuse connection with client certificate specified (something like using certificate.GetHashCode().ToString() as key to try get existing connection from the pool)

  Tuesday, February 12, 2019 8:46 AM
• 

  

  0

  Sign in to vote

   The error doesn't manifest itself by an exception being thrown, cheong00.  In all cases the login request returns a little JSON object, where the JSON object has a field called "loginStatus". When it works, "loginStatus"="SUCCESS", but when it fails "loginStatus"="CERT_AUTH_REQUIRED".  Also, the version of Excel that I'm using for this has all add-ins disabled. so there can't anything else using  the same certificate.  I've deliberately cut everything down to the bare bones here, to keep everything as simple as possible and hence just focus on the core issue.

  Today I did make some progress, because someone on stackoverflow.com told me how to enable debugging for http calls in .Net (see How do I see the raw HTTP request that the HttpWebRequest class sends for what I did).  So I ran the diagnostics on computer B for the two cases (calling the C# code directly from a console EXE, and then calling from Excel 2010 via com-interop).  The "System.Net Information" output for the code which calls directly from a console EXE was as follows:

      System.Net Information: 0 : [35268] Current OS installation type is 'Client'.
      System.Net Information: 0 : [35268] RAS supported: True
      System.Net Information: 0 : [35268] Associating HttpWebRequest#21454193 with ServicePoint#34640832
      System.Net Information: 0 : [35268] Associating Connection#43332040 with HttpWebRequest#21454193
      System.Net Information: 0 : [35268] Connection#43332040 - Created connection from XXX.XXX.XXX.XXX:53002 to YYY.YYY.YYY.YYY:443.
      System.Net Information: 0 : [35268] TlsStream#54444047::.ctor(host=<TargetWebSite>, #certs=1, checkCertificateRevocationList=False, sslProtocols=Tls12)
      System.Net Information: 0 : [35268] Associating HttpWebRequest#21454193 with ConnectStream#20234383
      System.Net Information: 0 : [35268] HttpWebRequest#21454193 - Request: POST /api/certlogin HTTP/1.1
      System.Net Information: 0 : [35268] ConnectStream#20234383 - Sending headers
      System.Net Information: 0 : [35268] SecureChannel#47891719::.ctor(hostname=<TargetWebSite>, #clientCertificates=1, encryptionPolicy=RequireEncryption)
      System.Net Information: 0 : [35268] Enumerating security packages:
      System.Net Information: 0 : [35268]     Negotiate
      System.Net Information: 0 : [35268]     NegoExtender
      System.Net Information: 0 : [35268]     Kerberos
      System.Net Information: 0 : [35268]     NTLM
      System.Net Information: 0 : [35268]     TSSSP
      System.Net Information: 0 : [35268]     pku2u
      System.Net Information: 0 : [35268]     CloudAP
      System.Net Information: 0 : [35268]     WDigest
      System.Net Information: 0 : [35268]     Schannel
      System.Net Information: 0 : [35268]     Microsoft Unified Security Protocol Provider
      System.Net Information: 0 : [35268]     Default TLS SSP
      System.Net Information: 0 : [35268]     CREDSSP
      System.Net Information: 0 : [35268] SecureChannel#47891719 - Attempting to restart the session using the user-provided certificate: [Version]
      System.Net Information: 0 : [35268] SecureChannel#47891719 - Left with 1 client certificates to choose from.
      System.Net Information: 0 : [35268] SecureChannel#47891719 - Trying to find a matching certificate in the certificate store.
      System.Net Information: 0 : [35268] SecureChannel#47891719 - Locating the private key for the certificate: [Version]
      System.Net Information: 0 : [35268] SecureChannel#47891719 - Certificate is of type X509Certificate2 and contains the private key.
      System.Net Information: 0 : [35268] SecureChannel#47891719::.AcquireClientCredentials, new SecureCredential() (flags=(ValidateManual, NoDefaultCred, SendAuxRecord, UseStrongCrypto), m_ProtocolFlags=(Tls12Client), m_EncryptionPolicy=RequireEncryption)
      System.Net Information: 0 : [35268] AcquireCredentialsHandle(package = Microsoft Unified Security Protocol Provider, intent  = Outbound, scc     = System.Net.SecureCredential)
      System.Net Information: 0 : [35268] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = (null), targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
      System.Net Information: 0 : [35268] InitializeSecurityContext(In-Buffer length=0, Out-Buffer length=184, returned code=ContinueNeeded).
      System.Net Information: 0 : [35268] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 227c85a89b0:2449d0deff0, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
      System.Net Information: 0 : [35268] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=CredentialsNeeded).
  
  However, when running from Excel 2010 via com-interop, instead of those last 4 InitializeSecurityContext lines, there are 6 InitializeSecurityContext lines as follows:
  
      System.Net Information: 0 : [39988] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = (null), targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
      System.Net Information: 0 : [39988] InitializeSecurityContext(In-Buffer length=0, Out-Buffer length=184, returned code=ContinueNeeded).
      System.Net Information: 0 : [39988] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 8a8e2f0:2449d0def90, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
      System.Net Information: 0 : [39988] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
      System.Net Information: 0 : [39988] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 8a8e2f0:2449d0def90, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
      System.Net Information: 0 : [39988] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
  
  The first two InitializeSecurityContext lines are the identical, so presumably crucial difference is on the third InitializeSecurityContext line, where the console EXE has
  
  context = 227c85a89b0:2449d0deff0
  
  but the failed run via com-interop executino has
  
  context = 8a8e2f0:2449d0def90
  
  After that the diagnostic output looks different, as one would expect.  Does anyone know what that difference means, and how to make the difference go away so that the com-interop execution behaves in the same way as the com-interop execution?

  Tuesday, February 12, 2019 10:09 PM
• 

  

  0

  Sign in to vote

  The context is just a token identifying partial formed security content returned by phNewContext so by itself has not much meaning.

  The relevent part is returned code = ContinueNeeded, where your side would need to make another call to InitializeSecurityContext with the token returned (as in the two-leg security context description on the Remark of linked page).

  Is there AcceptSecurityContext or AcquireCredentialsHandle call appear later in the log?

  
  

  • Edited by cheong00 Wednesday, February 13, 2019 1:58 AM

  Wednesday, February 13, 2019 1:56 AM
• 

  

  0

  Sign in to vote

  Thanks for helping me on this cheong00.  Here's more of the diagnostics.

  For the case that works (running from console EXE), the trace output (starting with the 4 InitializeSecurityContext lines shown previously) continues as follows:

  System.Net Information: 0 : [35268] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = (null), targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [35268] InitializeSecurityContext(In-Buffer length=0, Out-Buffer length=184, returned code=ContinueNeeded).
  System.Net Information: 0 : [35268] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 227c85a89b0:2449d0deff0, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [35268] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=CredentialsNeeded).
  System.Net Information: 0 : [35268] SecureChannel#47891719 - We have user-provided certificates. The server has not specified any issuers, so try all the certificates.
  System.Net Information: 0 : [35268] SecureChannel#47891719 - Selected certificate: [Version]
    V3
  [ Lots of lines describing the certificate that was loaded manually from file by the C# code ]
  System.Net Information: 0 : [35268] SecureChannel#47891719 - Left with 1 client certificates to choose from.
  System.Net Information: 0 : [35268] SecureChannel#47891719 - Trying to find a matching certificate in the certificate store.
  System.Net Information: 0 : [35268] SecureChannel#47891719 - Locating the private key for the certificate: [Version]
    V3
  [ Lots of lines, AGAIN describing the certificate that was loaded manually from file by the C# code ]
  System.Net Information: 0 : [35268] SecureChannel#47891719 - Certificate is of type X509Certificate2 and contains the private key.
  System.Net Information: 0 : [35268] SecureChannel#47891719::.AcquireClientCredentials, new SecureCredential() (flags=(ValidateManual, NoDefaultCred, SendAuxRecord, UseStrongCrypto), m_ProtocolFlags=(Tls12Client), m_EncryptionPolicy=RequireEncryption)
  System.Net Information: 0 : [35268] AcquireCredentialsHandle(package = Microsoft Unified Security Protocol Provider, intent  = Outbound, scc     = System.Net.SecureCredential)
  System.Net Information: 0 : [35268] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 227c85a89b0:2449d0deff0, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [35268] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=1551, returned code=ContinueNeeded).
  System.Net Information: 0 : [35268] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 227c85a89b0:2449d0deff0, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [35268] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
  System.Net Information: 0 : [35268] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 227c85a89b0:2449d0deff0, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [35268] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=OK).
  System.Net Information: 0 : [35268] Remote certificate: [Version]
    V3
  [ Lots of lines describing a certificate with [Subject]=<TargetWebSite>, [Issuer]=HydrantID SSL ICA G2, etc]
  System.Net Information: 0 : [35268] SecureChannel#47891719 - Remote certificate was verified as valid by the user.
  

  etc

  However, for the case that fails (running from Excel 2010 via com-interop), the trace output (starting with the 6 InitializeSecurityContext lines shown previously) continues as follows:

  System.Net Information: 0 : [39988] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = (null), targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [39988] InitializeSecurityContext(In-Buffer length=0, Out-Buffer length=184, returned code=ContinueNeeded).
  System.Net Information: 0 : [39988] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 8a8e2f0:2449d0def90, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [39988] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
  System.Net Information: 0 : [39988] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 8a8e2f0:2449d0def90, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [39988] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
  System.Net Information: 0 : [39988] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 8a8e2f0:2449d0def90, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [39988] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=326, returned code=ContinueNeeded).
  System.Net Information: 0 : [39988] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 8a8e2f0:2449d0def90, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [39988] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=ContinueNeeded).
  System.Net Information: 0 : [39988] InitializeSecurityContext(credential = System.Net.SafeFreeCredential_SECURITY, context = 8a8e2f0:2449d0def90, targetName = <TargetWebSite>, inFlags = ReplayDetect, SequenceDetect, Confidentiality, AllocateMemory, InitManualCredValidation)
  System.Net Information: 0 : [39988] InitializeSecurityContext(In-Buffers count=2, Out-Buffer length=0, returned code=OK).
  System.Net Information: 0 : [39988] Remote certificate: [Version]
    V3
  [ Lots of lines describing a certificate with [Subject]=<TargetWebSite>, [Issuer]=Kaspersky which is my antivirus program, etc]
  System.Net Information: 0 : [39988] SecureChannel#2383799 - Remote certificate was verified as valid by the user.
  etc
  

  
  

  FYI, this is the output from the code that I posted on stackoverflow.com, running on the .Net Framework 4.7.2, and with the line

          System.Net.ServicePointManager.SecurityProtocol = System.Net.SecurityProtocolType.Tls12;
  

  Added right at the start before the System.Net.WebRequest.Create(new Uri(WEBSITE)) line.

  Regarding AcceptSecurityContext and AcquireCredentialsHandle, AcceptSecurityContext occurs nowhere in either case.  But AcquireCredentialsHandle occurs twice for the case that succeeds (once shown in the previous reply, once shown in this post), and for the case that fails AcquireCredentialsHandle only occurs once (shown in the previous reply).

  Is that enough information?

  Thanks, Paul

  Wednesday, February 13, 2019 8:20 AM
• 

  

  0

  Sign in to vote

  AcquireCredentialsHandle exists in "success" case at line [35268] after the security context is initialized, and SSL channel has been established, therefore can proceed to authentication.

  Assuming you've not left out important detail in the logs, I think in the second case it's trying to do without client certs. (Look again for "We have user-provided certificates" in the failed logs)

  If that line does not exists, I think something makes the HttpWebRequest to create connection without attaching the certs in VBA case.

  EDIT: Simon @ StackOverflow might have nailed your case. The web scanner component acts as HTTP proxy on your outgoing traffic and may have influrence on the communication. Try disable it while you test again and see if this will fix the problem.

  • Edited by cheong00 Wednesday, February 13, 2019 10:17 AM
  • Marked as answer by Stochastically Wednesday, February 13, 2019 3:51 PM

  Wednesday, February 13, 2019 10:12 AM
• 

  

  0

  Sign in to vote

  Thanks cheong00.  I disabled Kaspersky anti-virus and the problem went away. Completely bizarre, because the identical version of Kaspersky and identical version of Excel with my identical C# program didn't cause the problem on computer A (with Windows 10 1803).  Anyway, thanks again for your help :-).

  Wednesday, February 13, 2019 3:51 PM