none
HPC 2008 SOA: How to avoid timeout when running long duration (>10 minutes) requests RRS feed

  • General discussion

  • Problem description

     

    In HPC 2008 (V2) SOA, a service call lasts more than 10 minutes will cause the broker closes the connection to the client. This will cause a timeout.

     

     

    Solution:

     

    Example:

    -      Assume call durations may be longer than 10 minutes but shorter than 30 minutes.

    -      My service dll is SimpleService.dll

    -      My service name is SOATests.SoaSimpleService

    -      My service contract is SOATests.ISoaSimpleService

     

    1.    Client: add the following to the client code.

    client.InnerChannel.OperationTimeout = TimeSpan.FromMinutes(30);

     

    2.    Service side.

     

    Add a <serviceAssembly>.dll.config and deploy it together with the <serviceAssembly>.dll to all nodes.

     

    Template:

     

    <?xml version="1.0" encoding="utf-8" ?>

    <configuration> 

    <system.serviceModel>

        <services>

          <service name="<ServiceImplName>">

            <endpoint binding="netTcpBinding" bindingConfiguration="ServiceBinding"

              name="tcpbinging0" contract="<ServiceContractName>" />

          </service>

        </services>

        <bindings>

          <netTcpBinding>

            <binding name="ServiceBinding" receiveTimeout="00:30:00" portSharingEnabled="true">

              <security mode="Transport" />

            </binding>

          </netTcpBinding>

        </bindings>

      </system.serviceModel>

    </configuration>

     

    Example: (file is called SimpleService.dll.config)

     

    <?xml version="1.0" encoding="utf-8" ?>

    <configuration>

     

      <system.serviceModel>

        <bindings>

          <netTcpBinding>

            <binding name="myBinding"  receiveTimeout="00:30:00">         

              <security mode="Transport" />

            </binding>

          </netTcpBinding>

        </bindings>

       

        <services>

          <service name="SOATests.SoaSimpleService">       

            <endpoint address="" binding="netTcpBinding"

                bindingConfiguration="myBinding"

                contract="SOATests.ISoaSimpleService"/>

           

          </service>

        </services>

      </system.serviceModel>

     

    </configuration>

     

     

    Friday, February 26, 2010 6:13 PM

All replies

  • More details about this scenario:

    1) I tested the above scenario using 30-minute and 80-minute request (by making timeout=90-minute). Both works.

    2) In case you have problems with the above configuration, please check your network settings. for example, firewall settings and IPSec settings.

    3) In case there is still problem after checking networking, please collect tracing data. The following is how to add tracing to HPC 2008 SOA


    1. add tracing to service host on each compute node.

     Edit file %CCP_HOME%\bin\HpcServiceHost.exe.config. section system.diagnostic

     <system.diagnostics>
         <sources>
           <source name="Microsoft.Hpc.HpcServiceHosting" switchValue="All">
             <listeners>
               <add name="Console" />
               <add name="ServiceHostTraceListener" />
             </listeners>
           </source>
         </sources>
         <sharedListeners>
           <add initializeData="\\<HEADNODE>\CcpSpoolDir\host.svclog" type="System.Diagnostics.XmlWriterTraceListener"
             name="ServiceHostTraceListener">
             <filter type="" />
           </add>
           <add type="System.Diagnostics.ConsoleTraceListener, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
             name="Console" traceOutputOptions="DateTime, ThreadId">
             <filter type="" />
           </add>
         </sharedListeners>
         <trace autoflush="true" />
     </system.diagnostics>

    2. add tracing to broker nodes.
     Edit file %CCP_HOME%\bin\HpcWcfBroker.exe.config. section system.diagnostic:

      <system.diagnostics>
        <sources>
          <source name="Microsoft.Hpc.ServiceBroker" switchValue="All">
            <listeners>
              <add name="Console">
                <filter type="" />
              </add>
              <add name="WSLBTraceListener">
                <filter type="" />
              </add>
              <remove name ="Default" />
            </listeners>
          </source>
        </sources>
        <sharedListeners>
          <add type="System.Diagnostics.ConsoleTraceListener, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
            name="Console" traceOutputOptions="DateTime, ThreadId">
            <filter type="" />
          </add>
          <add initializeData="\\<HEADNODE>\CcpSpoolDir\broker.svclog"
            type="System.Diagnostics.XmlWriterTraceListener, System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
            name="WSLBTraceListener" traceOutputOptions="Timestamp">
            <filter type="" />
          </add>
        </sharedListeners>
        <trace autoflush="true" />
      </system.diagnostics>

     

     

    Monday, June 28, 2010 5:25 PM