locked
Using compression for synchronization over wcf RRS feed

  • Question

  •  

    Hello,

     

    in order to reduce network traffic, i would be very interested if anybody has experience with using compression for sync with wcf? As fas as I can see Sync-Data is transferred with DataSets as transfer objects. So there should be a big potential for increasing performance and reducing traffic.

     

    Regards and thanks for input

     

    Thomas

    • Moved by Max Wang_1983 Thursday, April 21, 2011 11:11 PM forum consolidation (From:SyncFx - Microsoft Sync Framework Database Providers [ReadOnly])
    Thursday, January 24, 2008 2:59 PM

Answers

  • Hi Andrew,

     

    thanks for your post, it helped me to understand a little more about setting up the configuration. We use in fact only desktops to synchronize with a server and I want the gzip encoding adapted for that situation. I think the problem in my thinking was that I thought I had to use wsHttpBinding to set up the appropriate attributes and that binding does not combine with the encoding mechanism, since it is configured with customBinding. So this morning I got it working (at least I dont get any errors ;-) I have not measured yet the performance advantage, but I am sure it is considerable.

     

    So here are some steps to help others:

    1. Get the Demo running from the WCF Samples showing Gzip Encoding path: WcfSamples\TechnologySamples\Extensibility\MessageEncoder\Compression\..

    (you can download them from: http://msdn2.microsoft.com/en-us/library/ms751458.aspx)

     

    In your app.config and your web.config (at the server) do steps 2-5 (4 only for web.config)

     

    2. Make a reference to the GZipEncoder assembly from your wcf service and your app.

    3. Add an extensions section below system.serviceModel

    <system.serviceModel>

    <extensions>

    <bindingElementExtensions>

    <add name="gzipMessageEncoding" type="Microsoft.ServiceModel.Samples.GZipMessageEncodingElement, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />

    </bindingElementExtensions>

    </extensions>

    ....

    4. Configure a endpoint using the gzip custom binding:

    <!-- Define another endpoint using Compression -->

    <endpoint address="gzip" binding="customBinding"

    bindingConfiguration="gzipCustomBinding"

    bindingName="gzipCustomBinding"

    contract="WcfSyncService.ISyncService">

    </endpoint>

     

    5. Define the custom binding for that endpoint (adapt the attributes you require):

    <customBinding>

    <binding name="gzipCustomBinding">

    <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />

    <httpTransport hostNameComparisonMode="StrongWildcard"

    manualAddressing="False"

    maxReceivedMessageSize="6553600"

    authenticationScheme="Anonymous"

    bypassProxyOnLocal="False"

    realm=""

    useDefaultWebProxy="True" />

    </binding>

    </customBinding>

     

    Try to call the .svc in your browser e.g.http://localhost/SyncService/SyncService.svc. This will show up errors in your configuration and missing references.

     

    6. In your app.config provide the endpoint to use gzip compression

    <client>

    <endpoint address=http://servername/SyncService/SyncService.svc/gzip

    binding="customBinding" bindingConfiguration="gzipCustomBinding_ISyncService"

    contract="SyncService.ISyncService" name="gzipCustomBinding_ISyncService" />

    </client>

     

    That should do it. Hope this will help others. It would be interesting to hear in this forum of results regarding performance improvement.

     

    Happy coding!

    Thomas

     

    Tuesday, February 5, 2008 6:31 AM

All replies

  • Hello,

     

    well, I took a look at the wcf sample from the TechnologySamples\Extensibility\MessageEncoder\Compression\CS Folder provided with the VS 2008, which uses a gzip compression. My problem is now that I have a wsHttpBinding Section in my App.Config and the Web.Config of the service such as (for the client side):

     

    <wsHttpBinding>

    <binding name="WSHttpBinding_ISyncService" closeTimeout="00:10:00"

    openTimeout="00:10:00" receiveTimeout="00:10:00" sendTimeout="00:10:00"

    bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"

    maxBufferPoolSize="524288" maxReceivedMessageSize="65536000" messageEncoding="Text"

    textEncoding="utf-8" useDefaultWebProxy="true" allowCookies="false">

    <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="16384"

    maxBytesPerRead="4096" maxNameTableCharCount="16384" />

    <reliableSession ordered="true" inactivityTimeout="00:10:00"

    enabled="false" />

    <security mode="Message">

    <transport clientCredentialType="Windows" proxyCredentialType="None"

    realm="" />

    <message clientCredentialType="Windows" negotiateServiceCredential="true"

    algorithmSuite="Default" establishSecurityContext="true" />

    </security>

    </binding>

    </wsHttpBinding>

     

    the compression sample works with an extension section:

     

    <extensions>

    <bindingElementExtensions>

    <add name="gzipMessageEncoding" type="Microsoft.ServiceModel.Samples.GZipMessageEncodingElement, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />

    </bindingElementExtensions>

    </extensions>

     

    and uses a custom binding to use the gzipMessageEncoding:

    <binding name="ISampleServer">

    <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />

    <httpTransport manualAddressing="false"

    authenticationScheme="Anonymous"

    bypassProxyOnLocal="false"

    hostNameComparisonMode="StrongWildcard"

    proxyAuthenticationScheme="Anonymous"

    realm=""

    useDefaultWebProxy="true" />

    </binding>

    </customBinding>

    </bindings>

     

    now I think I cannot use the wsHttpBinding side by side with my custom binding since the endpoint adresses only one specific binding:

     

    <client>

    <endpoint address="http://localhost:8000/samples/GZipEncoder/HttpSampleServer"

    bindingConfiguration="ISampleServer"

    binding="customBinding"

    contract="ISampleServer">

    </endpoint>

    <metadata>

    <policyImporters>

    <extension type="Microsoft.ServiceModel.Samples.GZipMessageEncodingBindingElementImporter, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />

    </policyImporters>

    </metadata>

    </client>

     

    which probably means I have to get the wsHttpBinding properties somehow into my custombinding. Is that correct?

     

    I am a little lost here. Can you please give me a hint how to implement the compression feature?

     

    Regards and thanks

     

    Thomas

     

    Friday, January 25, 2008 4:54 PM
  • Hello,

    I suppose this is a wcf question. As soon as I have a running example and comparison of performance of synchronisation with and without compression I will post it here, if it is of any interest.

     

    Regards Thomas

     

    Monday, January 28, 2008 8:06 AM
  • Thomas,

     

    I would like to know, is there any performance improvement of synchronization with compression?

     

    Thanks,

    Sreevi

    Thursday, January 31, 2008 7:39 PM
  • Hello Sreevi,

     

    the data gets transferred as plain xml when syncing via wcf. So there should be a large improvement when compressing the xml data, especially for large data transfer. I have not yet completed the sample (in general there is some info on compression in the wcf forum). I will let you know in this forum of my results ( if i hopefully will get it working)

     

    Regards

     

    Thomas

     

    Thursday, January 31, 2008 9:02 PM
  • Hi Thomas,

     

    Adding the Gzip compression binding element should shrink your WCF message size considerably.  It sounds like you're struggling with getting wsHttpBinding and Gzip compression working together.  Let me see if I can help...

     

    First, as it sounds like you may already be aware, wsHttpBinding is not supported on .NET Compact Framework, so if you want to use wsHttpBinding in one of your endpoints, NetCF will not connect to that endpoint. 

     

    Second, since it sounds like you're trying to get two side-by-side endpoints, one for wsHttpBinding and one that uses Gzip, here's how you can do it: rather than try to describe two bindings on one endpoint, create a second endpoint tag in the Web.config file of your service.  The first endpoint tag should point to your wsHttpBinding, and the second endpoint should point to your custom binding that uses the Gzip encoder.  The two endpoints will share the same contract, but will have different endpoint URLs (yes, that's the way it has to be).  They can be very similar, and the same .svc file can serve both endpoints.  Your endpoint URLs could be something like:

    .../myservice.svc/ws

    .../myservice.svc/gzip

    And you can specify these URLs in your Web.config file, and just point the device at the /gzip one and your desktop clients at the /ws one. (you could even strip off the /ws suffix in your Web.config file so that desktop doesn't need anything special and devices need to add /gzip).

     

    Hope this helps.

    Monday, February 4, 2008 3:53 PM
  • Hi Andrew,

     

    thanks for your post, it helped me to understand a little more about setting up the configuration. We use in fact only desktops to synchronize with a server and I want the gzip encoding adapted for that situation. I think the problem in my thinking was that I thought I had to use wsHttpBinding to set up the appropriate attributes and that binding does not combine with the encoding mechanism, since it is configured with customBinding. So this morning I got it working (at least I dont get any errors ;-) I have not measured yet the performance advantage, but I am sure it is considerable.

     

    So here are some steps to help others:

    1. Get the Demo running from the WCF Samples showing Gzip Encoding path: WcfSamples\TechnologySamples\Extensibility\MessageEncoder\Compression\..

    (you can download them from: http://msdn2.microsoft.com/en-us/library/ms751458.aspx)

     

    In your app.config and your web.config (at the server) do steps 2-5 (4 only for web.config)

     

    2. Make a reference to the GZipEncoder assembly from your wcf service and your app.

    3. Add an extensions section below system.serviceModel

    <system.serviceModel>

    <extensions>

    <bindingElementExtensions>

    <add name="gzipMessageEncoding" type="Microsoft.ServiceModel.Samples.GZipMessageEncodingElement, GZipEncoder, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" />

    </bindingElementExtensions>

    </extensions>

    ....

    4. Configure a endpoint using the gzip custom binding:

    <!-- Define another endpoint using Compression -->

    <endpoint address="gzip" binding="customBinding"

    bindingConfiguration="gzipCustomBinding"

    bindingName="gzipCustomBinding"

    contract="WcfSyncService.ISyncService">

    </endpoint>

     

    5. Define the custom binding for that endpoint (adapt the attributes you require):

    <customBinding>

    <binding name="gzipCustomBinding">

    <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />

    <httpTransport hostNameComparisonMode="StrongWildcard"

    manualAddressing="False"

    maxReceivedMessageSize="6553600"

    authenticationScheme="Anonymous"

    bypassProxyOnLocal="False"

    realm=""

    useDefaultWebProxy="True" />

    </binding>

    </customBinding>

     

    Try to call the .svc in your browser e.g.http://localhost/SyncService/SyncService.svc. This will show up errors in your configuration and missing references.

     

    6. In your app.config provide the endpoint to use gzip compression

    <client>

    <endpoint address=http://servername/SyncService/SyncService.svc/gzip

    binding="customBinding" bindingConfiguration="gzipCustomBinding_ISyncService"

    contract="SyncService.ISyncService" name="gzipCustomBinding_ISyncService" />

    </client>

     

    That should do it. Hope this will help others. It would be interesting to hear in this forum of results regarding performance improvement.

     

    Happy coding!

    Thomas

     

    Tuesday, February 5, 2008 6:31 AM
  • Yes, I have used this the MessageEncoder, but i m afraid that the Performance Enhancement are not that great,

    I am transferring just 1.5 MB data and it is taking around 53 secs, when i dig more i found that actually data transfer on wire is not taking time  

    the ReadMessage (method of GZipMessageEncoder  class ) is called with in next 2 seconds, but after that the internal WCF message deserializing (I SUPPOSE) is taking time that is ( 53 - 2 = 51 secs approx. ), is there anything i can do. Any help will be greatly appreciated.

     

    One thing i wanna say here, this thing (WCF) looks big.

    Wednesday, August 6, 2008 4:33 PM
  • Hi,

     

    I am not able to bind the customBinding in the web.config file. In my web.config file of my wcf service

     

    I have the following configurations. but its says "The element binding has invalid child elements  'gZipCustomBinding' ".List of possible elements expected are  ' binaryMessageEncoding etc etc etc.

     

    The thing highlighted in yellow is giving me error.

     

    <bindings>

    <customBinding>

    <binding name="gzipCustomBinding">

    <gzipMessageEncoding innerMessageEncoding="textMessageEncoding" />

    <httpTransport hostNameComparisonMode="StrongWildcard"

    manualAddressing="False"

    maxReceivedMessageSize="6553600"

    authenticationScheme="Anonymous"

    bypassProxyOnLocal="False"

    realm=""

    useDefaultWebProxy="True" />

    </binding>

    </customBinding>

    </bindings>

     

    Wednesday, October 1, 2008 11:24 AM
  • I have the same problem
    Monday, October 20, 2008 4:50 PM
  • Hi,

     

    How can i increase the maxArrayLength value? When i try to send data to service for saving i get billow error.

     

    The formatter threw an exception while trying to deserialize the message: There was an error while trying to deserialize parameter http://tempuri.org/:bios. The InnerException message was 'There was an error deserializing the object of type System.Collections.Generic.IList`1[[ImageService.Images, ImageService, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null]]. The maximum array length quota (16384) has been exceeded while reading XML data. This quota may be increased by changing the MaxArrayLength property on the XmlDictionaryReaderQuotas object used when creating the XML reader. Line 1, position 118981.'.  Please see InnerException for more details.

     

    Thanks

     

    Arvind

     

    Thursday, November 6, 2008 11:11 AM
  • Hi Arvind,

     

    Not sure if you're still looking for an answer, but I had the same problem today and hit your post while looking for a solution. In the end, I went back and read the documentation on the samples and the "stack" bit right at the start gave away the solution. You can exposing the properties of the underlying TextMessageEncodingBindingElement as a ConfigurationProperty of your GZip class by adding this:

     

    Code Snippet

    public class GZipMessageEncodingElement : BindingElementExtensionElement
    {

    // existing code goes here

    // your new bits start here:

     

            [ConfigurationProperty("maxArrayLength", DefaultValue = "2147483647")]
            public string MaxArrayLength
            {
                get { return (string)base["maxArrayLength"]; }
                set { base["maxArrayLength"] = value; }
            }

     

    // existing code

    }

     

     

     

     

    for each element you wish to expose. Obviously choose a default which makes sense for your application, and the place to stick it is

     

    You then need to actually set those values on the underlying object, which I've done in the ApplyConfiguration method:

     

    Code Snippet

    //Called by the WCF to apply the configuration settings (the property above) to the binding element
    public override void ApplyConfiguration(BindingElement bindingElement)
    {
        GZipMessageEncodingBindingElement binding = (GZipMessageEncodingBindingElement)bindingElement;
        PropertyInformationCollection propertyInfo = this.ElementInformation.Properties;
        if (propertyInfo["innerMessageEncoding"].ValueOrigin != PropertyValueOrigin.Default)
        {
            switch (this.InnerMessageEncoding)
            {
                case "textMessageEncoding":
                    binding.InnerMessageEncodingBindingElement = new TextMessageEncodingBindingElement();
                    ((TextMessageEncodingBindingElement)binding.InnerMessageEncodingBindingElement).ReaderQuotas.MaxDepth = Int32.Parse(MaxDepth);
                    ((TextMessageEncodingBindingElement)binding.InnerMessageEncodingBindingElement).ReaderQuotas.MaxStringContentLength = Int32.Parse(MaxStringContentLength);
                    ((TextMessageEncodingBindingElement)binding.InnerMessageEncodingBindingElement).ReaderQuotas.MaxArrayLength = Int32.Parse(MaxArrayLength);
                    ((TextMessageEncodingBindingElement)binding.InnerMessageEncodingBindingElement).ReaderQuotas.MaxNameTableCharCount = Int32.Parse(MaxNameTableCharCount);
                    ((TextMessageEncodingBindingElement)binding.InnerMessageEncodingBindingElement).ReaderQuotas.MaxBytesPerRead = Int32.Parse(MaxBytesPerRead);
                    break;
                case "binaryMessageEncoding":
                    binding.InnerMessageEncodingBindingElement = new BinaryMessageEncodingBindingElement();
                    break;
            }
        }
    }

     

     

    Note that this will only set those properties if you explictly set innerMessageEncoding="textMessageEncoding" in your config - if you leave that attribute out, the others will be ignored also.

     

    You can then set the various attributes in your config file like this:

     

    Code Snippet

    <gzipMessageEncoding innerMessageEncoding="textMessageEncoding"
                maxDepth="2147483647" maxStringContentLength="2147483647" maxArrayLength="2147483647"
                maxNameTableCharCount="2147483647" maxBytesPerRead="2147483647" />

     

     

    Hope that helps,

     

    Kev

    Tuesday, December 30, 2008 1:25 PM
  • Uh, I have the same problem as SM_Boy and Luigi Conti. Have either of you figured that out yet?
    • Proposed as answer by Shabu283 Monday, March 23, 2009 8:20 AM
    Thursday, March 5, 2009 3:16 PM
  •  SM_Boy , Luigi Conti and Kaspernik I had the same problem. If you are Using Visual Studio 2008 , your App.config file will be validated against "DotNetConfig.xsd" Schema . But for the above mentioned problem you should also use "DotNetConfig02.xsd" Schema. Which will be found in App.config Properties. Provided you should be opening the app.config file, then u should look in to properties.
    This will resolve the problem
    Thankyou

    Sabarish
    Chennai,India
    • Proposed as answer by Shabu283 Monday, March 23, 2009 8:30 AM
    Monday, March 23, 2009 8:30 AM
  • Thanks Shabu283 for the answer. I actually have another thread dealing with this subject in particular. Which states:

    "This is just a designer error... you can ignore it. If you REALLY are bothered by it you can register the binding using the svcconfigeditor tool.  When you load the config file it will tell you the the binding could not be found.  Click Yes and locate the file to resolve the issue."

    That thread is located here:
    http://social.msdn.microsoft.com/Forums/en-US/uklaunch2007ado.net/thread/c9dd4b76-5afa-42aa-a81d-35abd512e645

    Monday, March 23, 2009 6:03 PM
  • Hi Thomas Moechel, thanks for the step by steps, it works so well in my application.
    I got a question, previously i am using the code below as my config:
    <bindings>
          <wsHttpBinding>
            <binding name="something" closeTimeout="00:10:00" openTimeout="00:10:00" receiveTimeout="00:10:00" sendTimeout="00:10:00" bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard" maxBufferPoolSize="2147483646" maxReceivedMessageSize="2147483646" messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true" allowCookies="false">
              <readerQuotas maxDepth="32" maxStringContentLength="2147483646" maxArrayLength="2147483646" maxBytesPerRead="4096000" maxNameTableCharCount="2147483646" />
              <security mode="Message">
                <message clientCredentialType="Windows" negotiateServiceCredential="true" establishSecurityContext="false" />
              </security>
            </binding>
          </wsHttpBinding>
        </bindings>
    After applied the solution, the message is compressed, but, may i know how to use the wsHttpBinding as well?
    like:
     <security mode="Message">
                <message clientCredentialType="Windows" negotiateServiceCredential="true" establishSecurityContext="false" />
     </security>
    Why i am doing this is because i would like to apply the same wsHttpBinding's security setting to my new custom binding (your step by step solution).

    thanks.
    Tuesday, October 6, 2009 3:15 AM