WCF Streamed Transfers, IIS6 and IIS7 HTTP KeepAlive

By Jay at July 10, 2009 21:18 Tags: ,

Ce billet est disponible en francais.

A while back, I was working on a client issue where I was having some kind of unusual socket exception from a WCF client connecting to an IIS6 hosted WCF service.

To get a long story short, if you're using the .NET 3.5 WCF streamed transfer on IIS6 and making a lot of transfers in a small time, disable the KeepAlive feature on your web site. The performance will be lower, but it will last longer (without a client support call).

Still here with me ? :) If you have a bit more time to read, here some detail about what I found on this issue...

The setup is pretty simple : A WCF client that is sending a stream over a WCF service that has the transferMode set to streamed. This allows the transfer of a lot of information using genuine streaming, which means that the client writes to a System.IO.Stream instance, and the server reads from an other System.IO.Stream, and the data does not need to be transferred all at once, like in a "normal" SOAP communication. I'm using the required basicHttpBinding for both ends.

The strange thing is that after having made more than 15000 requests to transfer streams,  I was receivig this exception :

System.ServiceModel.CommunicationException: Could not connect to http://server/streamtest/StreamServiceTest.Service1.svc.
TCP error code 10048: Only one usage of each socket address (protocol/network address/port) is normally permitted 10.0.0.1:80. 
---> System.Net.WebException: Unable to connect to the remote server
---> System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted 10.0.0.1:80

This is a rather common issue, which is mostly found where an application tries to bind to a TCP port but cannot do so, either because it is already being bound to an other application or because it does not use the SO_REUSEADDR socket option and the port was closed very recently.

What is rather unusual is that this exception is raised on the client side and not on the server side !

After a few netstat -an, I found out that an awful lot of sockets were lingering with the following state :

TCP    the.client:50819     the.server:80          TIME_WAIT

There were something like 15000 lines of this, with incrementing numbers for the local port. This state is normal, it's meant to be that way, but it's generally more found lingering on a server, much less on a client.

That could mean only one thing, considering that IIS6.0 is an HTTP/1.1 compliant web server: WCF is requesting that the connection to be closed at after a streamed transfer.

Wireshark being my friend, I started looking up at the content of the dialog between IIS6 and my client application :

POST /streamtest/StreamServiceTest.Service1.svc HTTP/1.1
MIME-Version: 1.0
Content-Type: multipart/related; type="application/xop+xml";start="<http://tempuri.org/0>";boundary="uuid:41d2cf74-aaa6-4a80-a6c4-0ec37692a437+id=1";start-info="text/xml"
SOAPAction: "http://tempuri.org/IService1/Operation1"
Host: the.server
Transfer-Encoding: chunked
Expect: 100-continue
Connection: Keep-Alive

The server answers this :

HTTP/1.1 100 Continue

 Then the stream transfer takes place, gets the SOAP response, then at the end :

HTTP/1.1 200 OK
Date: Sat, 11 Jul 2009 01:40:16 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Connection: close
MIME-Version: 1.0
Transfer-Encoding: chunked
Cache-Control: private

I quickly found out that IIS6.0, or the WCF handler is forcing the connection to close on this last request. That's not particularly unusual, since a server may explictly deny an HTTP client to keep alive the connection.

What's even more unusual is that out of luck by trying to deactivate the IIS6.0 keep alive setting on my web site, I noticed that all the connections were properly closed on the client...!

I tried analysing a bit deeper the dialog between the client and the server, and I noticed two differences :

  1. The content of the final answer of the IIS contains two "Connection: close" headers, which could mean one by the WCF handler, and one by IIS itself. I'm not sure if repeating headers is forbidden in the RFC, I'd have to read it again to be sure.
  2. It looks like the order of the FIN/ACK, ACK packets is a bit different, but I'm not sure either where that stands. Both the client and the server are sending FIN packets to the other side, probably the result of calling Socket.Close().

But then I found out something even stranger : It all works on IIS7 ! And the best of all, the KeepAlive status is honored by the web server. That obviously means that the global performance of the web service is better on IIS7 than it is on IIS6, since there is only one connection opened for all my 15000 calls, which is rather good. Too bad my client cannot switch to IIS7 for now...

It also seems that the WCF client is not behaving the same way it does with IIS6, because at the TCP level only the client is sending a TCP FIN packet and the server is not when the keep alive is disabled.

I think I'll be posting this on Microsoft Connect soon, but I'm not sure where the problem lies, whether it is in IIS6, the WCF client or the WCF server handler, but there is definitely an issue here.

WCF, NuSOAP and ArrayOfString

By Jerome at April 16, 2007 15:21 Tags: , , , ,

When exposing a WebService via WCF, you might want to expose something like this :


[DataContract]
public class SomeContract
{
  [DataMember] 
  public string[] Values { get; set; }
}

For that particular data contract, WCF will be generating a WSDL with something like this :


<xs:complexType name="SomeContract">

  <xs:sequence>

    <xs:element minOccurs="0" name="Values" nillable="true"

                type="q1:ArrayOfstring"

                xmlns:q1="http://schemas.microsoft.com/2003/10/Serialization/Arrays" />

  </xs:sequence>

</xs:complexType>

With ArrayOfString being defined like this :


  <xs:schema elementFormDefault="qualified"

             targetNamespace="http://schemas.microsoft.com/2003/10/Serialization/Arrays"

             xmlns:xs="http://www.w3.org/2001/XMLSchema"

             xmlns:tns="http://schemas.microsoft.com/2003/10/Serialization/Arrays">

    <xs:complexType name="ArrayOfstring">

      <xs:sequence>

        <xs:element minOccurs="0" maxOccurs="unbounded" name="string" nillable="true" type="xs:string"/>

      </xs:sequence>

    </xs:complexType>

    <xs:element name="ArrayOfstring" nillable="true" type="tns:ArrayOfstring"/>

  </xs:schema>

In general, that would be fine. The type "ArrayOfString" is defined in a different namespace, but this should not be a problem.

So, to use that particular type in a method call, you should have a document like this one :


<SomeContract>

  <Values xmlns:a="http://schemas.microsoft.com/2003/10/Serialization/Arrays">

    <a:string>My Value</a:string>

  </Values>

</SomeContract>

"string" elements are contained in a different namespace from the SomeContract element. However, the NuSOAP stock version 0.7.2 has a problem with that kind of schema, and generates instead something like this :


<SomeContract>

  <Values>

    <string>My Value</string>

  </Values>

</SomeContract>


When the WCF deserializer receives a document like this one, it does not find the "Values" member in the namespace he's looking and ends up creating a SomeContract instance with a null array of strings.

Since there's no way of fixing NuSOAP, you may need to tweak your contract to help NuSOAP serializing your data without a namespace.

The CollectionDataContract attribute seems to be the way to go, since there is a way to specify the namespace to use when generating the metadata. The service contract then looks like this :


  [DataContract(Namespace = "http://my.name.space")] 
  public class SomeContract 
  { 
    [DataMember] 
    public ArrayOfString InvalidIdentifiers { get; set; } 
  } 

  [CollectionDataContract(ItemName="string", Namespace="http://my.name.space")]
  public class ArrayOfString : List<string> { } 

Thereby placing everything in the "http://my.name.space" namespace.

You might need to tweak a bit the ArrayOfString class, especially if you need to assign it from an actual string[] instance, but you get the idea.

WCF WebService behind a NAT Gateway

By Jerome at April 15, 2007 11:06 Tags: , , ,

I'm currently working on a WCF service that's exposed via a basicHttpBinding, with some exposition of the metadata via a WSDL url.

The metadata generator has been improved a bit since it is now split into multiple files, with references from the first wsdl to other URI's. There's not much to do to have that WSDL generated, and the generator take the liberty of using the machine's name to create reference URIs.

Most of the time, this is a good idea, but sometimes when your machine is behind a NAT gateway doing some port forwarding, your machine probably won't have the public FQDN used to access your gateway. You then end up with a root document referencing URI's with a host name that is not valid on the other side of the gateway.

The solution to change that behavior is quite simple : Just change the host name IIS is listening on. Use the MMC snap-in, and specify that your port 80 (or whatever port you're using) is listening on your external FQDN. WCF will then use that host name when generating URI's. Also don't forget that if you specify a host name, you might need to add an other entry to be able to access your website using "localhost".

This operation is also needed for servers farm when doing some load balancing.

Playing with WCF and NuSOAP 0.7.2

By Jerome at January 25, 2007 10:24 Tags: , , , ,

.NET 3.0 has now been released, along with Windows Communication Foundation (WCF). I thought I could give it a shot for one of my projects at work, where I have to create an internal web service. The problem is that the client at the other end will be using NuSOAP 0.7.2 to contact this WebService, so I had to make sure it would work fine.

First observation, compared to ASP.NET generated WebService, the WCF wsdl is much more complex. Actually, it has a little bit more information such as the validation rules for a guid, but it also has its schema split up into multiple XSD files. I was a bit worried that NuSOAP wouldn't handle that well, but it does fine... I also wanted to be able to expose a signature like this :

1:     [DataContract]
2:     public class Parameter
3:     {
4:         private string _key;
5:         private object _value;
6:  
7:         [DataMember]
8:         public string Key
9:         {
10:             get { return _key; }
11:             set { _key = value; }
12:        }
13:  
14:        [DataMember]
15:        public object Value
16:        {
17:             get { return _value; }
18:             set { _value = value; }
19:        }
20:    }

You'll notice that the second property is an object, and that implies that the serializer/deserializer handles properly types defined at runtime, by using xsd:anyType in the schema.

So, after a few attempts to get working linux LiveCD distro, for which none of them had PHP compiled the correct --enable-soap flag, I decided to fall back from the native PHP SOAP extensions to the OpenSource SOAP library NuSOAP and use EasyPHP on Windows.

First, I had to change NuSOAP's response encoding to UTF-8 using soap_defencoding, which seems to be the default for WCF, then I had to figure out how to pass an arry of structures to call my method.

So, for the following signature :

    void MyMethod(string a, string b, Parameter[] parameters)

The caller's php parameter structure should be :

$params = array(
'a' => $a,
'b' => $b,
'parameters' => array(
'Parameter' => array(
array('Key' => 'abc', 'Value' => 10),
array('Key' => 'def', 'Value' => 42)
)
),
); 

Notice that you have to place a "Parameter" element inside the "parameters" parameter.

Then, to call the method, use the following line :

    $client->call("MyMethod", array($params));

by encapsulating the parameter array once again. This makes a lot of levels to call one little function... I prefer the .NET way of doing thing :)

An other problem came up right at this point : The array, although the information being in the soap body, was not filled on the .NET side. After comparing with a .NET to WCF call, I figured that there was a missing namespace. This is what is generated by default with the code I've presented above using NuSOAP :

<parameters>
<Parameter>
<Key>abc</Key>
<Value>10</Value>
</Parameter>
<Parameter>
<Key>def</Key>
<Value>42</Value>
</Parameter>
</parameters>

And this is what .NET is generating :

<parameters>
<Parameter xmlns="http://schemas.datacontract.org/2004/07/MyService">
<Key>abc</Key>
<Value>10</Value>
</Parameter>
<Parameter xmlns="http://schemas.datacontract.org/2004/07/MyService">
<Key>def</Key>
<Value>42</Value>
</Parameter>
</parameters>

For some reason, the WCF basicHttp binding point is generating two different namespaces for the service contract and for the data contract. To fix this, you just have to specify explicitly a namespace for each ServiceContract and DataContract attribute of your service :

    [DataContract("http://mywebservice.example.com")]

The other problem was about using an unspecified data type as a member of a structure, a System.Object in my case. Well, it turns out that NuSOAP does not support it, as it does not include the data type of the serialized element, so the WCF deserializer cannot interpret it. I changed the data type back to string, unfortunately losing the type information. I can get it from somewhere else but still, this can lead to serialization problems related to culture (comma being a dot and the opposite depending on systems, for instance).

Anyway, there are a few things to remember to get things to work fine with NuSOAP :

  • Change the encoding to UTF-8 or whatever encoding you choose to use,
  • Don't forget to specify the name of the type of an element in an array in PHP, 
  • Do not expose unspecified parameters or attributes,
  • Explicitly specify the namespace of each DataContract and ServiceContact attribute of your service.

It's been a while since I've written a line of PHP code, and I didn't miss it at all. I'm going back to WCF now :)

About me

My name is Jerome Laban, I am a Software Architect, C# MVP and .NET enthustiast from Montréal, QC. You will find my blog on this site, where I'm adding my thoughts on current events, or the things I'm working on, such as the Remote Control for Windows Phone.