A look at Linq to objects and the "let" keyword

By Jerome at April 26, 2008 13:08 Tags: , ,

Cet article est aussi disponible en français ici. 

I've had some time lately to use LINQ a bit more intensively and in particular to use the let keyword.

I had to process a lot of XML files placed in multiple folders, and I wanted to filter them using a regex. First, here's how to get all files from a directory tree :

  var files = from dir in Directory.GetDirectories(rootPath, SearchOption.AllDirectories)
              from file in Directory.GetFiles("", "*.*")
              select new { Path = dir, File = Path.GetFileName(file) };

At this point, I could have omitted the Directory.GetDirectories call because GetFiles can also search recursively. But since the GetFiles method only returns a string array and not an enumerator, it means that all my files would have been returned in one array, which is not memory effective. I'd rather have an iterator based implementation of GetDirectories and GetFiles for that matter, but the finest grained enumeration can only be done this way...

Anyway, having all my files, I now wanted to filter the collection with a specific Regex, as my legitimate files need to observe a specific pattern. So, I updated my query to this :

    Regex match = new Regex(@"(?<value>\d{4}).xml");
    var files2 = from dir in Directory.GetDirectories(args[0], "*", SearchOption.AllDirectories)
                 from file in Directory.GetFiles(dir, "*.xml")
                 let r = match.Match(Path.GetFileName(file))
                 where r.Success
                 select new {
                    Path = dir,
                    File = Path.GetFileName(file),
                    Value = r.Groups["value"].Value

This time, I've introduced the let keyword. This keyword is very interesting because it allows the creation of a query-local variable that can contain either collections or single objects. The content of this variable can be used in the where clause, as the source of another "from" query, or in the select statement.

In my case, I just wanted to have the result of the Regex match, so I'm just calling Regex.Match to validate the file name, and I'm placing the content of a Regex group in my resulting anonymous type.

Now, with all my files filtered I found that some XML files were not valid because they were not containing a specific node. So I filtered them again using this query :

    var files2 = from dir in Directory.GetDirectories(args[0], "*", SearchOption.AllDirectories)
                 from file in Directory.GetFiles(dir, "*.xml")
                 let r = match.Match(Path.GetFileName(file))
                 let c = XElement.Load(file).XPathSelectElements("//dummy")
                 where r.Success && c.Count() != 0
                 select new {
                    Path = dir,
                    File = Path.GetFileName(file),
                    Value = r.Groups["value"].Value

I've added a new let clause to add the loading of the file, and make sure there is a node named "dummy" somewhere in the xml document. By the way, if you're looking XPath in XLinq, just look over there.

You may wonder by looking at this query when the load is actually evaluated... Well, it is only evaluated when the c.Count() call is performed, which is only after the regex has matched the file ! This way, I'm not trying to load all the files returned by GetFiles. You need to always remember that queries are evaluated only when enumerated.

In conclusion, Linq is a very interesting piece of technology, definitely NOT reserved to querying databases. What I like the most is that one can write code almost without any loop, therefore reducing side effects.

If you haven't looking at Linq yet, just give it a try, you'll probably like it :)

Canadian Mobile Data Plans

By Jerome at April 13, 2008 20:06 Tags: ,

I've been interrogating myself a lot lately about the current state of mobile internet in Canada. I'm using my cell phone with a 25$ a month (excluding taxes) data plan for 5 Megabytes, which makes it almost useless for web browsing. Besides, using an HSDPA cell phone, I would deplete my data plan in about 5 minutes.

There's been a lot of buzz around the price of Canada data plans and the absence of the iPhone in Canada for the past year. Projections showed that using an iPhone would cost something like 300$ a month with data plans comparable to what AT&T is offering. I'm guessing that noone would be interested in paying that much to have a 500MB almost unlimited data plan. It seems that Rogers is not willing to let go of the current data plan rates to offer a service that appropriate to the iPhone.

I'm new to the Canadian environment, but for what I can tell when I sometimes hear that Canadians are not really into cell phones -- that they can live without it and do not really need it -- I have an impression of 'déjà vu'. French people had this kind of state of mind when there were only two carriers. People at that time also though they did not need cell phones. Except that it was not that they were technophics, it only was because it was darn too expensive !!

Now, prices in france have dropped a lot, and people are using a lot of services offered by the cell phone carriers. I'm insisting on the services word because I'm sensing that this is where canadians carriers are shooting themselves in the foot by only focusing on being "data pipes". They could expand their business by offering services that would be far more lucrative than only conveying data or voice. If I consider my own use of the voice plan, knowing that the person I am calling is paying the call when he did not initiate the call, makes me talk less.

Ok, there were some improvements the past few months, which Mark is pointing out, but which seem to have halted. Bell released a 7$/month plan which made Canada coming from the most expensive country for data plans to the less expensive in the world, which is a bit odd. With a twist though, it's HTC Touch only. The rest of "improvements" are crippled data plans that are only interesting if you're willing to use the internet that existed 10 years ago...

I'm guessing that breaking the monopoly will change the current state, and that the bidding for new frequencies will force existing carriers to lower their prices to keep their customer base. This is one bit of a stretch, but I'm comparing the state of the industry to the bad phase the music and movie industry is going through right now... There are now forced to understand that they can't sell their music as a product but as a service if they want to keep their business going.

The cell phone industry can be forced to do so to keep their customers, if some newcomer is not playing by the established rules by giving for instance, a flat rate of 45$/month, everything included. I'm not giving this example randomly; I'm referring to the French ISP Free.fr which made quite a perturbation when they offered for something like 45$ a month 100 TV channels, free phone calls to 100 countries in the world, 25Mbps internet, Wifi access point, and a lot more. All this with an excellent quality of service. They did not play by the established rules, and they are now the second most important player in the market and growing every day. I do not see any reason this would not happen for cell phone carriers the same way it did in France.

But maybe there is a good reason for all this though... Canada's a big "empty" space, and maybe expanding the cell coverage is not as money efficient as it is in France, or USA. I don't know all the details, so maybe I'm missing some things.

We'll see in the next few months...

A bug in VS2008 Code Analysis, Generics normal and nested classes

By Jerome at April 07, 2008 19:12 Tags: , ,

I've found a few days ago a small bug in the former "FxCop" now renamed Code Analysis part of Visual Studio 2008.

While compiling this little piece of code :

public class Dummy<T> where T : IDisposable
        public T Test
                new NestedDummy<T>(default(T));

        class NestedDummy<U>
            public NestedDummy(U item)
                this.Value = item;

            public U Value { get; private set; }

Which is a trimmed down version of the actual code, I saw a lot of errors like this :

MSBUILD : error : CA0001 : Rule=Microsoft.Reliability#CA2001, Target=ConsoleApplication1.Dummy`1+NestedDummy`1.#.ctor(!1) : The following error was encountered while reading module 'ConsoleApplication1': Could not resolve member reference: ConsoleApplication1.Dummy`1<type parameter.T>+NestedDummy`1<type parameter.U>::set_Value.

This means that for some reason, the Code Analysis tool is unable to parse the metadata to check for some analysis rule. This is not a blocking bug since it does not prevent the build from ending properly, but it displays a lot of error messages, which can be disturbing.

To fix to, I found two solutions : Either move the nested class out of its parents class, or remove the generic constraint on the parent class.

I posted the bug on Microsoft Connect, and I was pleasantly surprised to see that it has already been processed and David Kean from Microsoft wrote that the fix will be available in the next Service Pack of Visual Studio 2008.

Not a big issue but still, nice to see that Connect has an impact.

Bluetooth Remote Control 0.9.0, Round 2

By Jerome at April 01, 2008 18:39 Tags: ,

Two small errors have slipped into the configuration file of version 0.9.0, preventing it from controlling Windows Media Center. Since it is a rather small update, as this is only the application definition file that is modified, I did not increase the version number.

So, if you downloaded the 0.9.0 version before this post was published and if you use Windows Media Center, you may want to download it again.

Windows Server 2008 and Microsoft Bluetooth Stack trouble

By Jerome at March 24, 2008 13:49 Tags: ,

There's been a lot of Buzz lately about a "Windows Workstation 2008", which actually does not quite exist, but that should. It is actually installing Windows Server 2008 and making it a workstation platform, by enabling every workstation component that is disabled by default.

From my point of view, Vista is definitely interesting, though it has too many services that are enabled by default and that do not make sense in every situation. For a computer savvy user, all this stuff is not really interesting, and Windows Server 2008 with its "do not enable unused components" policy, is quite interesting.

I decided to give it a shot by installing it as my main (and only) laptop OS, and quite frankly, I'm pleasantly surprised ! I do get the same user experience that I did have with Windows Vista with Aero, the nifty new features like the new start menu, and I seem to get a performance improvement over Vista. (Performance improvement is only a feeling; I don't have any numbers to show, though some did).

Everything works as expected, except for the bluetooth part, for which I do not seem to be the only one having problem with. The microsoft stack does not seem to install completely, as there are three "unknown devices" left : BTH\MS_RFCOMM, BTH\MS_BTHBRB and BTH\MS_BTHPAN. All three of them are core components of the bluetooth stack, and are obiously needed to get bluetooth related software working properly. The interesting part is that there are actually all the driver and metadata files required to install these devices, but for some reason, Win2008 does not want to use them. The driver files seem to be identical to the files Vista SP1, so this is one bit of a mystery to me. Added to that, this installation issue seems to be related to the KB940199 where the infcache.1 file could not be found. Screwing with that file did not help either...

So as a backup plan, I decided to fall back on the Widcomm/Broadcom Stack with this guide, which seems to work fine, at least for the part I'm interested in, Bluetooth Remote Control . I still don't understand the licensing policy on this software... You need the hardware to get that software to work, why bother having an licensing scheme over this, haven't you already paid for it buying the hardware ?

Anyway, if you're a tech savvy user, give Windows Server 2008 a try as your workstation OS, you might be surprised :)

Now, I'm going back to adding new features to Bluetooth Remote Control !

Prevent ASP.NET web.config inheritance, and inheritInChildApplications attribute

By Jerome at March 23, 2008 12:34 Tags: , ,

Since I've changed my top level blog engine, I've had some troubles with YAF, the forum engine I'm using for my Remote Control software.

The forum I'm using is in a child directory, which is a child application defined in IIS as an other application. The BlogEngine.NET disables the use of Sessions, and YAF requires sessions to be enabled, plus BlogEngine.NET adds some custom HTTP handlers, which incidentally are not known but the forum application. This is quite a mess, and to be able to get both applications running without fine tuning each one to work with the other, I had to use the little known attribute inheritInChildApplications.

This attribute prevents an application from passing its configuration as a default to child applications. Using this attribute is a little tricky, and has to be used this way :

<!-- Root web.config file -->

<?xml version="1.0"?>


  <location path="." inheritInChildApplications="false">


      <compilation debug="false" />

      <!-- other configuration attributes -->




This way, any child application defined below this application will not use the current configuration. There's some mystery around the inheritInChildApplications attribute; it is not defined in the dotnetconfig.xsd file and it still is a rather helpful configuration option...

Visual Studio 2008 Solution Tree Items Collapse

By Jerome at March 17, 2008 19:26 Tags: ,

Even though there is a way to expand all nodes in the solution tree of Visual Studio 2008, there is no way to do the opposite, which is collapse all. Not collapse all top level nodes, but collapse all child nodes, one by one.

There's a bug in VS2005/2008 that prevents a node from being collapsed properly for some obscure reason. The Expanded property keeps on being "true" even if set to "false".

Fortunately, a fix by Scott Kuhl which was working with Visual Studio 2005 is also working with Visual Studio 2008. The script is doing some trick to simulate a DefaultAction each node, which seems to collapse a node without using the Expanded property. 

Nice trick :) It avoids me the burden of hitting the minus and enter keys numerous times...

Blog Update

By Jerome at March 16, 2008 17:54 Tags: , ,

Last Monday, I attended a GUVSM meeting with Rod Paddock and it was a great presentation. Rod is an interesting speaker, and he seems to have some sort of an independant view on Microsoft technologies. I'm also joining Guy Barette to thank Rod for his long trip visit in Montréal !

He's been talking about MIX'08 and all the Silverlight buzz. He was also here to talk about 10 Open Source tools worth looking at, and he's mentionned BlogEngine.NET. This blog engine kept my attention, not because it's particularly easy to install, but because it's able to use an XML datasource to store its data, that it has a pretty small foot print and that it is extensible.

Back at home, I gave it a look, and decided to switch to this engine. There's a bunch of features that can be a pain to develop and stabilize, like PingBack and TrackBack protocols. It's nice to have it out of the box.

I'm just hoping that it is going to increase the visibility of this blog. We'll see :)

.NET Threads, CallContext and ILogicalThreadAffinative

By Jerome at February 10, 2008 21:53 Tags: , ,

I've recently been looking for a way to automatically pass information from a thread's current call context to any thread that's been spawned from this thread. This can be useful for many reasons, and sometimes having some TLS information like the current user, or some custom context information.

This can be done by wrapping any thread entry point around some custom code that will effectively pass the context to the new thread. This is a bit annoying, especially if this is information that is generated internally by a framework, because it requires the user of the framework to always use the wrapping method.

There's a way around this by using the CallContext class, and particularly GetData/SetData methods. Problem is, if you set some data in the CallContext, it will not pass onto a spawned thread. Actually, it will if the type you are placing in the CallContext implements the ILogicalThreadAffinative interface.

This is a marker interface that is used to avoid context data that is not meant to "flow" through each spawned thread.

It's also interesting to know that ILogicalThreadAffinative flagged types will also be passed along to threads spawned by the thread pool, and incidentally to delegates enqueued via the BeginInvoke compiler generated method.

Finally, in case of a remoting call, any ILogicalThreadAffinative flagged type will also serialized to the remote context and be serialized back to the local context.

Being able to step through the Framework's code has been somehow a time saver to better understand this :)

Some news on Bluetooth Remote Control for Windows Mobile

By Jerome at January 15, 2008 01:00 Tags:

Some news around my little software utility...

I don't have plans for a major release yet, but maybe I'll make an other small release to add some really minor enhancements.

Stay tuned :) And thanks for using Bluetooth Remote Control for Windows Mobile !

About me

My name is Jerome Laban, I am a Software Architect, C# MVP and .NET enthustiast from Montréal, QC. You will find my blog on this site, where I'm adding my thoughts on current events, or the things I'm working on, such as the Remote Control for Windows Phone.