[WP7Dev] Beware of the [ThreadStatic] attribute on Silverlight for Windows Phone 7

By Admin at June 19, 2010 21:36 Tags: , , , , , , ,

Cet article est disponible en francais.

In other words, it is not supported !

And the worst in all this is that you don’t even get warned that it’s not supported... The code compiles, but the attribute has no effect at all ! Granted that you can read the msdn article about the differences between silverlight on Windows and Windows Phone, but well, you may still miss it. Maybe a custom code analysis rule could prevent this.

Still, you want to use ThreadStatic because you probably need it, somehow. But since it is not supported, you could try the Thread.GetNamedDataSlot, mind you.

Well, too bad. It’s not supported either.

That leaves us implementing or own TLS implementation, by hand...

 

Updating Umbrella for Silverlight on Windows Phone

I’m a big fan of Umbrella, and the first time I had to use Dictionary<>.TryGetValue and its magically aweful out parameter in my attempt to rewrite my Remote Control app for Windows Phone 7, I decided to port Umbrella to it. So I could use GetValueOrDefault without rewriting it, again.

I managed to get almost all the desktop unit tests to pass, except for those who emit code, use web features, use xml and binary serializers, call private methods using reflection, and so on.

There are a few parts where the code needed to be updated, because TypeDescriptor class is not available on WP7, you have to crash and burn to see if a value is convertible from one type to the other. But that’s not too bad, it works as expected.

 

Umbrella’s ThreadLocalSource

Umbrella has this nice ThreadLocalSource class that wraps the TLS behavior, and you can easily create a static variable of that type instead of the ThreadStatic static variable.

The Umbrella quick start samples make that kind of use for it :

    ISource<int> threadLocal = new ThreadLocalSource<int>(1);

    int valueOnOtherThread = 0;

    Thread thread = new Thread(() => valueOnOtherThread = threadLocal.Value);
    thread.Start();
    thread.Join();

    Assert.Equal(1, threadLocal.Value);
    Assert.Equal(0, valueOnOtherThread);

The main thread set the value to 1, and the other thread tries to get the same value from the other thread and it should be different (the default value of an int, which is 0).

 

Updating the ThreadLocalSource to avoid the use of ThreadStatic

The TLS in .NET is basically a dictionary of string/object pairs that is attached to each running threads. So, to mimic this, we just need to make a list of all threads that want to store something for themselves and wrap it nicely.

We can create a variable of this type :

    private static Tuple<WeakReference, IDictionary<string, T>>[] _tls;

That variable is intentionally an array to try to make use of memory spacial locality, and since on that platform we won’t get a lot of threads, this should be fine when we got through the array to find one. This approach is trying to be lockless, by using a retry mechanism to update the array. The WeakReference is used to avoid keeping a reference to the thread after it has been terminated.

So, to update the array, we can do as follows :

    private static IDictionary<string, T> GetValuesForThread(Thread thread)
    {
        // Find the TLS for the specified thread
        var query = from entry in _tls

                    // Only get threads that are still alive
                    let t = entry.T.Target as Thread

                    // Get the requested thread
                    where t != null && t == thread
                    select entry.U;

        var localStorage = query.FirstOrDefault();

        if (localStorage == null)
        {
            bool success = false;

            // The storage for the new Thread
            localStorage = new Dictionary<string, T>();

            while(!success)
            {
                // store the original array so we can check later if there has not
                // been anyone that has updated the array at the same time we did
                var originalTls = _tls;

                var newTls = new List<Tuple<WeakReference, IDictionary<string, T>>>();

                // Add the slots for which threads references are still alive
                newTls.AddRange(_tls.Where(t => t.T.IsAlive));

                var newSlot = new Tuple<WeakReference, IDictionary<string, T>>()
                {
                    T = new WeakReference(thread),
                    U = localStorage
                };

                newTls.Add(newSlot);

                // If no other thread has changed the array, replace it.
                success = Interlocked.CompareExchange(ref _tls, newTls.ToArray(), originalTls) != _tls;
            }
        }

        return localStorage;
    }

Instead of the array, another dictionary could be created but I’m not sure of the actual performance improvement that would provide, particularly for very small arrays.

Using a lockless approach like this one will most likely limit the contention around the use of that TLS-like class. There may be, from time to time, computations that are performed multiple times in case of race conditions on the update of the _tls array, but that is completely acceptable. Additionally, livelocks are also out of the picture on that kind of preemptive systems.

I think developing on that platform is going to be fully of little workarounds like this one... This is going to be fun !

Using Multiple Where Clauses in a LINQ Query

By Jay at December 06, 2008 15:32 Tags: , , ,
Cet article est disponible en francais.
 
After writing my previous article where I needed to intercept exceptions in a LINQ Query, I found out that it is possible to specify multiple where clauses in a LINQ Query.

Here is the query :


var q = from file in Directory.GetFiles(@"C:\Windows\Microsoft.NET\Framework\v2.0.50727", "*.dll")
        let asm = file.TryWith(f => Assembly.LoadFile(f))
        where asm != null
        let types = asm.TryWith(a => a.GetTypes(), (Exception e) => new Type[0])
        where types.Any()
        select new { asm, types };


This query is able to find assemblies for which it is possible to list types. The point of using multiple Where clauses is to avoir evaluating chunks of a query if previous chunks can prevent it. By the way, the TryWith around the Assembly.GetTypes() is there to intercept exceptions raised when loading types, in case dependencies would not be available at the moment of the enumeration.

A useful LINQ trick to remember !

F#, TryWith, Maybe and Umbrella

By Jay at December 06, 2008 13:49 Tags: , , ,
Cet article est disponible en Français.
 
I've thrown myself a bit in the discovery of F#, and even though I do not intend to make it my first language, I intend to use techniques and features found in it and try to port them into C#. New additions in C# 3.0 make it a good target for functional concepts.

There seem to be a consensus for the fact that F# is not a multi-purpose language, as C# is also not, for instance with the writing of parallel code. C# is not a perfect language for this, but F# seems to be. At the opposite, F# does not seem to be a language of choice for writing GUI code. For my part, and considering that F# if not really official, reusing concepts will be enough for now.

TryWith Extension

Using F#, I had to write this:


   let AllValidAssemblies = [
        for file in Directory.GetFiles(@"C:\Windows\Microsoft.NET\Framework\v2.0.50727", "*.dll") ->
        System.Reflection.Assembly.LoadFile(file)
    ]

This code creates a list of assemblies that can be loaded in the current AppDomain. There is however an issue with the invocation of the Assembly.LoadFile method, because it raises an exception when the file is not loadable for some reason. This is a non-modifiable behavior, even though we would like to return a null instead of an exception.

To work around this, there is a feature in F# that can do this :


    let EnumAllTypes = [
        for file in Directory.GetFiles(@"C:\Windows\Microsoft.NET\Framework\v2.0.50727", "*.dll") ->
        try System.Reflection.Assembly.LoadFile(file) with _ -> null
    ]

The point of the try/with block is to transform any exception into a null reference.

To transpose the creation of this list in C# with a LINQ query, the same problem arises. We must intercept the exception raised by LoadFile and convert it to a null reference.

Here is the equivalent in C#, without the exception handling :


    var q = from file in Directory.GetFiles(@"C:\Windows\Microsoft.NET\Framework\v2.0.50727", "*.dll")
            let asm = Assembly.LoadFile(file)
            select asm;

When LoadFile raises an exception, the execution of the request is interrupted, which is a problem.

Extension Methods can be of a great value here, and even though a normal method could do the trick, we can write this :


    public static class Extensions
    {
        public static TResult TryWith<TInstance, TResult>(this TInstance instance, Func<TInstance, TResult> action)
        {
           try {
              return action(instance);
           }
           catch {
              return default(TResult);
           }
        }
    }


The idea behind this method is to reproduce the behavior of the try/with F# construct. With this method, we can update the LINQ query into this :


    var q = from file in Directory.GetFiles(@"C:\Windows\Microsoft.NET\Framework\v2.0.50727", "*.dll")
            let asm = file.TryWith(f => Assembly.LoadFile(f))
            select asm;


This is creates the same list the F# code does, with null references for assemblies that could not be loaded.

The TryWith method can be overloaded to be a bit more flexible, like calling a method for a specific exception :



    public static TResult TryWith<TInstance, TResult, TException>(
           this TInstance instance, Func<TInstance, TResult> action,
           Func<TException, TResult> exceptionHandler
        )
        where TException : Exception
    {
        try {
           return action(instance);
        }
        catch (TException e) {
           return exceptionHandler(e);
        }
        catch {
           return default(TResult);
        }
    }


By the way, there is an interesting bug with this code. If we execute this :


    string value = null;
    var res = value.TryWith(s => s.ToString(), (Exception e) => "Exception");


The behavior is different depending on whether it is executed with or without the debugger with the x86 runtime. It seems that the code generator "forgets" to add the handler for the TException typed exception, which is annoying. This is not a big bug, mainly because it only appears when the x86 debugger is present. With the x64 runtime debugger, there is no problem though. For those interesting in this, the bug is on Connect.

Maybe Extension

I also added recently in Umbrella an extension named Maybe, which has a behavior rather similar to TryWith, but without the exceptions :


    public static TResult Maybe<TResult, TInstance>(
         this TInstance instance,
         Func<TInstance, TResult> function)
    {
       return instance == null ? default(TResult) : function(instance);
    }


The point of this method is to be able to execute code if the original value is not null. For instance :


    object instance = null;
    Console.WriteLine("{0}", instance.Maybe(o => o.GetType());


This allows the evaluation of the GetType call, only if "instance" is not null. With a method call like this, it is possible to write an "if" block, with inside a LINQ query, this because a bit more complex.

The idea for the code is not new and is similar to the functional Monad concept. It has been covered numerous times, and an implementation more in line with F# can be found on Matthew Podwysocki's Blog.

Pollution ?

When we're talking about pollution linked to Extension Methods, we're talking about Intellisense pollution. We can quickly find ourselves dealing with a bunch of extensions that are useful in the context of the current code, which renders Intellisense unusable. With Umbrella, these two extensions are somehow polluting all types, because they are generic without constraints.

Although these are only two very generic extensions, this can apply to almost any block of code, but they could find themselves better placed in an Umbrella Extension Point, rather than directly on every types.

We could have this :



    object instance = null;
    Console.WriteLine("{0}", instance.Control().Maybe(o => o.GetType());


But I have some issues with this approach : To be able to create an extension point the Control methd has to create an instance of the IExtensionPoint. This add a new instantiation during the execution, although the lambda also creates itself a hidden instance, we're counting anymore... There is also the fact that it lengthens the code line, but is only aesthetics. We'll see what pops out ...

Anyway, it is interesting to see the impact the learning a new language has on the style of writing code with another language that one's been using for a long time...

Lambda Expression and ForEach loops

By Jay at November 18, 2008 20:58 Tags: , ,

Cet article est disponible en français.

To enhance the performances of a type serializer, and to use a small extension that I recently wrote for Umbrella I stumbled upon an interesting small "Side Effect" seen when creating lambda expressions inside a foreach loop.

Let's take this simple piece of code :


    var actionList = new List<Func<int>>();

    foreach (var value in Enumerable.Range(0, 10))
    {
       actionList.Add(() => value);
    }

    actionList.ForEach(func => Console.Write("{0} ", func()));

Which outputs this :


   9 9 9 9 9 9 9 9 9 9



Which is, of course, what we could have expected.

Lambda expression have the ability to use variables that are in the scope when they are declared. This makes them very interesting, but to properly use them, it is best to understand how they are "materialized" by the compiler.

Like a lot of features of C#, like the using, foreach, iterators or lock, lambdas are syntactic sugar destined to simplify the writing of code that is most of the time pretty verbose. It is possible to write the expanded code for these keyswords in C#.

Let's take this other piece of code :


    int a = 0;
    Action action = () => Console.WriteLine(a);
    action();


The lambda expression is "materialized" by the C# compiler under the form of a "Display Class", that allows the storage of the local variable "a" :



    [CompilerGenerated]
    private sealed class <>c__DisplayClass1
    {
       public int a;

       public void <Main>b__0()
       {
          Console.WriteLine(this.a);
       }
    }

We can see that the indentifiers for the generated class are not valid in C#, but are valide from the CLR point of view. We can also see that the local variable used during the declaration is present as a member variable, in the class that contains the code of the lambda expression. The compiler will then write this to create an instance of the lambda expression :


    int a = 0;

    var display = new <>c__DisplayClass1();
    display.a = a;

    Action action = new Action(display.<Main>b__0);

    action();


There also, this is not valid C#.

But then, what happens for the foreach case so that the content of the variable is repeated ?

If we analyze the first code sample generated by the compiler with Reflector, there is nothing much fancy to see with the C# visualizer :


    List<Func<int>> actionList = new List<Func<int>>();
    using (IEnumerator<int> CS$5$0000 = Enumerable.Range(0, 10).GetEnumerator())
    {
       while (CS$5$0000.MoveNext())
       {
          int value = CS$5$0000.Current;
          actionList.Add(delegate {
             return value;
          });
       }
    }



The lambda expression is represented as an anonymous method, which is a synonym of lambda, but that does not explain the behavior.

We must look at the generated IL to understand the behavior, and this is the correct C# code that is generated :


    List<Func<int>> actionList = new List<Func<int>>();
    using (IEnumerator<int> CS$5$0000 = Enumerable.Range(0, 10).GetEnumerator())
    {
       var myLambda = new <>c__DisplayClass4();

       while (CS$5$0000.MoveNext())
       {
          int value = CS$5$0000.Current;

          myLambda.value = value;

          actionList.Add(new Func<int>(myLambda.b_0));
       }
    }



We can easily see what the problem is : The instance of the class containing the lambda is created only once, and reused many times to assign a new value for each iteration. This explains why the execution of all the lambdas return the last enumerated value, because they all refer to the same instance of the "DisplayClass" type.

However, if we write the code this way :


    foreach (var value in Enumerable.Range(0, 10))
    {
       int myValue = value;
       actionList.Add(() => myValue);
    }

The behavior changes, and this time, each lambda has the correct value.

From the compiler point of view, the creation of a new instance for the container class must be the consequence of the creation of a new variable. For the case of a ForEach statement, this is not the case and the variable is treated as created only once, then reused..

So, from the compiler point of view, a ForEach loop is expanded like this :


    using (var it = Enumerable.Range(0, 10).GetEnumerator())
    {
       int value;

       while (it.MoveNext())
       {
          value = it.Current;
          actionList2.Add(() => value);
       }
    }



This is probably a question of interpretation, but I wasn't exactly expecting this...

So, one must pay attention to the way local variables are used in lambda expressions, depending on their declaration location.

I'll explain in a later post why I did have to use lamda expressions in a ForEach loop.

Working with Umbrella in .NET 3.5

By Jay at November 08, 2008 16:06 Tags: ,
If you've been using .NET 3.5, and the new features that are provided by C# 3.0, and especially LINQ, you must have wondered why, oh why, there isn't an extension named ForEach on an IEnumerable<T>.

Well, I still haven't figured that out, though it appears that it might have something to do with the fact that by nature an Action<T> is most of the time not "pure", which means that it modifies some states, somewhere. I can't remember where I've found that explanation, though it "might" make sense from a functional point of view.

But you may wonder why, then, is there a ForEach method on List<T> ? Well, I don't know why there is that kind of consistency issue, but I know there is one library that somehow tries to fix this :
 
 
This library fills the gaps left by the BCL, and adds a whole bunch of new extension methods that eases the use of .NET 3.5 classes. This library is not a framework though, mainly because you don't need to redesign your whole application to use it.

A few weeks ago during a presentation of Umbrella at the .NET User Group of Montreal, the creators Francois Tanguay and Erik Renaud from nVentive, were asked the question "Where do we start ?". This is a tough question to answer, because of the nature of umbrella, which "plugs" itself everywhere it can.

Using Umbrella

Let’s see a simple example:

    Enumerable
        .Range(20, 30)
        .ForEach(
            (index, value) => "Value #{0}={1}".InvariantCultureFormat(index, value)
        );

As you can see, using Umbrella requires most of the time the use of lambda expressions. Here, this example is an answer to the question "How can I get the index of the enumerated value in a foreach statement ?".

Another extension provided by Umbrella is the Remove method on an ICollection by specifying a predicated to select elements to be removed.

            var values = new List<int>(Enumerable.Range(1, 10));
            values.Remove(v => v > 4);

Sure, you can write some code that does this, but you'll have to create a temporary list to store elements to be removed, then remove them from the collection.

There is also an extension on IDictionary<> that simplifies the use of TryGetValue which requires an “out” argument, which is particularly annoying to write. Umbrella provides a TryGetValueOrDefault that either gives you the value for the key you requested, or default(TValue) if the key is not found.
 

The Action.ToDisposable() extension method

One particularly interesting extension is the ability to encapsulate an Action delegate into an anonymous disposable instance. Let’s say that we need to profile the duration of a particular scope of code. We would need to write something like this :


    var w = Stopwatch.StartNew();
    for (int i = 0; i < 1000; i++)  { }
    Console.WriteLine(w.Elapsed);



We would have to write the enclosing code to time for each portion of code we would need to profile. You could of course write a class that would do this on purpose :


    class ProfileScope : IDisposable
    {
        Stopwatch w = Stopwatch.StartNew();
        void IDisposable.Dispose()
        {
            Console.WriteLine(w.Elapsed);
        }
    }


And it’s being used like with a “using” statement :


    using (new ProfileScope())
    {
        for (int i = 0; i < 1000; i++)
        {
        }
    }

This time, timing a block of code is somehow easier to write, however, there is a way to simplify the writing of the writing of the utility class by doing this:

    static IDisposable CreateProfileScope()
    {
        var w = Stopwatch.StartNew();
        return Actions.Create(() => Console.WriteLine(w.Elapsed)).ToDisposable();
    }

We can use here the ToDisposable extension to be able to call the specified action in the dispose method of some disposable class.

The point of this code is to avoid exposing the inner details of the profiling code, and just expose a known and generic way for executing something at the end of a scope.

Extension Points

There is one drawback when using extension methods: They tend to “pollute” the original type. By pollute I mean that they appear in the IntelliSense window. Because it is easy to add extension methods, you end up having hundreds of new methods on a type, which make IntelliSense much less useable.

The Umbrella guys came up with the idea of Extension Points, which are a way to group extension methods the namespaces do for types.

This is somehow a hack from a code perspective, but is rather ingenious from a usability perspective. For instance, extending the serialization applies to every type that exists, and placing all serialization extension methods directly on System.Object is not a good idea.

Extensions points are used like this:

    using (MemoryStream stream = new MemoryStream())
    {
        int a = 0;
        a.Serialization().Binary(stream);
        // ...
    }



Serialization is an extension method placed on every object that returns an SerializationExtensionPoint instance, and the “Binary” method extends that type instead of System.Object.

It is important to remember that Umbrella is for most of its code is not "rocket science", as it probably contains code that you may already have partially developed for your own project. It's only the amount of small but useful utility methods that Umbrella provides that makes it worth using.

I’ll be writing a few other posts about Umbrella; there are a lot of extensions in there that are great time savers.

 

About me

My name is Jerome Laban, I am a Software Architect, C# MVP and .NET enthustiast from Montréal, QC. You will find my blog on this site, where I'm adding my thoughts on current events, or the things I'm working on, such as the Remote Control for Windows Phone.