[WPDev] The hidden cost of IL Jitting

By jay at December 02, 2011 22:35 Tags: , , , , , , ,

We’ve gotten used to it. Method jitting is negligible. Or is it really ?

 

IL JITing

The compilation from IL to the native architecture assembly language (or JITting) has been part of the CLR from the very beginning. That’s an operation that was added to make the code execute faster, as interpreting the IL was too slow. By default, it’s happening on the fly, when the code path comes to a method that needs to be jitted, and that impacts the code speed when executing the method the first time.

That compilation step is not exactly free. A lot of code analysis and CPU specific optimizations are performed during this step. This is what arguably makes already JITted code run faster than generic compiled code, where the compiler has no knowledge of the target architecture.

This analysis takes a bit of time, but it is taking less and less time to execute, due to CPUs getting faster, or multi-core JITting features like the one found in .NET 4.5.

We’ve come to a point, on desktop and server machines, where the JIT time is negligible, since it’s gotten fast enough not to be noticed, or be an issue, in the most common scenarios.

Still, if there were times when JITing would be an issue, like it used to be around .NET 1.0, NGEN would come to the rescue. This tool (available in a standard .NET installation) pre-compiles the assemblies for the current platform, and creates native images stored on the disk. When an assembly is NGENed, they appear in the debugger’s “module” window named as “your_assembly.il.dll”, along with some other fancy decorations.

But while there are some caveats, like the restrictions with cross assembly method inline being ignored. It always comes down to a balance between start-up speed and code execution speed.

 

JITing on Windows Phone

On the phone though, CPU is very limited, especially on Generation 1 (Nodo) phones. The platform is too, considering is relative young age. At least on surface.

We’ve got used to create quite a bit of code to ease the development, add levels of abstraction for many common concepts, and lately, for asynchrony.

I’ll take the example of Reactive Extensions (Rx) in this article, just to make a point.

If you execute the following code on a Samsung Focus:

    
    List<timespan> watch = new List<timespan>();

    var objectObservable = Observable.Empty<object>();

    var w = Stopwatch.StartNew();
    Observable.Timeout<object>(objectObservable, TimeSpan.FromSeconds(1));
    watch.Add(w.Elapsed);

    w = Stopwatch.StartNew();
    Observable.Timeout<object>(objectObservable, TimeSpan.FromSeconds(1));
    watch.Add(w.Elapsed);

    output.Text = string.Join(", ", watch.Select(t => t.TotalMilliseconds.ToString()));

You'll consistently get something similar to this :

    20.60, 1.19


Calling an Rx method like this does almost nothing, it’s basicallt just setup. But 20ms is a long time ! Particularly when done on the UI thread, or any other thread for that matter.

These rough measurements tend to show that the Windows Phone platform (as of Mango at least) is not performing any NGEN like pre-jitting, leaving the app the burden of jitting code on the fly.

Still, not everything can be accounted to JITing, there must be type metadata loading, type constructors that are called.

 

Generating code with T4

So to sort that out a bit more, let’s use some T4 templates to generate code and isolate the JIT a bit more :

<#@ template language="C#" #>
using System;
using System.Collections.Generic;

public class Dummy
{
   public static void Test()
   {
      List<int> list = new List<int>();

      <#for (int i = 0; i < 100; i++) { #>
	  list.Add(<#= i.ToString() #>);
      <#} #>
   }
}

 

For different values of iterations, here's what gets out, when timing the call to the method :

Calls First call Subsequent calls
100 1.6ms > 0.03ms
1000 15.7ms > 0.09ms
5000 72.8ms > 2 ms
10000 148ms > 2ms

 

While this type of code is not exactly a good real-life scenario, this shows a bit the cost the IL jitting step. These are very simple method calls, no branching instructions, no virtual calls, … in short, nothing complex.

But with real code, the IL is a bit more intricate, and there’s got to be more logic involved in the JIT when generating the native code.

 

Wrapping up

Unfortunately, there’s not much that can be done here, except by reducing the amount of actual lines of IL that are generated. But that can be a though job, particularly when customers are expecting a lot from applications.

One could suggest to push as much code as  possible on a background thread, even code that seemingly does nothing particularly expensive. But that cannot always be performed on an other thread, particularly if that code depends on UI elements.

Finally, pre-jitting assemblies when installing the applications could be an interesting optimization for the Windows Phone platform, and I’m wondering why this has not made its way to the platform yet…

Asynchronous Programming with the Reactive Extensions (while waiting for async/await)

By jay at November 25, 2011 20:43 Tags: , , , , , ,

This article was originally published on the MVP Award Blog in December 2011.

Nowadays, with applications that use more and more services that are in the cloud, or simply perform actions that take a user noticeable time to execute, it has become vital to program in an asynchronous way.

But we, as developers, feel at home when thinking sequentially. We like to send a request or execute a method, wait for the response, and then process it.

Unfortunately for us, an application just cannot wait synchronously for a call to end anymore. Reasons can be that the user expects the application to continue responding, or because the application joins the results of multiple operations, and it is necessary to perform all these operations simultaneously for good performance.

Frameworks that are heavily UI dependent (like Silverlight or Silverlight for Windows Phone) are trying the force the developer's hand into programming asynchronously by removing all synchronous APIs. This leaves the developer alone with either the Begin/End pattern, or the plain old C# events. Both patterns are not flexible, not easily composable, often lead to memory leaks, and are just plain difficult to use or worse, to read.

C# 5.0 async/await

Taking a quick look at the not so distant future, Microsoft has taken the bold approach to augment its new .NET 4.5 to include asynchronous APIs and in the case of the Windows Runtime (WinRT), restrict some APIs to be asynchronous only. These are based on the Task class, and are backed by languages to ease asynchronous programming.

In the upcoming C# 5.0 implementation, the async/await pattern is trying to handle this asynchrony problem by making asynchronous code look synchronous. It makes asynchronous programming more "familiar" to developers.

If we take this example:

    static void Main(string[] args)
    {
        // Some initialization of the DB...
        Task<int> t = GetContentFromDatabase();

        // Execute some other code when the task is done
        t.ContinueWith(r => Console.WriteLine(r.Result));

        Console.ReadLine();
    }

    public static async Task<int> GetContentFromDatabase()
    {
        int source = 22;

        // Run starts the execution on another thread
        var result = (int) await Task.Run(
            () => { 
                // Simulate DB access
                Thread.Sleep(1000);
                return 10; 
            }
        );

        return source + result * 2;
    }

The code in GetContentFromDatabaselooks synchronous, but under the hood, it's actually split in half (or more) where the await keyword is used.

The compiler is applying a technique used many times in the C# language, known as syntactic sugar. The code is expanded to a form that is less readable, but is more of a plumbing code that is painful to write – and get right – each time. The using statement, iterators and more recently LINQ are very good examples of that syntactic sugar.

Using a plain old thread pool call, the code actually looks a lot more like this, once the compiler is done:

    public static void Main()
    {
        MySyncMethod(result => Console.WriteLine(result));
        Console.ReadLine();
    }

    public static void GetContentFromDatabase (Action<int> continueWith)
    {
        // The first half of the async method (with QueueUserWorkItem)
        int source = 22;

        // The second half of the async method
        Action<int> onResult = result => {
            continueWith(source + result * 2);
        };

        ThreadPool.QueueUserWorkItem(
            _ => {
                // Simulate DB access
                Thread.Sleep(1000);

                onResult(10);
            }
        );
    }

This sample somewhat more complex, and does not properly handle exceptions. But you probably get the idea.

Asynchronous Development now

Nonetheless, you may not want or will be able to use C# 5.0 soon enough. A lot of people are still using .NET 3.5 or even .NET 2.0, and new features like async will take a while to be deployed in the field. Even when the framework has been offering it for a long time, the awesome LINQ (a C# 3.0 feature) is still being adopted and is not that widely used.

The Reactive Extensions (Rx for friends) offer a framework that is available from .NET 3.5 and functionality similar to C# 5.0, but provide a different approach to asynchronous programming, more functional. More functional is meaning fewer variables to maintain states, and a more declarative approach to programming.

But don't be scared. Functional does not mean abstract concepts that are not useful for the mainstream developer. It just means (veryroughly) that you're going to be more inclined to separate your concerns using functions instead of classes.

But let's dive into some code that is similar to the two previous examples:

    static void Main(string[] args)
    {
        IObservable<int> query = GetContentFromDatabase();

        // Subscribe to the result and display it
        query.Subscribe(r => Console.WriteLine(r));

        Console.ReadLine();
    }

    public static IObservable<int> GetContentFromDatabase()
    {
        int source = 22;

        // Start the work on another thread (using the ThreadPool)
        return Observable.Start<int>(
                   () => {
                      Thread.Sleep(1000);
                      return 10;
                   }
               )

               // Project the result when we get it
               .Select(result => source + result * 2);
    }

From the caller's perspective (the main), the GetContentFromDatabase method behaves almost the same way a .NET 4.5 Task would, and the Subscribe pretty much replaces the ContinueWith method.

But this simplistic approach works well for an example. At this point, you could still choose to use the basic ThreadPoolexample shown earlier in this article.

A word on IObservable

An IObservable is generally considered as a stream of data that can push to its subscribers zero or more values, and either an error or completion message. This “Push” based model that allows the observation of a data source without blocking a thread. This is opposed to the Pull model provided by IEnumerable, which performs a blocking observation of a data source. A very good video with Erik Meijer explains these concepts on Channel 9.

To match the .NET 4.5 Task model, an IObservable needs to provide at most one value, or an error, which is what the Observable.Start method is doing.

A more realistic example

Most of the time, scenarios include calls to multiple asynchronous methods. And if they're not called at the same time and joined, they're called one after the other.

Here is an example that does task chaining:

    public static void Main()
    {
        // Use the observable we've defined before
        var query = GetContentFromDatabase();

              // Once we get the token from the database, transform it first
        query.Select(r => "Token_" + r)

             // When we have the token, we can initiate the call to the web service
             .SelectMany(token => GetFromWebService(token))

             // Once we have the result from the web service, print it.
             .Subscribe(_ => Console.WriteLine(_));
    }

    public static IObservable<string> GetFromWebService(string token)
    {
        return Observable.Start(
            () => new WebClient().DownloadString("http://example.com/" + token)
        )
        .Select(s => Decrypt(s));
    }

The SelectMany operator is a bit strange when it comes to the semantics of an IObservable that behaves like a Task. It can then be thought of a ContinueWith operator. The GetContentFromDatabase only pushes one value, meaning that the provided lambda for the SelectMany is only called once.

Taking the Async route

A peek at WinRT and the Build conference showed a very interesting rule used by Microsoft when moving to asynchronous API throughout the framework. If an API call nominally takes more than 50ms to execute, then it's an asynchronous API call.

This rule is easily applicable to existing .NET 3.5 and later frameworks by exposing IObservable instances that provide at most one value, as a way to simulate a .NET 4.5 Task.

Architecturally speaking, this is a way to enforce that the consumers of a service layer API will be less tempted to synchronously call methods and negatively impact the perceived or actual performance of an application.

For instance, a "favorites" service implemented in an application could look like this, using Rx:

    public interface IFavoritesService
    {
        IObservable<Unit> AddFavorite(string name, string value);
        IObservable<bool> RemoveFavorite(string name);
        IObservable<string[]> GetAllFavorites();
    }

All the operations, including ones that alter content, are executed asynchronously. It is always tempting to think a select operation will take time, but we easily forget that an Addoperation could easily take the same amount of time.

A word on unit: The name comes from functional languages, and represents the void keyword, literally. A deep .NET CLR limitation prevents the use of System.Void as a generic type parameter, and to be able to provide a void return value, Unit has been introduced.

Wrap up

Much more can be achieved with Rx but for starters, using it as a way to perform asynchronous single method call seems to be a good way to learn it.

Also, a note to Rx experts, shortcuts have been taken to explain this in the most simple form, and sure there are many tips and tricks to know to use Rx effectively, particularly when it is used all across the board. The omission of the Completed event is one of them.

Finally, explaining the richness of the Reactive Extensions is a tricky task. Even the smart guys of the Rx team have a hard time doing so... I hope this quick start will help you dive into it!

WinRT and the syntactic sugar around .NET event handlers

By jay at October 17, 2011 19:48 Tags: , , , , ,

If you've watched the great number of videos available from the Build conference, you've probably noticed that the layer between .NET and WinRT is very thin.

So thin that in permeates through to C# 5.0, even though it's not immediately visible to the naked eye.

 

Also, that Windows 8 developer preview is pretty stable... I'm writing this blog post using it, and it's pretty good :) (lovin' the inline spell checker, everywhere !!)

 

What about WinRT ?

The Windows Runtime has been explained here, there and by Miguel de Icasa (and there too by Julien Dollon), but to summarize in other words, WinRT is (at least for now) the new way to communicate with the Windows Core, with an improved developer experience. It's the new preferred (and only, as far as I know) way to develop Metro style applications, in many languages like C#/F#/VB, C++, JavaScript and more...

The API is oriented toward developing tablet applications, with the power and connectivity limitation that kind of platform has, plus the addition of what makes Windows Phone so interesting. That means Live Tiles, background agents, background transfers, XAML, background audio, social APIs, camera, sensors, location, and new features like sharing and search contracts, ...

My favorite part of all this is the new addition of a rule that make a LOT of sense : If an API call nominally takes more than 50ms to execute, then it's an asynchronous api call. But not using the ugly Begin/End pattern, rather through the nice async/await pattern, WinRT style (I'll elaborate on that in a later post). I've even started to apply that rule to my existing development with the Reactive Extensions (And that's yet an other later post).

Microsoft has taken the approach of cleaning up the .NET framework with the ".NET Core" profile. For instance, the new TypeInfo class now separates the introspection part from the type safety part that were historically merged in the System.Type type. This segregation limits the loading of type metadata only when necessary, and not when just doing a simple typeof(). Now, the System.Type type is fairly simple, and to get back all the known methods like GetMethods() or GetProperties() there's an extension method called Type.GetTypeInfo() in System.Reflection that gives back all the reflection side.

There are a lot of other differences, I'll discuss in a later post. (yeah, that's a lot to talk about !)

For the .NET developer, WinRT takes the form of *.winmd files that follow the .NET standard metadata format (kind of like TLB files on steroids, if you know what I mean...). These files can be directly referenced from .NET code like any other assembly, it's then very easy to call the underlying Windows platform. No more P/Invoke.

Just before you start freaking out, WinRT does not replace the standard .NET 4.5 full platform you already know, remember that. That's just a new profile, much like Windows Phone or Xbox 360 are profiles, but targeted at Metro style applications. (It's not applications anymore, it's apps :) just so you know...)

 

But how thin is the layer, really ?

To accommodate all these languages, compromises had to be made and underneath, WinRT is native code. Native code means no garbage collection, limited value types, a pretty different exception handling (SEH), and so on.

The CLR and C# compiler teams have made a great job at trying to hide all this but there are still some corner cases where you can see those differences appear.

For instance, you'll find that there are two EventHandler types : the existing System.EventHandler, and the new Windows.UI.Xaml.EventHandler. What's the difference ? See for yourself :

namespace System
{
    [ComVisible(true)]
    public delegate void EventHandler(object sender, EventArgs e);
}

And the other one :

namespace Windows.UI.Xaml
{
    // Summary:
    //     Represents a basic event handler method.
    [Version(100794368)]
    [WebHostHidden]
    [Guid(3817893849, 19739, 19144, 164, 60, 195, 185, 8, 116, 39, 152)]
    public delegate void EventHandler(object sender, object e);
}

The difference is subtle, but it's there : the second parameter is an object. This is kind of troubling, and having to juggle between the two is going to be a bit messy. That's going to be the forced return of conditional compilation and the myriads of #if and #endif...

But the difference does not stop here though. Let's look at how the WinRT handler can be used :

public class MyCommand : Windows.UI.Xaml.Input.ICommand
{
    public event Windows.UI.Xaml.EventHandler CanExecuteChanged;

    public bool CanExecute(object parameter) { }

    public void Execute(object parameter) { }
}

Translates to this, after the compiler does its magic :

using System.Runtime.InteropServices.WindowsRuntime;
public class MyCommand : Windows.UI.Xaml.Input.ICommand
{
    public event Windows.UI.Xaml.EventHandler CanExecuteChanged
    {
        add
        {
            return this.CanExecuteChanged.AddEventHandler(value);
        }
        remove
        {
            this.CanExecuteChanged.RemoveEventHandler(value);
        }
    }

    public bool CanExecute(object parameter) { }

    public void Execute(object parameter) { }

    public MyCommand()
    {
        this.CanExecuteChanged = 
           new EventRegistrationTokenTable();
    }
}

The delegates are not stored in a multicast delegate instance like they used to be, but are now stored in an EventRegistrationTokenTable type instance, and provides a return value for the add handler ! Also, the remove handler "value" is a EventRegistrationToken instance.

That construct is so new that even the intellisense engine is mistaken by this new syntax if you try to write it by yourself, but it compiles correctly.

The return value is of type EventRegistrationToken, and I'm guessing that it must be something WinRT must keep track of to call marshaled managed delegates.

The calling part is also very interesting, if you try to register to that event :

// Before
MyCommand t = new MyCommand();
t.CanExecuteChanged += (s, e) => { };
// After
MyCommand t = new MyCommand();
WindowsRuntimeMarshal.AddEventHandler(
   new Func(t.add_CanExecuteChanged)
 , new Action(t.remove_CanExecuteChanged)
 , delegate(object s, object e) { }
);

Quite different, isn't it ?

But this syntactic sugar seems only to be related to the fact that the WinRT EventHandler delegate type is exposed as a implemented interface member, like in ICommand. It does not appear if it is used somewhere else.

 

Cool. Why should care ?

Actually, you may not care at all, unless you write ICommand implementations.

If you write a command, and particularly ICommand wrappers or proxies, you may want to write your own add/remove handlers and to be able to do so, you'll need to return that EventRegistrationToken too, and map that token to your delegate.

Here's what I came up with :

public class MyCommand : Windows.UI.Xaml.Input.ICommand
{
    EventRegistrationTokenTable _table = new EventRegistrationTokenTable();
    Dictionary _reverseTable = new Dictionary();
        
    public event EventHandler CanExecuteChanged
    {
        add
        {
            var token = _table.AddEventHandler(value);
            _reverseTable[token] = value;

            // do something with value

            return token;
        }

        remove
        {
            // Unregister value 
            RemoveMyHandler(_reverseTable[value]);

            _table.RemoveEventHandler(value);
        }
    }
}

All this because the EventRegistrationTokenTable does not expose a bi-directional mapping between event handlers and their tokens.

But remember, WinRT and Dev11 are in Developer Preview state. That's not even beta. This will probably change !

Why using a timer may not be the best idea

By jay at July 25, 2011 00:00 Tags: , , , , , ,

TLDR: The Reactive Extensions provide a TestScheduler class that abstracts time and allows for a precise control of a virtual time base. Using the Rx Schedulers mecanism instead of real timers enables the creation of fast running unit tests that validate time related algorithms without the inconvients of the actual time.

 

Granted, the title is a bit provocative, but nonetheless it's still a bad idea to use timer classes like System.Threading.Timer.

Why is that ? Because classes like this one are based on the actual time, and that makes it a problem because it is non-deterministic. This means that each time you want to test a piece of code that depends on time, you'll be having a somehow different result, and particularly if you're using very long delays, like a few hours, you probably will not want to wait that long to make sure your code works as expected.

What you want actually, to avoid the side effect of "real" time passing by, is virtual time.

 

Abstracting Time with the Reactive Extensions

The reactive extensions are pretty good at abstracting time, with the IScheduler interface and TestScheduler class.

Most operators in the Rx framework have an optional scheduler they can use, like the Dispatcher or the ThreadPool schedulers. These schedulers are used to change the context of execution of the OnNext, OnCompleted and OnError events.

But for the case of time, the point is to freeze the time and make it "advance" to the point in time we need, and most importantly when we need it.

Let's say that we have a view model with a command that performs a lengthy action on the network, and that we need that action to timeout after a few seconds.

Consider this service contract :

public interface IRemoteService
{
    /// 
    /// Gets the data from the server,
    /// returns an observable call to the server 
    /// that will provide either no values, one
    /// value or an error.
    /// 
    IObservable<string> GetData(string url);
}

This implementation is exposing an observable based API, where the consumer of this contract must take into account the fact that getting data must be performed asynchronously, because it may not provide any value for a long time or even not return anything at all.

Next, a method of a view model that is using it :

public void GetServerDataCommand()
{
    IsQueryRunning = true;
    ShowError = false;

    Service.GetData(_url)
           .Timeout(TimeSpan.FromSeconds(15))
           .Finally(() => IsQueryRunning = false)
           .Subscribe(
               s => OnQueryCompleted(s),
               e => ShowError = true
           );
}

And we can test it using Moq like this :

var remoteService = new Mock<IRemoteService>();

// Returns far in the future
remoteService.Setup(s => s.GetData(It.IsAny()))
             .Returns(
                Observable.Return("the result")
                          .Delay(TimeSpan.FromDays(1))
             );

var model = new ViewModel(remoteService.Object);

// Call the fake server
model.GetServerDataCommand();
Assert.IsTrue(model.IsQueryRunning);
Assert.IsFalse(model.ShowError);

// Sleep for a while, before the timeout occurs
Thread.Sleep(TimeSpan.FromSeconds(5));
Assert.IsTrue(model.IsQueryRunning);
Assert.IsFalse(model.ShowError);

// Sleep beyond the timeout
Thread.Sleep(TimeSpan.FromSeconds(11));

// Do we have an error ?
Assert.IsFalse(model.IsQueryRunning);
Assert.IsTrue(model.ShowError);

The problem with this test is that it depends on actual time, meaning that it takes at least 16 seconds to complete. This is not acceptable in a automated tests scenario, where you want your tests to run as fast as possible.

 

Adding the IScheduler in the loop

We can introduce the use of an injected IScheduler instance into the view model, like this :

.Timeout(TimeSpan.FromSeconds(15), _scheduler)

Meaning that the both Start and Timeout will get executed on the scheduler we provide, for which the time is controlled.

We can update the test like this :

var remoteService = new Mock<IRemoteService>();
var scheduler = new TestScheduler();

// Never returns
remoteService.Setup(s => s.GetData(It.IsAny<string>()))
             .Returns(
                Observable.Return("the result", scheduler)
                          .Delay(TimeSpan.FromDays(1), scheduler)
            );

var model = new ViewModel(remoteService.Object, scheduler);

// Call the fake server
model.OnGetServerData2();
Assert.IsTrue(model.IsQueryRunning);
Assert.IsFalse(model.ShowError);

// Go before the failure point
scheduler.AdvanceTo(TimeSpan.FromSeconds(5).Ticks);
Assert.IsTrue(model.IsQueryRunning);
Assert.IsFalse(model.ShowError);

// Go beyond the failure point
scheduler.AdvanceTo(TimeSpan.FromSeconds(16).Ticks);

// Do we have an error ?
Assert.IsFalse(model.IsQueryRunning);
Assert.IsTrue(model.ShowError);

When the scheduler is created, it holds of all the scheduled operations until AdvanceTo is called. Then the scheduled actions are executed according to the virtual current time.

That way, your tests run at full speed, and you can test properly your time depedent code.

[Rx] Using the ObserveOn and SubscribeOn operators

By jay at July 24, 2011 20:04 Tags: , , , , ,

TLDR: This post talks about how the Reactive Extensions ObserveOn operator changes the execution context (the thread) of the IObservable OnNext/OnComplete/OnError methods, whereas the SubscribeOn operator changes the execution context of the implementation of the Subscribe method in the chain of observers. Both methods can be useful to improve the performance of an application's UI by putting work on background threads.

 

When developing asynchronous code, or consuming asynchronous APIs, you find yourself forced to use specific methods on a specific thread.

The most prominent examples being WPF/Silverlight and Winforms, where you cannot use UI bound types outside of the UI Thread. In the context of WPF, you'll find yourself forced to use the Dispatcher.Invoke method to manipulate the UI in the proper context.

However, you don't want to execute everything on the UI Thread, because the UI performance relies on it. Doing too much on the UI thread can lead to a very bad percieved performance, and angry users...

 

Rx framework's ObserveOn operator

I've discussed a few times the use of ObserveOn in the context of WP7, where it is critical to leave the UI thread alone and avoid choppy animations, for instance.

The ObserveOn operator changes the context of execution (scheduler) of a chain of operators until an other operator changes it.

To be able to demonstrate this, let's write our own scheduler :

public class MyScheduler : IScheduler
{
    // Warning: ThreadStatic is not supported on Windows Phone 7.0 and 7.1
    // This code will not work properly on this platform.
    [ThreadStatic]
    public static int? SchedulerId;

    private int _schedulerId;
    private IScheduler _source;
        
    public MyScheduler(IScheduler source, int schedulerId)
    {
        _source = source;
        _schedulerId = schedulerId;
    }

    public DateTimeOffset Now { get { return _source.Now; } }

    public IDisposable Schedule<TState>(
              TState state, 
              Func<IScheduler, TState, IDisposable> action
           )
    {
        return _source.Schedule(state, WrapAction(action));
    }

    private Func<IScheduler, TState, IDisposable> WrapAction<TState>(
              Func<IScheduler, TState, IDisposable> action)
    {
        return (scheduler, state) => {

            // Set the TLS with the proper ID
            SchedulerId = _schedulerId;

            return action(_source, state);
        };
    }
}

This scheduler's purpose is to intercept calls to the ISchedule methods (You'll fill the missing Schedule methods by yourself) and flag them with a custom thread ID. That way, we'll know which scheduler is executing our code.

Note that this code will not work properly on Windows Phone 7, since ThreadStaticAttribute is not supported. And it's still not supported on 7.1... Seems like not enough people are using ThreadStatic to make its way to the WP7 CLR...

Anyway, now if we write the following Rx expression :

Observable.Timer(TimeSpan.FromSeconds(1), new MyScheduler(Scheduler.ThreadPool, 42))
          .Do(_ => Console.WriteLine(MyScheduler.SchedulerId))
          .First();

We force the timer to raise OnNext on the ThreadPool through our scheduler, and we'll get the following :

42

Which means that the lambda passed as a parameter to the Do operator got executed in the context of the Scheduler used when declaring the Timer operator.

If we go a bit farther :

Observable.Timer(TimeSpan.FromSeconds(1), new MyScheduler(Scheduler.ThreadPool, 42))
          .Do(_ => Console.WriteLine("Do(1): " + MyScheduler.SchedulerId))
          .ObserveOn(new MyScheduler(Scheduler.ThreadPool, 43))
          .Do(_ => Console.WriteLine("Do(2): " + MyScheduler.SchedulerId))
          .ObserveOn(new MyScheduler(Scheduler.ThreadPool, 44))
          .Do(_ => Console.WriteLine("Do(3): " + MyScheduler.SchedulerId))
          .Do(_ => Console.WriteLine("Do(4): " + MyScheduler.SchedulerId))
          .First();

We'll get the following :

Do(1): 42
Do(2): 43
Do(3): 44
Do(4): 44

Each time a scheduler was specified, the following operators OnNext delegates were executed on that scheduler.

In this case, we're using the Do operator which does not take a scheduler as a parameter. There some operators though, like Delay, that implicitly use a scheduler that changes the context.

Using this operator is particularly useful when the OnNext delegate is performing a context sensitive operation, like manipulating the UI, or when the source scheduler is the UI and the OnNext delegate is not related to the UI and can be executed on an other thread.

You'll find that operator handy with the WebClient or GeoCoordinateWatcher classes, which both execute their handlers on the UI thread. Watchout for Windows Phone 7.1 (mango) though, this may have changed a bit.

 

An Rx Expression's life cycle

Using an Rx expression is performed in a least 5 stages :

  • The construction of the expression,
  • The subscription to the expression,
  • The optional execution of the OnNext delegates passed as parameters (whether it be observers or explicit OnNext delegates),
  • The observer chain gets disposed either explicitly or implicitly,
  • The observers can optionally get collected by the GC.

The third part's execution context is covered by ObserveOn. But for the first two, this is different.

The expression is constructed like this : 

var o = Observable.Timer(TimeSpan.FromSeconds(1));

Almost nothing's been executed here, just the creation of the observers for the entire expression, in a similar way IEnumerable expressions work. Until you call the IEnumerator.MoveNext, nothing is performed. In Rx expressions, until the Subscribe method is called, nothing is happening.

Then you can subscribe to the expression :

var d = o.Subscribe(_ => Console.WriteLine(_));

At this point, the whole chain of operators get their Subscribe method called, meaning they can start sending OnNext/OnError/OnComplete messages.

 

The case of Observable.Return and SubscribeOn

Then you meet that kind of expressions :

Observable
   .Return(42L)
   // Merge both enumerables into one, whichever the order of appearance
   .Merge(
      Observable.Timer(
         TimeSpan.FromSeconds(1), 
         new MyScheduler(Scheduler.ThreadPool, 42)
      )
   )
   .Subscribe(_ => Console.WriteLine("Do(1): " + MyScheduler.SchedulerId));

Console.WriteLine("Subscribed !");

This expression will merge the two observables into one that will provide two values, one from Return and one from the timer.

And this is the output :

Do(1):
Subscribed !
Do(1): 42

The Observable.Return OnNext was executed during the call to Subscribe, and has that thread has no SchedulerId, meaning that a whole lot of code has been executed in the context of the caller of Subscribe. You can imagine that if that expression is complex, and that the caller is the UI Thread, that can become a performance issue.

This is where the SubscribeOn operator becomes handy :

Observable
   .Return(42L)
   // Merge both enumerables into one, whichever the order of appearance
   .Merge(
      Observable.Timer(
         TimeSpan.FromSeconds(1), 
         new MyScheduler(Scheduler.ThreadPool, 42)
      )
   )
   .SubscribeOn(new MyScheduler(Scheduler.ThreadPool, 43))
   .Subscribe(_ => Console.WriteLine("Do(1): " + MyScheduler.SchedulerId));

Console.WriteLine("Subscribed !");

You then get this :

Subscribed !
Do(1): 43
Do(1): 42

The first OnNext is now executed under of a different scheduler, making subscribe a whole lot faster from the caller's point of view.

 

Why not always Subscribe on an other thread ?

That might come in handy, but you may not want that as an opt-out because of this scenario :

Observable.FromEventPattern(textBox, "TextChanged")
          .SubscribeOn(new MyScheduler(Scheduler.ThreadPool, 43))
          .Subscribe(_ => { });

Console.WriteLine("Subscribed !");

You'd get an "Invalid cross-thread access." System.UnauthorizedAccessException, because yo would try to add an event handler to a UI element from a different thread. 

Interestingly though, this code does not work on WP7 but does on WPF 4.

An other scenario may be one where delaying the subscription may loose messages, so you need to make sure you're completely subscribed before raising events.

 

So there you have it :) I hope this helps you understand a bit better those two operators.

[WP7] HttpWebRequest and the Flickr app "Black Screen" issue

By jay at April 22, 2011 14:54 Tags: , , , , ,

TL;DR: While trying to fix the "Black Screen" issue of the Windows Phone 7 flickr app 1.3, I found out that HttpWebRequest is internally making a synchronous call to the UI Thread, making a network call negatively impact the UI. The entire building of an asynchronous web query is performed on the UI thread, and you can't do anything about it.

Edit: This post was formerly named "About the UI Thread performance and HttpWebRequest", but was in fact about Yahoo's Flickr application and was enhanced accordingly.

When programming on Windows Phone 7, you'll hear often that to improve the perceived performance, you'll need to get off of the UI Thread (i.e. the dispatcher) to perform non UI related operations. By good perceived performance, I mean having the UI respond immediately, not stall when some background processing is done.

To acheive this, you'll need to use the common asynchrony techniques like queueing in the ThreadPool, create a new thread, or use the Begin/End pattern.

All of this is very true, and one very good example of bad UI Thread use is the processing of the body of a web request, particularly when using the WebClient where the raised events are in the context of the dispatcher. From a beginner's perspective, not having to care about changing contexts when developing a simple app that updates the UI, provides a particularly good and simple experience.

But that has the annoying effect of degrading the perceived performance of the application, because many parts of the application tend to run on the UI thread.

 

HttpWebRequest to the rescue ?

You'll find that the HttpWebRequest is a better choice in that regard. It uses the Begin/End pattern and the execution of the AsyncCallback is performed in the context of ThreadPool. This performs the execution of the code in that callback in a way that does not impact the perceived performance of the application.

Using the Reactive Extensions, this can be written like this :

var request = WebRequest.Create("http://www.google.com");

var queryBuilder = Observable.FromAsyncPattern(
                                (h, o) => request.BeginGetResponse(h, o),
                                ar => request.EndGetResponse(ar));

queryBuilder()
                /* Perform the expensive work in the context of the AsyncCall back */
                /* from the WebRequest. This will be the ThreadPool. */
                .Select(response => DoSomeExpensiveWork(response))

                /* Go back to the UI Thread to execute the OnNext method on the subscriber */
                .ObserveOnDispatcher()
                .Subscribe(result => DisplayResult(result));

That way, you'll get most of your code to execute out of the UI thread, where that does not impact the perceived performance of the application.

 

Why would it not be to the rescue then ?

Actually, it will always be (as of Windows Phone NoDo), but there's a catch. And that's a big deal, from a performance perspective.

Consider this code :

 public App()
 {
  /* some application initialization code */


  ManualResetEvent ev = new ManualResetEvent(false);

     ThreadPool.QueueUserWorkItem(
  d =>
  {
      var r = WebRequest.Create("http://www.google.com");
      r.BeginGetResponse((r2) => { }, null);

      ev.Set();
  }
     );

     ev.WaitOne();
 }

This code is basically beginning a request on the thread pool, while blocking the UI thread in the App.xaml.cs file. This makes the construction (but not the actual call on the network) of the WebRequest synchronous, and makes the application wait for the request to begin before showing any page to the user.

While this code is definitely not a best practice, there was a code path in the Flickr 1.3 application that was doing something remotely similar, in a more convoluted way. And if you try it for yourself, you'll find that the application hangs in a deadlock during the startup of the application, meaning that our event is never set.

 

What's happening ?

If you dig a bit, you'll find that the stack trace for a thread in the thread pool is the following :

  mscorlib.dll!System.PInvoke.PAL.Threading_Event_Wait() 
  mscorlib.dll!System.Threading.EventWaitHandle.WaitOne() 
  System.Windows.dll!System.Windows.Threading.Dispatcher.FastInvoke(...) 
  System.Windows.dll!System.Net.Browser.AsyncHelper.BeginOnUI(...)
  System.Windows.dll!System.Net.Browser.ClientHttpWebRequest.BeginGetResponse(...) 
  WindowsPhoneApplication2.dll!WindowsPhoneApplication2.App..ctor.AnonymousMethod__0(...)

The BeginGetResponse method is trying to execute something on the UI thread. And in our example, since the UI thread is blocked by the manual reset event, the application hangs in a deadlock between a resource in the dispatcher and our manual reset event.

This is also the case for the EndGetResponse method.

But if you dig even deeper, you'll find in the version of the System.Windows.dll assembly in the WP7 emulator (the one in the SDK is a stub for all public types), that the BeginGetResponse method is doing all the work of actually building the web query on the UI thread !

That is particularly disturbing. I'm still wondering why that network-only code would need to be executed to UI Thread.

 

What's the impact then ?

The impact is fairly simple : The more web requests you make, the less your UI will be responsive, both for processing the beginning and the end of a web request. Each call to the methods BeginGetResponse and EndGetResponse implicitly goes to the UI thread.

In the case of Remote Control applications like mine that are trying to have remote mouse control, all are affected by the same lagging behavior of the mouse. That's partially because the UI thread is particularly busy processing Manipulation events, this explains a lot about the performance issues of the web requests performed at the same time, even by using HttpWebRequest instead of WebClient. This also explains why until the user stops touching the screen, the web requests will be strongly slowed down.

 

The Flickr 1.3 "Black Screen" issue

In the Flickr application for which I've been able to work on, a lot of people were reporting a "black screen" issue, where the application stopped working after a few days.

The application was actually trying to update a resource from the application startup in an asynchronous fashion using the HttpWebRequest. Because of a race condition with an other lock in the application and UI Thread that was waiting in the app's initialization, this resulted in an infinite "Black Screen" that could only be bypassed by reinstalling the application.

Interestingly enough, at this point in the application's initialization, in the App's class constructor, the application is not killed after 10 seconds if it is not showing a page to the user. However, if the application stalls in the constructor of the first page, the application is automatically killed by the OS after something like 10 seconds.

Regarding the use of the UI Thread inside the HttpWebRequest code, applications that are network intensive to get a lot of small web resources like images, this is has a negative impact on the performance. The UI thread is constantly interrupted to process network resources query and responses.

 

Can I do something about it ?

During the analysis of the emulator version of the System.Windows.dll assembly, I noticed that the BeginGetResponse is checking whether the current context is the UI Thread, and does not push the execution on the dispacther.

This means that if you can group the calls to BeginGetResponse calls in the UI thread, you'll spend less time switching between contexts. That's not the panacea, but at the very least you can gain on this side.

 

What about future versions of Windows Phone ?

On the good news side, Scott Gu annouced at the Mix 11 that the manipulation events will be moved out the the UI thread, making the UI "buttery smooth" to take his words. This will a lot of applications benefit from this change.

Anyway, let's wait for Mango, I'm guessing that will this will change is a very positive way, and allow us to have high performance apps on the Windows Phone platform.

[WP7] A nasty concurrency bug in the bundled Reactive Extensions

By Admin at April 21, 2011 20:05 Tags: , , , , ,

The Reactive Extensions have been included in Windows Phone 7, which comes out of the box, and that you can include the Microsoft.Phone.Reactive.dll assembly.

 

This is a good thing, because it participates in the democratization of the Rx framework, and aside from the fact that the namespace is not exactly the same as the desktop version, Microsoft.Phone.Reactive instead of System.Concurrency and System.Linq, there were no major bugs until recently.

A few applications I've worked on that use the Rx framework, a very interesting unhandled exception was popping-up from time to time in my exception tracker. Unlike the double tap issue I found the same way on my Remote Control application, that one was a bit more tricky, see for yourself :

Exception : System.NullReferenceException: NullReferenceException
at Microsoft.Phone.Reactive.CurrentThreadScheduler.Trampoline.Run()
at Microsoft.Phone.Reactive.CurrentThreadScheduler.EnsureTrampoline(Action action)
at Microsoft.Phone.Reactive.AnonymousObservable`1.Subscribe(IObserver`1 observer)
at Microsoft.Phone.Reactive.ObservableExtensions.Subscribe[TSource](IObservable`1 source, Action`1 onNext, Action`1 onError, Action onCompleted)
at Microsoft.Phone.Reactive.ObservableExtensions.Subscribe[TSource](IObservable`1 source, Action`1 onNext)

This exception popped-up out of nowhere, without anything from the caller that would make that "Trampoline" method fail. No actual parameter passed to the Subscribe method was null, I made sure of that by using a precondition for calling Subscribe.

 

Well, it turns out that it's actually a bug that is related to the use of the infamous not-supported-but-silent ThreadStaticAttribute, for which I've had to work around to make umbrella work properly on Windows Phone 7.

The lack of a Thread Local Storage creates a concurrency issue around a priority queue that is kept by the CurrentThreadScheduler to perform delayed operations. The system wide queue was accessed by multiple threads at the same time, creating random NullReferenceExceptions.

This means that any call to the Subscribe method may have failed if an other call to that same method was being made at the same time.

 

In short, do not use the bundled version of Rx in Windows Phone 7 (as of NoDo), but prefer using the latest release from the DevLabs, which does not have this nasty bug.

[Reactive] Being fluent with CompositeDisposable and DisposeWith

By jay at April 04, 2011 13:48 Tags: , , ,

When you're writing a few queries with the reactive extensions, you'll probably end up doing a lot of this kind of code :

moveSubscription = Observable.FromEvent<MouseEventArgs>(this, "MouseMove")
                             .Subscribe(_ => { });

clickSubscription = Observable.FromEvent<MouseEventArgs>(this, "MouseClick")
                              .Subscribe(_ => { }); 

You'll probably call the dispose method on both subscriptions in some dispose method in the class that creates the subscriptions. But this is a bit too manual for my taste.

Rx provides the CompositeDisposable class, which is basically a list of IDisposable instances that all get disposed when CompositeDisposable.Dispose() is called. So we can write it like this :

var cd = new CompositeDisposable();

var moveSubscription = Observable.FromEvent<MouseEventArgs>(this, "MouseMove")
                                 .Subscribe(_ => { });
cd.Add(moveSubscription);

var clickSubscription = Observable.FromEvent<MouseEventArgs>(this, "MouseClick")
                                  .Subscribe(_ => { });
cd.Add(clickSubscription);

That's better in a sense that it is not needed to get my subscriprions out, but this is still too "manual". There are two lines for each subscriptions, and the need for temporary variables.

That's where a simple DisposeWith extension comes in handy :

public static class DisposableExtensions
{
 public static void DisposeWith(this IDisposable observable, CompositeDisposable disposables)
 {
     disposables.Add(observable);
 }
}

Very simple extension, a bit more fluent, and that allows to avoid creating a temporary variable just to unsubscribe on a subscription:

var cd = new CompositeDisposable();

Observable.FromEvent<MouseEventArgs>(this, "MouseMove")
          .Subscribe(_ => { })
          .DisposeWith(cd);

Observable.FromEvent<MouseEventArgs>(this, "MouseClick")
          .Subscribe(_ => { })
          .DisposeWith(cd);

This is a method with a side effect, but that's acceptable in this case, and it always ends the declaration of a observable query.

[WP7Dev][Reactive] Safer Reactive Extensions

By jay at September 06, 2010 20:26 Tags: , , , ,

Cet article est disponible en français.

When developing .NET applications, unhandled exception in threads have the undesirable effect of terminating the current process.

In the following example :

    static void Main(string[] args)
    {
        var t = new Thread(ThreadMethod);
        t.Start();

        Console.ReadLine();
    }

    private static void ThreadMethod()
    {
        Thread.Sleep(1000); throw new Exception();
    }

The basic exception will invariably terminate the process, and to prevent this, the exception needs to be handled properly :

    private static void ThreadMethod()
    {
        try
        {
            Thread.Sleep(1000); throw new Exception();
        }
        catch (Exception e)
        {
            // TODO: Log and report the exception
        }
    }

This makes classes like System.Threading.Thread, System.Threading.Timer or System.Threading.ThreadPool very dangerous to use if one wants to have an always running application. It is then required that no unhandled exception gets out of the custom handlers for these classes.

Even if it is possible to be notified when an exception has been raised and not handled properly, using the AppDomain.UnhandledException event, most the time this leads to the application being terminated. This termination behavior has been introduced in .NET 2.0, to prevent unhandled exception to be silently ignored.

While this is a very appropriate default behavior, in an enterprise environment, I’m usually enforcing custom static analysis or NDepend rules to prevent the use of these classes directly. This forces new code to use wrappers that provide a very wide exception handler and logs and reports the exception, but does not terminate the process. That also implies that there is still a very valid bug to be investigated, because exceptions should not be handled that late.

 

The case of the Reactive Framework

In Silverlight for Windows Phone 7, and in any other .NET 3.5 or .NET 4.0 application that uses the Reactive Extensions, it is very easy to switch between threads.

Reactive operators like Timer, BufferWithTime, ObserveOn or SubscribeOn allow for specific Schedulers like ThreadPool, TaskPool or NewThread to be used, and if a subscriber does not handle exceptions properly, it ends up with a terminated application.

The same exemple here also terminates the application :


    static void Main(string[] args)
    {
     Observable.Timer(TimeSpan.FromSeconds(10), TimeSpan.FromSeconds(10))
                   .Subscribe(_=> ThreadMethod());

            Console.ReadLine();
    }

    private static void ThreadMethod()
    {
            throw new Exception();
    }

The Observable.Timer operator uses the System.Threading.Timer class and that makes it vulnerable to the same termination problems. Every subscriber needs to handle exceptions thrown in the OnNext delegate, or the application will terminate.

Also, do not think that the OnError delegate passed to Observable.Subscribe will handle exceptions thrown during the execution of OnNext code. OnError only notifies of errors generated by previous Reactive operators, not the current.

 

The IScheduler.AsSafe() extension method

Unfortunately, it is not possible for now to override the default schedulers used internally by the Reactive operators. The only way to handle all unhandled exceptions properly is to use the ObserveOn operator and intercept calls to IScheduler.Schedule methods. Calls can then be decorated with appropriate exception handlers to log and report the exception without terminating the process.

So, to be able to generalize this logging and reporting behavior, I created the AsSafe() extension that I place at the very top of a Reactive expression :

    Observable.Timer(TimeSpan.FromSeconds(10), TimeSpan.FromSeconds(10))
              .ObserveOn(Scheduler.ThreadPool.AsSafe())
              .Subscribe(_=> ThreadMethod());


And here is the code of this very simple extension method :


public static class SafeSchedulerExtensions
{
    public static IScheduler AsSafe(this IScheduler scheduler)
    {
        return new SafeScheduler(scheduler);
    }

    private class SafeScheduler : IScheduler
    {
        private IScheduler _source;

        public SafeScheduler(IScheduler scheduler) {
            this._source = scheduler;
        }

        public DateTimeOffset Now { get { return _source.Now; } }

        public IDisposable Schedule(Action action, TimeSpan dueTime)
        {
            return _source.Schedule(Wrap(action), dueTime);
        }

        public IDisposable Schedule(Action action)
        {
            return _source.Schedule(Wrap(action));
        }

        private Action Wrap(Action action)
        {
            return () => {
                try  {
                    action();
                }
                catch (Exception e) {
                    // Log and report the exception.
                }
            };

        }
    }
}

Revisited with the Reactive Extensions: DataBinding and Updates from multiple Threads

By Admin at July 26, 2010 18:42 Tags: , , ,

Cet article est disponible en francais.

Recently, I wrote an article about WinForms, DataBinding and Updates from multiple threads, where I showed how to externalize the execution of event handler code on the UI thread.

I used a technique based on Action<Action> that takes advantage of closures, and the fact that an action will carry its context down to the point where it is executed. All this with no strings attached.

This concept of externalization can be revisited with the Reactive Extensions, and the IScheduler interface.

 

The Original Sample

But let's get right to the original code sample :

    public MyController(Action<Action> synchronousInvoker)
    {
        _synchronousInvoker = synchronousInvoker;
        ...
    }

This code is the constructor of the controller for my form, and the synchronous invoker action will take something like this :

    _controller = new MyController(a => Invoke(a));

And the invoker lambda is used like this :


    _synchronousInvoker(
        () => PropertyChanged(this, new PropertyChangedEventArgs("Status"))
    );

Where Invoke is actually Control.Invoke(), used to execute code on the UI Thread where updates to the UI controls can be safely executed.

While the Action<Action> trick is working just fine in acheiving the isolation of concerns, it is not very obvious just by looking at the constructor signature what you are supposed to pass to it.

 

Using the IScheduler interface

To be able to abstract the location used to execute the content of Reactive operators, the Rx team introduced the concept of Scheduler, with a bunch of default scheduler implementations.

It basically exposes an abstracted way for users of the IScheduler instance to schedule the execution of a method in the specific way defined by the scheduler. In our case, we want to execute our handler code on the WinForms message pump, and "there's a scheduler for that".

The sample can be easily updated to use the IScheduler instead of the Action<Action> delegate, and make use of the IScheduler.Schedule() method.

    public MyController(ISheduler scheduler)
    {
        _scheduler = scheduler;
        ...
    }

And replace the call by :


    _scheduler.Schedule(
        () => PropertyChanged(this, new PropertyChangedEventArgs("Status"))
    );

Not a very complex modification, but it is far more readable.

And we can use the provided scheduler for the Winforms, the yet undocumented System.Concurrency.ControlScheduler which is not in the Scheduler class because it cannot be created statically and requires a Control instance :

    _controller = new MyController(new ControlScheduler(this));

where this is an instance of a control.

This is much better, and for the unit testing of the Controller, you can easily use the System.Concurrency.CurrentThreadScheduler, because you don't need to switch threads in this context.

 

What about the Reactive Extensions and Silverlight for Windows Phone 7 ?

In a very strange move, the WP7 team moved the IScheduler class from System.Concurrency to the very specific Microsoft.Phone.Reactive namespace.

I do not quite understand the change of namespace, and it makes code that used to compile easily on both platforms not compatible.

Maybe they considered the Reactive Extensions implementation for Windows Phone too different from the desktop implementation... But the compact framework was built around that assertion, and most of the common code stays in the same namespaces.

If someone has an explanation for that strange refactoring, I'm listening :)

About me

My name is Jerome Laban, I am a Software Architect, C# MVP and .NET enthustiast from Montréal, QC. You will find my blog on this site, where I'm adding my thoughts on current events, or the things I'm working on, such as the Remote Control for Windows Phone.