C# 3.0, a sneak peek

By Jerome at July 22, 2006 10:22 Tags: ,

If you've used both DataSet and DataTable, you must have seen the DataTable.Select method. This is an interesting method that allows to select rows using a set of criterias, like IS NULL and comparison operators referencing columns of the current table, as well as columns from other tables using relations. The problem with method is that is returns a DataRow[], on which you cannot perform an other select.

The solution is actually quite simple : Just copy the rows you'll answer me. Yes, but you can't just reference rows in two DataTable instances, so you also have to perform a deep copy of the rows. So, with a little digging in the DataTable methods, here is what you get :

public static DataTable Select(DataTable table, string filter, string sort)
{
   DataRow[] rows = table.Select(filter, sort);
   DataTable outputTable = table.Clone();

   outputTable.BeginLoadData();

   foreach(DataRow row in rows)
      outputTable.LoadDataRow(row.ItemArray, true);

   outputTable.EndLoadData();
   return outputTable;
}

Clone is used to copy the table schema only, BeginLoadData/EndLoadData to disable any event processing during the load operation, and LoadDataRow to effectively load each row. This seems to be a fairly fast way to copy a table's data.

Now, I wondered how they would do this in C# 3.0, since there is a lot of data manipulation with the new LINQ syntax. This version is quite interesting because instead of evolving the runtime, they chose to upgrade only the language by adding features that generate a lot of code under the hood. That was the case in C# 2.0 with iterators and anonymous methods. C# 1.0 also had this with foreach, using or lock for instance.

In the particular case of Linq, C# 3.0 generates a method invocation list  of a LINQ query, producing standard C# 3.0 code with the help of lambda expressions. For example, these two lines are equivalent :

   var query = from a in test where a > 2 select a;
   var query2 = Sequence.Where(test, a => a > 2);

This ties a little more the compiler to the system asssemblies, but this does not matter anymore.

By the way, you can apply queries to standard arrays and join them :

static void Main(string[] args)
{
   var names = new[] {
      new { Id=0, name="test" },
      new { Id=1, name="test1" },
      new { Id=2, name="test2" },
      new { Id=4, name="test2" },
   };

   var addresses = new[] {
      new { Id=0, address="address" },
      new { Id=1, address="address1" },
      new { Id=2, address="address2" },
      new { Id=3, address="address2" },
   };

   var query = from name in names
      join address in addresses on name.Id equals address.Id
      orderby name.name
      select new {name = name.name, address = address.address};

   foreach(var value in query)
      Console.WriteLine(value);
}

I've joined the two arrays using the Id field, and creating a new type that extracts both name and address. I really like inline querying because you can query anything that implements IEnumerable.

I'm also wondering how it'll fit into eSQL (Entity SQL)...

But back to the original subject of this post. They had to do some kind of a DataTable copy in the C# 3.0 helper library, which uses extension methods :

   DataTableExtensions.ToDataTable<T>(IEnumerable<T>)

And with some further digging, I found that the LoadDataRow method for copying data is the fastest way to go.

I also found out using the great reflector that there is an Expression compiler in System.Expressions.Expression<T>. Maybe they finally did expose an expression parser that we can use... I'll try this one too !

BTRemote Control and Windows XP 64 Bits

By Jerome at May 26, 2006 23:21 Tags:

Quite a few people have now been using the latest beta for a while, and I get most of the time feature requests, sometimes non critical bugs like the Remote Client not connecting if the host PC is not discoverable, and not so often blocking bugs.

The one Krijn Wijnands (thanks !) has uncovered might involve Windows XP 64 Bits. The remote client does not connect to the server, and he's using an hardware configuration that's proved to work fine.

I do not own a 64 bits machine and if any of readers of this blog or user of this utility uses Windows XP 64 bits, please leave me a message. I'll be glad to hear from you.

The client version of this software do log much out of the debug build for now, but I'll add some more for extensive diagnostics.

From a pure developer point of view, .NET 2.0 64 Bits should not change a thing, but I suspect a P/Invoke issue that I've not covered. That should give me a reason to upgrade my server to 64 Bits ;) For the sake of the testing, of course.

For the next release, I'll be adding generic support for applications, configurable using a static xml file (there's no point on modifying this at runtime)

If you have some other wishes that come to mind, feel free to speak !

Precision Timer in .NET 2.0

By Jerome at November 18, 2005 16:25 Tags: ,

If you've been using the .NET Framework since the beginning, you must have had to do some early code profiling, or framerate computation for realtime graphics.

The first idea that pops up to acheive this is to use the DateTime.Now property and do some subtraction of two instances. This is not a good idea since the resolution of this timer is around 10ms or so, which is clearly not enough (and your framerate counter may not go higher than 100 FPS or worse may not work at all).

If you've been in the business for long enough, and been doing some plain old "native" code in, say, C++ on Win32, you should probably used the couple QueryPerformanceFrequency/QueryPerformanceCounter to get the job done. And the same goes for .NET 1.0/1.1. Well, I don't know for you, but each time I have a project that reaches a certain critical size, I always need to use this kind of timer and I end up by writing the P/Invoke wrapper to reach these two methods.

Good news is, .NET 2.0 already has this class integrated in the form of System.Diagnostics.Stopwatch, so you don't have to write it from scratch again and again because you can't find on the net the right "free" class that does enough for you.

The BCL team has added some other nice utility classes like this one, and this saves quite some time.

Reflective Visitor using C#

By Jerome at April 04, 2005 11:35 Tags: ,

There is a well known design pattern in the Object Oriented world: The Visitor pattern.

This pattern, among other things, allows to extend an object without actually modifying it. It is fairly easy to implement in any good OO language such as C++, Java or C#.

However, there is a problem with the implementation of this pattern, or rather an implementation limitation. It requires the base interface or abstract class for all visitors to define at most one method for each type that may visit it. This is not a problem by itself, but it requires to modify the visitor base each time you add a new type. The best way to do this would be to call the appropriate method based on the caller type.

.NET provides that kind of behavior through reflection, as it is possible to find a method based on its parameters at runtime.

I decided to try this out with the C# 2.0 and its generics :)

Here what I came up with :

 

public interface IOperand
{
 
IOperand Accept(Visitor visitor, IOperand
right);
}

public class Operand< T> : IOperand
{
  private
T _value;

  public
Operand(T value)
 
{
   
_value = value;
 
}
  public IOperand Accept(Visitor v, IOperand
right)
 
{
    return v.Visit(this
, right);
  }
  public T Value
  {
    get { return
_value; }
 
}
  public override string ToString()
  {
    return string.Format("{0} ({1})", _value, GetType());
  }
}

This is the definition of an operand, which is used in a abstract machine to perform operations on abstract types. This class is generic, I did not want to implement all the possible types.

Then here is the visitor :

 

public class Visitor
{
 
public virtual IOperand Visit(IOperand left, IOperand right)
 
{
   
MethodInfo info = GetType().GetMethod("Visit", new Type[] { left.GetType(), right.GetType() });
   
   
if (info != null && info.DeclaringType != typeof(Visitor))
     
return info.Invoke(this, new object[] { left, right }) as IOperand;

    
Console.WriteLine("Operation not supported");
   
return null;
  
}
}

This method search in the current type all methods named "Visit" that take the actual type of the parameters left and right and tries to match a method with it. Also, to avoid looping through the same method we're in since it's matching everything, there is a test for the type declaring the method.

Now the AdditionVisitor :

 

public class AdditionVisitor : Visitor
{
  public IOperand Visit(Operand<int> value, Operand<int> right)
  {
    return new Operand<int>(value.Value + right.Value);
  }
  public IOperand Visit(Operand<int> value, Operand<short> right)
  {
    return new Operand<int>(value.Value + right.Value);
  }
  public IOperand Visit(Operand<double> value, Operand<int> right)
  {
    return new Operand<double>(value.Value + right.Value);
  }
}

Which defines a bunch of visitable methods used to add different operations on IOperand-like types.

And finally to use it :

 

class Program
{
  static void Main(string[] args)
  {
    Operand<int> a = new Operand<int>(21);
    Operand<short> b = new Operand<short>(21);
    Console.WriteLine(Add(a, b));
  }

 
static IOperand Add(IOperand a, IOperand b)
 
{
   
AdditionVisitor addVisitor = new AdditionVisitor();
   
return a.Accept(addVisitor, b);
 
}
}

Using this Reflective Visitor, modifying the base visitor class is not needed anymore, which limits the modifications to one class only. Of course, there's room for optimization, for instance by avoiding the method lookup using the System.Reflection namespace, but you get the picture.

Some asked me what could be done in .NET that could not be done in C++, this is an example of it :)

C# 2.0, Closures and Anonymous Delegates

By Jerome at April 04, 2005 11:33 Tags: ,

I was looking around the web about new features in C# 2.0, and I came across this article about the support for closures in C# 2.0. The article explains that the support for closures in C# 2.0 takes the form of anonymous delegates.

There are some examples of closures like this one :

public List<Employee> Managers(List<Employee> emps)
{
  return emps.FindAll(
    delegate(Employee e)
    {
      return e.IsManager;
    }
  );
}

Which is interesting, but less than this one :

public List<Employee> HighPaid(List<Employee> emps)
{
  int threshold = 150
;
  return emps.FindAll(
    delegate(Employee
e)
    {
      return
e.Salary > threshold;
    }
 
);
}

The interesting part here is that the delegate is actually allowed to use a variable that is local to the method where it is defined. You might wonder how this is implemented by the C# compiler.
It may become even less obvious with this example :

public Predicate<Employee> PaidMore(int amount)
{
  return delegate(Employee e)
  {
    return e.Salary > amount;
  };
}

Ok, where does the compiler stores the value of "amount" since the delegate method is only returned to be executed later... ?

In fact, the compiler only generates a "DisplayClass" that containts amount as a field initialized when the anonymous delegate is created, and the implementation of the delegate itself.

Easy.

BartPE using PXE, Again...

By Jerome at February 22, 2005 11:31 Tags:

If you are in the Windows world, you might know that there is an upcoming Service Pack for Windows 2003 Server, which has reached the RC2 stage quite recently. This Service Pack brings a lot of improvements in the area of security, management and scaleability.

However, there are a few other interesting new features that have been added : The ability to boot an ISO image from the network.

The interesting thing about all this is that you don't have to install a RIS server anymore, only a simple TFTP server is needed (even under linux ;)). The procedure is quite simple and can be found here. The interesting part of this procedure is the content of the winnt.sif file :

[SetupData]
BootDevice = "ramdisk(0)"
BootPath = "\i386\System32\"
OsLoadOptions = "/noguiboot /fastdetect /minint /rdexportascd /rdpath=<image>.iso"

I don't know where the guy got his informations from, but this solves a lot of my problems.

Now, we're going to be able to allow a multi network-boot, Win/NetBSD/Linux from any workstation on the network. All I have to do now is populate the iso image with the necessary software and we're good to go !

I know some people that are going to be happy :)

Mono 1.0.5 support for NetBSD 2.0

By Jerome at December 15, 2004 11:37 Tags: ,

As I promised earlier, here is the patch for Mono 1.0.5 to run on NetBSD 2.0-Release.

I've managed to get MonoDoc 1.0.5 to run on my box and near being able to run MonoDevelop too. It seems that somewhere, a mutex is unlocked twice and since the libpthread is asserting on that kind of invalid behavior, MonoDevelop stops. But even if assertions are disabled, MonoDevelop stops while not being able to read a perfectly valid file... Well :) There's some work to be done here.

Earlier I talked about the fact that I forgot to save the stack's address of suspended threads. Under Linux, where signals are used to "suspend" threads, the signal handler looks like this :

   void GC_suspend_handler(int sig)
   {
      int dummy;

      /* some not important stuff... */

      pthread_t my_thread = pthread_self();

      me -> stop_info.stack_ptr = (ptr_t)(&dummy);  /* Get the top of the stack address */

      sigsuspend(&suspend_handler_mask); /* Wait for signal to resume */
   }

The thread being stopped is held into its signal handler, waiting for the signal to resume and exit the signal handler. The main reason for this "trick" is that there is no standard way to suspend a thread with the libpthread.

However, NetBSD's libpthread has two nice methods pthread_suspend_np and pthread_resume_np, which are able to suspend and resume arbitrarily a specific thread. So in the GC code, instead of raising signals to specific threads, calling these two methods is only necessary.

The thing I missed in the first patch, is the storage of the top stack's address, retreived by using the dummy variable in the signal handler. So, to work around this problem, I used a third non standard function that is able to retreive the base adress, not the top address, of the stack for the suspended thread and give it to the GC.

Although, the patch seems to please mono, which is running fine, the stack address given to the GC is not the perfect one. Unfortunately, there is no way, as far as I know, to get the top of stack pointer for suspended threads. I can't tell for now the impact of this modification, but still, mono does not seem to complain, neither does the GC.

I also tried to mix signals and thread suspension, but it seems that signals in general are breaking GC's operations.

Maybe some pthread/GC guru could enlighten me :)

Update : It also works nicely with Mono 1.1.3. Actually, it runs better as it has a little bug that has been fixed in System.Timers.Timer, but this has nothing to do with NetBSD ;)

ASP.NET Remote Debugging, Windows XP SP2 and .NET Framework 2.0

By Jerome at December 13, 2004 11:38 Tags: ,

I did not have to create and debug any ASP.NET application for a long time, but since I'm creating an online Questions/Answers application, I had to use the really nice debugging features brought by Visual Studio .NET.

To be specific, I did not have to debug any remote web site since I installed the Windows XP SP2. My configuration is quite simple, I host my application on a development Windows 2003 Server and I design with VS.NET on my Windows XP machine.

So when I trying to debug my web application, all I could get was : "Unable to debug the application" or "The remote debugger could not connect to the local machine" or a really helpfull "Cannot debug process".

The first reaction when seeing things like this is to check the local and remote VS Developers and Debugger Users security groups. But every thing was fine there... In fact, the problem lies in the DCOM security configuration. Installing the SP2 removed the right for the anonymous account to use DCOM remotely... but not for the Everyone account. Odd.
The only thing to do is to get there : Run / dcomcnfg.exe / Component Services / Computers / My Computer / Properties / COM Security / Edit Limits, and to check "Remote Access / Allow". Easy.

This solved my first problem, the remote debugging. The second one is still ASP.NET debugging, but locally this time.

I have both VS.NET 2003 and 2005 installed on my local machine, and so are both 1.1 and 2.0 .NET Framework versions. Installing 2.0 over 1.1 changes the default framework used by the Windows XP IIS to version 2.0 which breaks the 1.1 debugger :)
The simple thing to do here is this : IIS / Web Sites / [WebSite] / Properties / ASP.NET, then select 1.1.4322 for the ASP.NET Version field and that's it :)

Man, I really like being able to debug my Web Applications, I really missed it ! (And so was Karouman actually, who was debugging blindly by guessing exceptions :p)

Inter-Domain Trust Relationship and lmhosts Text Casing

By Jerome at October 13, 2004 11:40 Tags: ,

Among the things that are interesting in latest versions of Samba, there are NT4 Inter-Domain Trust Relationships. In the Windows world, the original NT4 domain model was not scalable enough when NT4 was omnipresent. Companies fusions did not mix well with the single windows domain to merge all user accounts and everything else.

One solution brought back then was the ability to have an NT4 domain to trust users authenticated in an other NT4 domain. This comes in multiple flavors known as incoming, outgoing, bidirectional trust to allow users to be authenticated in both or only selected domains. Other things like transitive and non-transitive trusts can also be used in at least a 3 domains interaction to allow users from domain, say, A to be authenticated in domain C through B, if the trust between A and B is transitive.

This facility is now fully integrated in Windows 2000/2003 domains and this allows to trust NT4 style domains, which includes Samba managed domains.

Here at epitech we a growing need to interconnect a number of small domains to allow users to connect to a variety of services and to allow an unified password management, a samba domain is used to map users between the Windows and the Unix world.

When establishing the trust between two domains , there are at least two possible scenarios :

  • The domain can be identified using a Netbios broadcast or WINS resolution, which implies that the domain is known and has registerd itself to the wins or is located on the same physical IPv4 subnet,
  • Or, the domain cannot be identified using a single broadcast and must be identified via a manual addition of the domain in the WINS or the domain and the associated domain controller have been added to the lmhosts file.

During a lot of trying, I have found that a combination of both the WINS and lmhosts modification are needed to establish the trust. If you don't do both, the domain controller that wants to establish the trust tries to partially resolve the remote DC by using a really weird NETLOGON/UDP packet that is of course rejected (and not even logged) by Samba. The rest is done by an attempt to locate the remote DC by a local subnet broadcast, which of course fails.

You might say : "Well, modifying the lmhosts file should be a piece of cake !". Actually yes, writing it is. So here is what you might want to add in your lmhosts file :

     10.0.0.1    mysamba-dc             #PRE   #DOM:midhearth
     10.0.0.1   "midearth       \0x1b"  #PRE

Which seems to be good. By the way, the \x1b is used to identify the midearth entry as a domain and should be placed at the 16 byte index. Historical laziness... What a shame.  Anyway, this does not work and produces the strange behavior I described earlier. Here is a version of the same block of text that really works :

     10.0.0.1    MYSAMBA-DC             #PRE   #DOM:MIDEARTH
     10.0.0.1   "MIDEARTH       \0x1b"  #PRE

Noticing anything ? Yes ! All names are uppercased... And no error or warning message notifies you about this when casing is not correct. After that, everything works fine and the trust can be establised. I kind of hate losing time with that kind of tricks, but hey, it works now :)

Multiple MassStorage Drivers with Windows 2000/XP/2003 and INACCESSIBLE_BOOT_DEVICE

By Jerome at October 08, 2004 11:42 Tags:

One annoying thing about Windows is the Mass Storage drivers management. The Windows Setup only installs the necessary drivers for the current system, which is generally fine most of the time. As long as you change your hardware but not the Mass Storage chipset, there is no problem. Windows is just restarting its Plug and Play stage to re-detect all the new devices and peripherals and this works really fine.

Here at Epitech, computers were Via based for two years in a row and changing from one to the other was not a problem. This year's new computers are now Intel based. Nice computers, really.

But one problem : Via based Windows installation don't boot anymore. There is a nice 0x7B stop mode (a bsod) which means INACCESSIBLE_BOOT_DEVICE. Windows was unable to find any suitable boot device, because it does not have the appropriate drivers for the current hardware.

Microsoft has a KBase article (KB314082) about this particular issue, which states that you can force a Windows installation to try every known MassStorage driver during the startup. Since the procedure implies the copying of some Intel drivers, I assumed a while ago that it would only work in the Via to Intel direction. Well, apparently not. It also works in the Intel to Via direction, which is really nice :) Actually, it works for any to any chipset, as long as the hardware is natively known by Windows.

This solves a lot of problems for many people here that do really want to reinstall their Windows.

About me

My name is Jerome Laban, I am a Software Architect, C# MVP and .NET enthustiast from Montréal, QC. You will find my blog on this site, where I'm adding my thoughts on current events, or the things I'm working on, such as the Remote Control for Windows Phone.