Obstacles of unit testing & methods to side step them – Part 2: multi-threading

Introduction

Unit testing is good. Sure it’s no silver bullet but there is no such thing! Looking back I’m not sure how I managed to work without writing unit tests the number of time having tests has saved me from tracking down bugs isn’t worth thinking about :)

However everything isn’t all rosy once you start unit testing against your code, there are a number of items that can put a spanner in the works, and that’s why I have created this series of posts to try and help other people resolve some of these common issues you may find yourself up against.

  • Date & Time
  • Multi-threading
  • External Dependencies
  • Sharing Static items
Prerequisites
  • You have already been writing unit tests
  • You are familiar with state based testing and behaviour based testing & when best to use either
  • Are familiar with a mocking framework (these examples use Rhino Mocks).
Multi-threading

Let’s face it once we introduce multi-threading things usually get more complicated and trying to unit test multi-threaded code is no exception! Why? Well if we think about how our code is going to be executed it will be ran on the test runner thread by NUnit, MBUnit MSTest etc… And if we have code which is going to perform work on a separate thread this causes a problem because the test runner thread will not know that it should really be waiting for work on the other thread to finish before carrying on, let’s get into some code:

   1: [TestFixture]

   2: public class when_adding_two_ints_together

   3: {

   4:     [Test]

   5:     public void should_give_us_the_correct_result()

   6:     {
   8:         var processor = new Processor();

   7:         var sut = new AddTwoIntsWorkItem(1, 2);

   9:         processor.DoWork(sut);

  10:  

  11:         Assert.That(sut.Result, Iz.EqualTo(3));

  12:     }

  13: }

So for the example I created a simple Processor object and as simple command based interface and a concrete object:

   1: public class Processor

   2: {

   3:     public void DoWork(IWorkItem workItem)

   4:     {

   5:         ThreadPool.QueueUserWorkItem((state) =>

   6:             {

   7:                 workItem.Work();

   8:                 Thread.Sleep(2000);

   9:             });

  10:     }

  11: }

  12:  

  13: public interface IWorkItem

  14: {

  15:     void Work();

  16: }

  17:  

  18: public class AddTwoIntsWorkItem : IWorkItem

  19: {

  20:     protected int first;

  21:     protected int second;

  22:     protected int result;        

  23:  

  24:     public AddTwoIntsWorkItem(int first, int second)

  25:     {

  26:         this.first = first;

  27:         this.second = second;

  28:     }

  29:  

  30:     public void Work()

  31:     {

  32:         result = first + second;

  33:     }

  34:  

  35:     public int Result

  36:     {

  37:         get { return result; }

  38:     }

  39: }

You could imagine that this could represent something more real such as a scheduling object that hold a list of IWorkItem’s and can make calls to them in a async manner, so now we have everything setup we can run the unit test:

image

Hmm… were expecting 3 but getting 0 strange, well not really let’s try to breakdown what’s happening:

image

So our assertion Assert.That(sut.Result, Iz.EqualTo(3)); is being called before the work is done hence we get zero.

So how can we tell the test runner thread to wait till the work is done? Well there’s thread.Join() however we A don’t want to expose the thread object outside of the Processor object as this breaks encapsulation and B In this case we use the threadpool so don’t actually have a new thread created.

With thread.Join() out of the question we need to look for an alternative, luckily the Threading namespace gives a good way of handling this scenario via the AutoResetEvent/ManualResetEvent (the difference being that AutoResetEvent will set it’s state to false after being signalled automatically) that’s great we now have a mechanism for telling the thread to wait however how do we know when to signal the event and allow the test runner thread to carry on? at the moment we have no way of knowing when the Processor has processed the IWorkItem.

I don’t think it’s unreasonable to provide someway of knowing when the DoWork is complete, we can provide a WorkItemComplete event that can be subscribed to:

   1: public class Processor

   2: {

   3:     public void DoWork(IWorkItem workItem)

   4:     {

   5:         ThreadPool.QueueUserWorkItem((state) =>

   6:             {

   7:                 workItem.Work();

   8:                 Thread.Sleep(2000);

   9:                 RaiseWorkComplete();

  10:             });

  11:     }

  12:  

  13:     protected void RaiseWorkComplete()

  14:     {

  15:         if (WorkComplete != null)

  16:             WorkComplete(this, EventArgs.Empty);

  17:     }

  18:  

  19:     public EventHandler WorkComplete;

  20: }

I have introduced the event the next step is to amend our unit test to take advantage of this event and to synchronize the test runner thread with the threadpool thread doing the work:

   1: [Test]

   2: public void should_give_us_the_correct_result()

   3: {

   4:     // this is used to synchronize between this current thread &

   5:     // the thread that performs the work in Processor

   6:     var evt = new ManualResetEvent(false); 

   7:     var sut = new AddTwoIntsWorkItem(1, 2);

   8:     var processor = new Processor();

   9:     

  10:     processor.WorkComplete += (sender, e) => evt.Set(); // signal that work is done

  11:     processor.DoWork(sut);

  12:     // tells current thread to wait till work has been done in Processor

  13:     evt.WaitOne(3000); 

  14:  

  15:     Assert.That(sut.Result, Iz.EqualTo(3));

  16: }

It shouldn’t be too hard to see what’s going on here, I create a new ManualResetEvent and set it’s state to unsignalled, I subscribe to the WorkComplete event via a lambda which simply sets the ManualResetEvent to signalled, after the call the DoWork is made we wait for the ManualResetEvent to be signalled. So let’s give the unit test another go:

image 

Great! that worked a treat one thing to bear in mind is when waiting for the event to be signalled is what timeout to use, in the case above I used 3000 milliseconds you typically want to use something around this figure and should be used as a failover in case the event/method that your expecting to be called isn’t and you don’t get to signal the event, this will prevent the test runner thread from getting stuck.

You should now be more comfortable when having to unit test code that performs work in a separate thread, if you don’t want to expose an event to know when a particular task is finished you could use subclass and override to only provide the event on the test subclass object.

Registering a mock/stub with Windsor

I have some code that uses Castle Windsor to retrieve instances this helps to manage dependencies that I don’t have to worry about, the side effect of this is that when performing unit tests against the code that needs to get an instance from Windsor I want to provide a way to inject a mock/stub object is can be accomplished with the following:

container.Register(Component.For()
                                        .Instance(mockFoo));

To help reduce the amount of code in my test setups I usually have this method in my Unit Testing base class

protected void RegisterDependencyAgainst(TInterface instance)
{
    container.Register(Component.For()
                                .Instance(instance));
}

Which is then called like this:

// in setup method
mockFoo = MockRepository.CreateMock();
RegisterDependencyAgainst(mockFoo);

Enforcing Conventions

In my last post I demonstrated adding AOP to cut down on cross cutting code, and at the end mentioned that it would be nice to enforce a convention throughout the system, the example being each public method in the task layer being decorated with a certain attribute.

I was unsure about how to do this until recently seeing a post by Ayende http://www.ayende.com/Blog/archive/2008/05/05/Actively-enforce-your-conventions.aspx in it he references an article by Glenn Block whereby both of them came up with a unit test called PrismShouldNotReferenceUnity in the test they use reflection to check that there are indeed no references from the Prism assembly to the Unity assembly.

This is a great idea! You now have a repeatable test that can be run to make sure the conventions for your system are met, so armed with this technique I created the following test:

[Test]
public void task_class_methods_should_be_marked_with_wrap_exception_attributes()
{
    try
    {
        Assembly asm = Assembly.LoadFrom( "MCromwell.StaffIntranet.Task.dll" );
        var wrapExceptionType = asm.GetType( "MCromwell.StaffIntranet.Task.Infrastructure.WrapExceptionWithAttribute" );
        Assert.IsNotNull(wrapExceptionType);

        foreach (Type current in asm.GetTypes())
        {
            if (current.FullName.StartsWith( "MCromwell.StaffIntranet.Task.Tasks." ) && (!current.IsInterface) && (!current.IsAbstract))
            {
                foreach (var method in current.GetMethods())
                {
                    if ((method.IsPublic) && (method.DeclaringType.Name != "Object"))
                    {
                        if (method.GetCustomAttributes(wrapExceptionType, false).Length <= 0)
                            Assert.Fail("no wrap exception attribute found on type '{0}', method '{1}'", current.FullName,method.Name);
                    }
                }
             }
        }
    }
    catch (ReflectionTypeLoadException rex)
    {
        foreach (var current in rex.LoaderExceptions)
            Console.WriteLine(current.ToString());
        throw;
    }
    catch (Exception ex)
    {
        Console.Error.Write(ex.Message);
        throw;
    }
}

In here you can see by leveraging reflection I can browse all the public methods for my task layer classes and make sure they do indeed have a WrapExceptionWithAttribute the other cool thing is that by doing it this way I can freely add new classes and they will need to comply with the conventions set out or the testing will fail, cool eh!

One thing to point out is that if you start increasing the number of conventions and want a better way to control and report on, you probably want to look into something like FXCop or NDepend’s CQL.

Staff Intranet on Codeplex

I have not posted here for a little the main reason being I have been putting the final touches to another project I have been working on in my spare time, it’s a staff intranet application what I have tried to do with this project as with the previous projects is try and use it as a learning exercise, so I have incorporated some best practices and design patterns and used them for a real world example, in there you will find:

  • Inversion of control – This has been decoupled from a particular IoC container but I have utilised Castle Windsor under the hood
  • TDD/BDD – Around 98% of the behaviour of the application was designed test first, the reason I have put BDD as well is because I like to think of my tests as defining the behaviour of the application rather than of just testing assertions in my code, this comes out through the naming of my test cases
  • NHibernate – This was my first use of NHibernate and overall I was very pleased with what this very powerful ORM gives you and also keeping the domain clean from database artifacts (persistence ignorance)
  • Model View Presenter – This is the first time I have emplyed model view presenter for web and it acts as a nice interim between moving logic from the codebehind and going the whole hog and using an MVC framework such as monorail or ASP.NET MVC, I’m completely sold on having tests against the UI logic
  • SQL Server 2005 – This was probably the most dissapointing aspect of developing this application, I was hoping that the new client tools would be great and easy to get stuff done with however I found it the opposite things that used to be easy using the SQL server 2000 enterprise manager were not intuituve at all with the Management Studio, examples:
    • Wanting to remove a database that already exists
    • Setting permissions from the users perspective

Anyway after that intro you can find more details over at my project homepage on codeplex.

My hope is that this project will help others who are learning about the above but would like to see them used in a real world context rather than just in hello worls context.

First codeproject article posted

I just finished posting my first article to codeproject (only taken 3 years since I joined!), it covers unit testing against a database and also provides a handy little testing library that can be used to make it a little easier you can check it out at http://www.codeproject.com/KB/cs/unittestingdblib.aspx

Unit testing threading using rhino mocks

Update: I have recently put together a much more detailed look at this in my obstacles of unit testing series https://journalofasoftwaredev.wordpress.com/2009/05/02/obstacles-of-unit-testing-methods-to-side-step-them-part-2-multi-threading/

I recently had to do some unit testing against my TaskScheduler class this presented a problem… the TaskScheduler performs it’s monitoring by spawning a worker thread in the background so I needed a way to test this asynchronous behaviour. Thanks to the excellent Rhino Mocks and using the ManualResetEvent I come up with the following solution:

[Test]
public void Should_run_scheduled_task_when_datetime_met()
{
	ManualResetEvent evt = new ManualResetEvent(false);
	DateTime runAt = new DateTime(2008, 1, 1, 12, 0, 0);
	ITask mockTask = mocks.DynamicMock();
	ITaskRunner mockTaskRunner = mocks.DynamicMock();

	using (mocks.Record())
	{
		SetupResult.For(mockTask.RunAt()).Return(runAt);

		mockTaskRunner.Run(mockTask);
		LastCall.Constraints(Is.Equal(mockTask));

		LastCall.On(mockTaskRunner).Do(new TaskRunnerRunDelegate(delegate(ITask task)
		{
			evt.Set();
		}));
	}

	using (mocks.Playback())
	{
		ITaskScheduler sut = createSUT(new MockDateTimeSource(runAt.AddSeconds(-2)), mockTaskRunner);
		sut.Schedule(mockTask);
		sut.Start();
   	        evt.WaitOne(4000, false);
	}
}

When a Task that the TaskScheduler is monitoring reaches its scheduled execution date/time the TaskScheduler Hands off to an ITaskRunner in this case it’s a mocked out interface, we can use this to listen out for a call to the Run method, and set the ManualResetEvent thus exiting the WaitOne block. The downside to this approach is establishing what the timeout should be in most cases this will be a few seconds.

Unit testing date & time related objects

I was recently working on a scheduler object that relied on retrieving the current date and time to see if any tasks it had were due for execution, this presented a problem, how do you unit test an object that relies on the current date and time?

We could just make no changes to the object and instead make the unit tests highly dependant on the date and time it is ran on and in some cases it might be ok (although highly impractical and dangerous) however for a scheduler this is not feasible.

What we need can be solved by “introducing another level indirection” paraphrased from David Wheeler’s famous quote. What we can then do is setup an interface that can bring us back a DateTime:

public interface IDateTimeSource
{
 DateTime CurrentDateTime();
}

Now we can implement a real-time version that will be used most of the time:

internal class RealTimeDateTimeSource : IDateTimeSource
{
 public DateTime CurrentDateTime()
 {
 	return DateTime.Now;
 }
}

And also a mock version that in this case we can specify when it should start and it just ticks along representing time:

internal class MockDateTimeSource : IDateTimeSource
{
 DateTime providedDateTime;
 Thread realTimeThread;
 readonly object syncLock = new object();

	private DateTime ProvidedDateTime
 {
 	get
 	{
 		lock (syncLock)
 		{
 			return providedDateTime;
 		}
 	}
 	set
 	{
 		lock (syncLock)
 		{
 			providedDateTime = value;
 		}
 	}
 }

	public MockDateTimeSource(DateTime startAt)
 {
 	providedDateTime = startAt;

		realTimeThread = new Thread(delegate()
 	{
 		while (true)
 		{
 			ProvidedDateTime = ProvidedDateTime.AddSeconds(1);
 			Console.WriteLine(providedDateTime.ToString("hh:mm:ss"));
 			Thread.Sleep(1000);
 		}
 	}
 	);
 	realTimeThread.IsBackground = true;
 	realTimeThread.Start();
 }

	public DateTime CurrentDateTime()
 {
 	return ProvidedDateTime;
 }

}

The locking is to ensure that the write in the background thread and the read in the calling thread take place see http://www.yoda.arachsys.com/csharp/threads/volatility.shtml for more info.Now we need to provide the object with the version to use as it’s date and time source, we can do this numerous ways here is an example using constructor injection:

public Scheduler() : this(new RealTimeDateTimeSource())
{
}

public Scheduler(IDateTimeSource dateTimeSource)
{
 this.dateTimeSource = dateTimeSource;
 Initialize();
}

The parameter-less constructor will be used in most cases however for unit testing we can provide the mock version so that our tests can be decoupled from the date and time they are ran.If you wanted to provide extra security so that when the code goes into production users of the scheduler object can never pass in an IDateTimeSource you could change the constructors and also add a conditional preprocessor directive so that the release build never includes the testing constructor.