Can’t we do better than null?

So you want to retrieve an aggregate from a repository (assuming DDD) stored in some data store so you make a call not unlike the following:

Product product = productRepository.FindById(id);

You then go to execute some method on the Product aggregate and boom the dreaded NRE (Null Reference Exception) great, now we can just go to tried and tested !== null check however I want to explore different ways we can actually deal with values that may or may not be present and make this more explicit and give the consumer a better experience.

Is it an exceptional circumstance?

I guess the first question we could ask is would it be considered exceptional that we cannot find the aggregate we are looking for, in the example above we are using the id to find the aggregate which is about as specific as we can get and in most cases the id would have been assigned against a selectable item so not being able to retrieve the aggregate but likely be:

  1. Infrastructure issue in which case you would get an exception thrown at the infrastructure level
  2. The id has been tampered with to make it invalid or non accessible
  3. The aggregate has been removed

Now a case could be made that these are pretty exceptional conditions, however if we change is so that we are trying to retrieve an aggregate based off a natural id for instance a SKU then it becomes a non exceptional circumstance as it could just be that we have not assigned to the SKU to the product or has been done incorrectly.

I also find that your in the same boat with regard to the consumers point of view as the null return but in this case you will get a custom exception thrown at you which is slightly better (checked exceptions in java would make it explicit it could throw).

Return result with output

The next way we could make it more explicit to the caller is by changing the signature to this:

bool FindById(ProductId id, out Product product);

This goes back to the C days if you check out some of the WinAPI you would typically call a function that would return the result of the call and assign a value to a pointer passed in, thankfully the C# is a bit nicer but from a consumer point of view it is explicit but generally not a nice API to expose:

Product product = null;
bool found = productRepository.FindById(id, out product);

Personally I have never liked using out parameters I think mainly due to the declaring of the output parameter(s) before making the call.

Null Object

Another option is to have the repository return a Null Object if it cannot find the aggregate, I don’t think this option would be suitable for a repository as aggregates tend to be treated in much higher regard than optional objects however this strategy can work well in other scenarios where you could provide a default behaving object.

Why returning collections causes no problem

If we look at another method on our Product repository:

IList<Product> FindByCategory(ProductCategoryId productCategoryId);

When we use this method and it doesn’t find any Products for the category id supplied we don’t have the same problem as the FindById instead we just have an empty list (there’s a special place in hell reserved for people who return null out of these type of methods) so our consumer does not need to perform anything differently:

var foundProducts = productRepository.FindByCategory(productCategoryId);
foreach (var product in foundProducts)
{
    // do stuff here
}

The key difference between this signature and the FindById is that the actual products are now housed inside a container (in this case an implementation of IList<Product>) in order to get to the Product we have to go through the container.

This is one area where functional languages have the edge as they don’t have the concept of a null instead you have to be explicit by using a Maybe/Option and the consumer is forced to handle what to do in both cases (this is made easier by pattern matching).

What if we took a similar approach for returning our single Product.

Using a Tuple

The simplest way we could achieve this is by using a Tuple:

Tuple<bool, Product> FindById(ProductId id);

Now we are returned a result that gives us a container in that provides us with more context we can now use this as a consumer like this:

var result = productRepository.FindById(id);
if (result.Item1)
{
    // product was found
}
else
{
    // product not found
}

Now the consumer has a bit more to go on with regard to how the repository behaves and could probably assume that the first item corresponds to whether a Product has been found, as mentioned in the last section functional languages have pattern matching which makes this a bit more explicit, here is an example of what C# could look like if it had pattern matching:

var result = productRepository.FindById(id);
switch (result)
{
    case (false, _):
        // product not found
    case (true, product):
        // product found
}

Query Result

The Tuple approach is a step in the right direction however it’s not very expressive what if  we were to create our own object that represents the results of a query ideally as a consumer we could then use it like this:

QueryResult<Product> result = productRepository.FindById(id);
if (result.HasResult)
{
    var product = result.Result;
    // product found
}
else
{
    // product not found
}

// or if your feeling functional
result.Found(product => // product found);
result.Missing(() => // product not found);

I have shown 2 different API’s depending on the consumers preference, we have taken the approach of returning a container so the consumer must go through this in order to get to the Product, this makes it really explicit on how you deal with the result and no surprises.

I have pushed up a working version of this object to https://github.com/stavinski/joasd-queryresult so feel free to take it and do what you want with it.

Advertisements

The importance of idempotent services

Introduction

We all know the importance of performing a unit of work (UoW) in a transaction scope so that if anything goes wrong we can rollback the actions that have taken place and be sure that the state is exactly how it was before the UoW took place, this is were we can use a messaging framework (i.e. MSMQ) that can support transactional capabilities using a transaction coordinator (i.e. MSDTC) so that when we receive a message perform some work, but when saving to the DB there is an issue (i.e. Network problem, transaction deadlock victim) we know that the message received will be put back on the queue and any messages that have been attempted to be sent will be returned (to prevent duplicates):

public void DoWork()
{
    using (var scope = new TransactionScope())
    {
        var command = MsmqInboundWork.NextMessage();
        // *** processing here ***
        DbGateway.Save(workItem);
        MsmqOutboundResult.SendMessage(new ResultMessage { CompletedAt = DateTime.UtcNow; });

        scope.Complete();
    }
} 

If this happens we have options (i.e. retry for a limited amount of goes and then put the message into an error queue) however we don’t always have the option of using a reliable/durable/transactional transport layer such as messaging, for instance if we are integrating with a 3rd party over the internet they are going to provide usually a HTTP based transport layer such as SOAP/REST or like in my case recently integrating with an internal legacy system that only provides a HTTP transport layer (SOAP webservices was the only option in my case).

This post will demonstrate how we can work with services exposed on unreliable transport layer to perform actions but still not end up with duplicates by making the operations idempotent.

What is an idempotent operation?

This basically boils down to being able to call a function with the same arguments and have the same result occur, so if we have a method:

public int Add(int first, int second)
{
    return first + second;
} 

This method is idempotent because if I called it multiple times with Add(2,2) it will always return 4 and there are no side effects, compare this to the next method (demo only!):

public void DebitAccount(string accountNumber, decimal amount)
{
    var account = FindAccount(accountNumber);
    account.Balance -= amount;
    // account saved away
} 

If I call this multiple times like this DebitAccount("12345678", 100.00m) the client of that account is going to be more out of pocket each time a call is made.

How does this relate to my webservice method?

You may be thinking to yourself we have a system running that exposes a webservice with a method to perform a similar action to the example above or that you make a call to a 3rd party to perform an action (i.e. send an SMS) and so far we haven’t had any problems so what’s the big deal!? You’ve been lucky so far but remember the first fallacy of distributed computing

The network is reliable

Once you get network issues this is where everything goes wrong, this may not be a big deal you may decide to retry the call and the user ends up with another SMS sent to their mobile however if you’re performing a journal on an account this is a big problem.

If we look at a typical call to a webservice, in this case we are going to use the example of a façade service of a legacy banking system:

Account Service
fig 1 – If all goes to plan

In this case everything has gone ok and the client has received a response back from the account service so we know that the account has been debited correctly, what if we have the following:

Account Service
fig 2 – Network issue causes loss of response to client

Now we have a problem because the client has no way of knowing whether the debit was performed or not, in the example above the journal has taken place but the machine running the account service could have a problem and not get round to telling the banking system to perform a journal.

Fixing the problem

We established in the last section that the client has no way of knowing whether the debit took place if we do not receive a response from the account service so the only thing we can do would be assume that it did not take place and retry the service call given the scenario from fig 2 this would be very bad as the debit did take place so we would duplicate the debiting of the account.

So we have a few options of how we can deal with this issue:

  1. We don’t perform retries on the client side automatically instead we handle the situation manually and could perform a compensating action before attempting a retry (i.e. perform a credit journal)
  2. We check on the account service whether we have already had a debit for the chosen account and amount in a specific time frame and if so we ignore the call (this would not work in this example as we may have a genuine need to debit the account twice within a specific time frame)
  3. Have the client send something unique (i.e. GUID) for the current debit that you want to take place this can then be used by the account service to check if we have already performed the journal associated with this request.

The last 2 options are to make the account service idempotent my recommendation would be to use the last option and this is the option I will demonstrate for the rest of the post.

Ok so the first change we should make is to add a unique id to the request that gets sent, it is the responsibility of the client application to associate this unique id with the UoW being carried out (i.e. if the client application was allowing an overdrawn account be settled then the UoW would be a settlement and we can then associate this id to the settlement) so the client needs be changed to store this unique id.

The next change is for the account service to perform a check to make sure that we have not already performed a journal for the id passed to it, this is accomplished by storing each id against the result (in this case the journal reference) once the journal has taken place and querying it as the first task, if we find the id then we simply return the journal reference in the response back to the client.

If the id is not found we must not have performed a journal so we need to make the call to the banking system as we were before and then store away the id and the journal reference and return the response back to the client.

Now that this is in place we can have the client perform retries (within a certain threshold) if we receive no response back from the account service safe in the knowledge that we won’t get duplicate debits from the account, here is a sequence diagram to outline the new strategy:

New idempotent account service

Ah the joys of distributed architecture!

Using dynamic for Stored Procedures

It can be quite cumbersome to call an SP with ado.net when you also need to supply parameters, you usually end up with this:

using (var connection = new SqlConnection(connectionString))
{
    connection.Open();
    var cmd = connection.CreateCommand();
    cmd.CommandType = CommandType.StoredProcedure;
    cmd.Text = "MyStoredProcedure";
    var firstParameter = new SqlParameter("FirstParameter", value);
    var secondParameter = new SqlParameter("SecondParameter", DBNull.Value);
    cmd.Parameters.Add(firstParameter);
    cmd.Parameters.Add(secondParameter);
    var reader = cmd.ExecuteReader();
}

There is quite a lot of ceremony in there to make a simple SP call, using a dynamic object we can reduce this down to:

using (var connection = new SqlConnection(connectionString))
{
    connection.Open();
    dynamic myStoredProcedure = new DynamicStoredProcedure("MyStoredProcedure");
    myStoredProcedure.FirstParameter = value;
    myStoredProcedure.SecondParameter = null;
    var reader = myStoredProcedure.ExecuteReader(connection);
}

As you can see it still requires the caller to look after the connection so that it can manage it properly and make sure that it is closed however now using the dynamic keyword we can make calls to properties that don’t exist but instead are resolved dynamically and treated as a SqlParameter using the name of the property as the name of the parameter and the value is used as the parameter value, also nulls are automatically converted to DBNull.Value as the value for the parameter.

The DynamicStoredProcedure object declaration looks like this:

public class DynamicStoredProcedure : DynamicObject
{
	protected string m_Name = string.Empty;
	protected IDictionary<string, SqlParameter> m_Parameters = new Dictionary<string, SqlParameter>();
	
	public DynamicStoredProcedure(string storedProcedureName)
	{
		m_Name = storedProcedureName;
	}

	public void AddParameter(SqlParameter parameter)
	{
		if (m_Parameters.ContainsKey(parameter.ParameterName))
		{
			m_Parameters[parameter.ParameterName] = parameter;
		}
		else
		{
			m_Parameters.Add(parameter.ParameterName, parameter);
		}
	}

	public override bool TrySetMember(SetMemberBinder binder, object value)
	{
		var convertedValue = (value == null) ? DBNull.Value : value;

		if (m_Parameters.ContainsKey(binder.Name))
		{
			m_Parameters[binder.Name].Value = convertedValue;
			return true;
		}

		var param = new SqlParameter(binder.Name, convertedValue);
		m_Parameters.Add(binder.Name, param);
		return true;
	}
	
	public SqlDataReader ExecuteReader(SqlConnection connection)
	{
		var cmd = CreateCommand(connection);
		var reader = cmd.ExecuteReader();
		return reader;
	}

	public T ExecuteScalar<T>(SqlConnection connection)
	{
		var cmd = CreateCommand(connection);
		var result = cmd.ExecuteScalar();
		var type = typeof(T);

		if (result.GetType().IsAssignableFrom(type))
			return (T)result;

		throw new InvalidOperationException(string.Format("Cannot convert result [{0}] to type [{1}]", result, type));
	}

	public void ExecuteNonQuery(SqlConnection connection)
	{
		var cmd = CreateCommand(connection);
		cmd.ExecuteNonQuery();
	}

	private SqlCommand CreateCommand(SqlConnection connection)
	{
		var cmd = connection.CreateCommand();
		cmd.CommandType = CommandType.StoredProcedure;
		cmd.CommandText = m_Name;
		cmd.Parameters.AddRange(m_Parameters.Values.ToArray());
		return cmd;
	}
}

I have added the standard Execute methods that you find on SqlCommand There are also times were you need to add a parameter explicitly for instance for output parameters and if you want more control over the configuration of the parameter in this case I have added an explicit AddParameter for these cases.

Lightweight Object Mapper Infomercial

Tired with writing monotonous & repetitive mapping code, wish their was an easier way to map object A to object B, well look no further, introducing the Lightweight Object to Object Mapper.

As Seen on TV

The Lightweight Object to Object Mapper is designed to make object mapping a doddle! By employing the latest techniques in convention over configuration reduced coding is guaranteed[1] and for those tricky to reach places explicit property mapping can be achieved with minimal effort and fuss.

So don’t lose out, download Lightweight Object Mapper right now and start enjoying object to object mapping, but that’s not all for a limited time period if you download the Lightweight Object to Object Mapper you will receive accompanying unit tests… so don’t delay download now!

  1. Guarantee is subject to both the mapped from & mapped to object structure

Null Object Pattern by example

I feel sorry for the Null Object pattern, while Abstract Factory Pattern, Strategy and the other patterns from the GoF book get all the attention the Null Object pattern doesn’t get a look in. This is a shame because this pattern can come in very useful, saving you many scattered ==null checks across your code, I’m going to show a real world use of Null Object pattern (and Adapter pattern as a by-product) using example logging objects:

Lets get to it, we have an ILog it looks like this:

public interface ILog
{
    void Debug(string message);
}

Ok granted I have made it a simple example at least it ain’t an ICar or IAnimal 😉 how do we get hold of our ILog with an ILogFactory of course:

public interface ILogFactory
{
    ILog GetLog(Type type);
}

We can make this easier to get hold of by having a static gateway so we aren’t having to pass around an ILogFactory to every object or having to new up a ILogFactory instance everytime we want to log:

public static class Log
{
    private static logFactory;

    public static void InitializeWith(ILogFactory logfactory)
    {
	this.logFactory = logFactory;
    }

    public static ILogFactory Current(Type type)
    {
        return logFactory.GetLog(type);
    }
}

You could also leverage a IoC container here instead.

Now we want some implementations I’m a fan of log4net so I would implement the following:

public class Log4NetLogFactory
{
    public ILog GetLog(Type type)
    {
	//retrieve an adapted log4net log something like...
	log4net.ILog rawLog = log4net.LogManager.GetLogger(type);
	return new Log4NetLog(rawLog);
    }
}

With Log4NetLog looking like this:

public class Log4NetLog : ILog
{
    private log4net.ILog log;

    public Log4NetLog(log4net.ILog logToAdapt)
    {
	this.log = logToAdapt;
    }

    public void Debug(string message)
    {
	if (log.IsDebugEnabled)
	    log.Debug(message);
    }
}

Now we want to be able to log in our client code like this:

Log.Current(this.GetType()).Debug("some debug message");

The problem of course is that if we try to do this without hooking up our static Log class to an ILogFactory we get the dreaded null reference exception, this is bad! Our logging code should not throw exceptions, enter the Null Object pattern:

public static class Log
{
    private static logFactory = new NullLogFactory();

    public static void InitializeWith(ILogFactory logfactory)
    {
	if (logFactory != null)
	    this.logFactory = logFactory;
    }

    public static ILogFactory Current(Type type)
    {
        return logFactory.GetLog(type);
    }
}

We instantiate a new NullLogFactory object and assign it to the static logFactory member, we also perform a check to make sure no nulls are sent to the InitializeWith method, here is the NullLogFactory:

public class Log : ILogFactory
{
    public ILog GetLog(Type type)
    {
        return new NullLog();
    }
}

All this does is new up a NullLog object:

public class NullLog : ILog
{
    public void Debug(string message)
    {
    }
}

The NullLog does nothing whatsoever apart from prevent us from receiving null reference exceptions, the neat thing about this though is that we could potentially capture attempts to call methods the NullLog and perform some sort of internal debugging:

public class NullLog : ILog
{
    public void Debug(string message)
    {
        if (Config.IsInternalDebugEnabled)
            Debug.Write("'{0}' sent to NullLog", message);
    }
}

That way users of the logging objects can be notified if logging has taken place but a log factory has not been hooked up.

Multiple instances of same windows service

There are times were you want to be able install multiple instances of the same windows service onto the same server (i.e. to support different environments but you have limited servers at your diposal) this poses a problem, you are probably aware windows will not allow more than 1 windows service to be installed with the same service name and when you create a windows service project and an installer you assign the service name at design time.

We need is someway to assign the service name at runtime, well after some extensive googling around I found a way of achieving this, the ProjectInstaller has 2 methods you can override Install and Uninstall which enables us to make changes to the service installer at the at runtime 🙂 Yeah that’s great however how do we assign the service name?! Fear not! at our disposal is the Context class that happens to have a Parameters dictionary so we can now capture custom command arguments passed to installutil.exe, enough yapping time for some code:

public override void Install(System.Collections.IDictionary stateSaver)
{
	RetrieveServiceName();
	base.Install(stateSaver);
}

public override void Uninstall(System.Collections.IDictionary savedState)
{
	RetrieveServiceName();
	base.Uninstall(savedState);
}

private void RetrieveServiceName()
{
	var serviceName = Context.Parameters["servicename"];
	if (!string.IsNullOrEmpty(serviceName))
	{
		this.SomeService.ServiceName = serviceName;
		this.SomeService.DisplayName = serviceName;
	}
}

In this instance I have made the servicename argument optional in which case it will use the default assigned at design time, you could enforce it in your version.

We can now use this like this:

installutil /servicename=”My Service [SysTest]” d:\pathToMyService\Service.exe

and for uninstall:

installutil /u /servicename=”My Service [SysTest]” d:\pathToMyService\Service.exe

Note for the uninstall you need to supply the servicename so windows how to resolve the service to uninstall

One thing that suprises me is how hidden this implementation is, most results bring up people wanting the same functionality and being told it ain’t possible and articles that suggest messing with config files this to me seems to be a much simpler and nicer way to achieve multiple instances.

New Staff Intranet Release

I have found enough spare time to put up a new release of the staff intranet project, for those who are not aware of this project it is a demonstration of using best practices, principles & patterns in a real world web application so if your looking for pointers or some code to use for your own applications go give it a look on codeplex.

In this newest version I have added AOP support to cut down on cross cutting code and also the ability to delete staff members from the GridView, most of the time spent was fighting against the asp.net controls (suprise, surprise) such as the GridView and the ObjectDataSource, I’m not sure what the guy(s) who created the ObjectDataSource object was smoking at the time but it must have been stronger than just tobacco 🙂

My next release I want to demonstrate adding some service support showing how we can re-use existing code so they become little more than a remote facade (in theory!).