Multi Search Engine Enhancement Using Design Patterns

Before reading it might be worth having a look at my previous post on this mini project, the current structure looks like this:

Current Class Structure

As you can see it is fairly straight forward to plugin your own SearchEngine implementation, either by taking advantage of the AbstractSearchEngine or by implementing the ISearchEngine interface.

In both cases the client wanting to plugin their custom searchengine implementation only needs to concentrate on writing code to extract search results from the searchengine they are querying.

One of the problems at the moment is that the searches are performed synchronously which could potentially make the UI un-responsive if the query takes a while, this is a big no-no, so ideally I would like to make the calls asynchronously and then the UI (or any other listening object) can be notified when results are returned.

We could start by making some changes to the structure:

Updated Structure V1

Now we have introduced a new method SearchAsync and also a new event SearchCompleted we have also introduced the same to the AbstractSearchEngine class, there are some issues with these changes:

  1. This could be a first sign of polluting an interface unrelated responsibility – Any client who wants to plug in their own searchengine cannot just concentrate on implementing code to bring back results from a searchengine, they must now implement a asynchronous searching strategy.
  2. Using this approach as stated above we have made it the clients responsibility to add the asynchronous code this could bring about some issues such as the client just not bothering and instead just making a synchronous call to the searchengine or badly implemented asynchronous code that may use threading badly.
  3. There may already be quite a few implemented searchengines that rely on this interface by making this change we are breaking there compatibility with ISearchEngine for a change that may not be affecting them at all

We need to think of another way… We already have a nicely defined ISearchEngine interface which we know we can call into and it will bring back results and allow us to navigate those results, the only thing we are adding is the extra behaviour to perform the calls asynchronously ideally we would like to add this behaviour without effecting what we have already! Enter Decorator stage right ๐Ÿ™‚

The intent of the decorator is to add behaviour dynamically to existing objects which is what were after, So lets come up with an alternative structure:

Updated Structure V2

We now have a brand new class irrelevant fields have been omitted which takes the ISearchEngine in its constructor in most cases this new class will delegate calls straight through to the decorated class however if a search is in progress it needs to take other actions. Using TDD I came up with the class below:

public class AsyncSearchEngineDecorator : ISearchEngine
{
 private delegate void PagingDelegate();
 private delegate void SearchingDelegate(string SearchString);

	private ISearchEngine decoratedSearchEngine;
 private volatile bool resultsReturned = false;
 private readonly object locker = new object();

	public event EventHandler SearchComplete;

	private void SetResultsReturnedTo(bool value)
 {
 	lock (locker)
 	{
 		resultsReturned = value;
 	}
 }

	private bool ResultsReturnedValue()
 {
 	lock (locker)
 	{
 		return resultsReturned;
 	}
 }

	public AsyncSearchEngineDecorator(ISearchEngine searchEngineToDecorate)
 {
 	if (searchEngineToDecorate == null)
 		throw new ArgumentNullException("searchEngineToDecorate", "searchEngineToDecorate cannot be null");

		this.decoratedSearchEngine = searchEngineToDecorate;
 }

	public void Search(string searchText)
 {
 	SetResultsReturnedTo(false);

		PerformAsync(new SearchingDelegate(delegate(string text)
 	{
 		decoratedSearchEngine.Search(text);
 	}), new object[] {searchText});
 }

	private void RaiseSearchComplete()
 {
 	if (SearchComplete != null)
 		SearchComplete(this, EventArgs.Empty);
 }

	public string Name
 {
 	get { return decoratedSearchEngine.Name; }
 }

	public bool HasResults
 {
 	get {
 		if (ResultsReturnedValue())
 			return decoratedSearchEngine.HasResults;
 		else
 			return false;
 	}
 }

	public IList Results
 {
 	get {
 		if (ResultsReturnedValue())
 			return decoratedSearchEngine.Results;
 		else
 			return new List();
 	}
 }

	public void NextPage()
 {
 	if (ResultsReturnedValue())
 	{
 		PerformAsync(new PagingDelegate(delegate
 		{
 			decoratedSearchEngine.NextPage();
 		}));
 	}
 	else
 		RaisePagingWhileSearchInProgressException("NextPage");
 }

	public void PreviousPage()
 {
 	if (ResultsReturnedValue())
 	{
 		PerformAsync(new PagingDelegate(delegate
 		{
 			decoratedSearchEngine.PreviousPage();
 		}));
 	}
 	else
 		RaisePagingWhileSearchInProgressException("PreviousPage");
 }

	private static void RaisePagingWhileSearchInProgressException(string method)
 {
 	throw new InvalidOperationException(string.Format("{0} cannot be called while search is in progress", method));
 }

	public int PageSize
 {
 	get
 	{
 		return decoratedSearchEngine.PageSize;
 	}
 	set
 	{
 		decoratedSearchEngine.PageSize = value;
 	}
 }

	public int CurrentPage
 {
 	get {
 		if (ResultsReturnedValue())
 			return decoratedSearchEngine.CurrentPage;
 		else
 			return -1;
 	}
 }

	private void PerformAsync(Delegate action)
 {
 	PerformAsync(action, null);
 }

	private void PerformAsync(Delegate action, object[] args)
 {
 	SetResultsReturnedTo(false);

		ThreadPool.QueueUserWorkItem(delegate
 	{
 		action.DynamicInvoke(args);
 		SetResultsReturnedTo(true);
 		RaiseSearchComplete();
 	});
 }
}

If there is any issues with the threading code please give me heads up ๐Ÿ™‚ Now in my UI I can use the new class as so:

searchEngine = new AsyncSearchEngineDecorator(searchEngine);
searchEngine.SearchComplete += new EventHandler(HandleSearchPerformed);

You may thinking at this point… “hang on you class diagram looks nothing like the class diagram from the GoF book” …and you would be right, one mistake that some people can make when learning about design patterns is that the structure of your implementation must match the structure shown, this is not the case there are many ways to implement a design pattern I too got hung up on using the class diagram as a template for how design patterns should be created, however I read a chapter from Refactoring to Patterns that brought it into perspective that if you need to introduce a design pattern then implement it in the best way for your surroundings not the best way according to a class diagram from GoF or some design patterns website.

So what have we ended up with, well we have created a new class which has the responsibility of making calls against an ISearchEngine aysnchronously by doing this we have kept both the ISearchEngine and AbstractSearchEngine responsibilities to be just concerned with pulling results back from a custom searchengine. The changes to the UI have been minimal it simply wraps the existing searchEngine into the new AsyncSearchEngineDecorator object and handles the SearchComplete event.

Advertisements

Syntax Trees using Composite/Intepreter Pattern Part 3

Part 1

Part 2

Where we left off last time we had 2 leaf nodes that peformed operations against two operands (LessThanWhere & EqualToWhere) now I want to add a composite class to perform some logical operations.

Creating the Composite Class

Like before we will drive out the requirements TDD style:

[Test]
public void Should_add_multiple_and_where_clause_with_string_value_on_BuildQuery()
{
	string expected = "WHERE ((col1 = 'test') AND (col2 = 'test2'))";
	SelectQueryBuilder queryBuilder = new SelectQueryBuilder();
	EqualToWhere where1 = new EqualToWhere("col1", "test");
	EqualToWhere where2 = new EqualToWhere("col2", "test2");
	queryBuilder.AddWhere(new AndWhere(where1, where2));

	string output = queryBuilder.WhereSection();
	Assert.AreEqual(expected, output);
}

Now we need to add the new class to get compilation:

 public class AndWhere : AbstractWhere
{
	public AndWhere(AbstractWhere left, AbstractWhere right)
	{
	}

	public override string Output()
	{
		return "";
	}
}

It should now compile so we run the tests and get a fail, lets go and make them pass:

public class AndWhere : AbstractWhere
{
	AbstractWhere left;
	AbstractWhere right;

	public AndWhere(AbstractWhere left, AbstractWhere right)
	{
		this.left = left;
		this.right = right;
	}

	public override string Output()
	{
		StringBuilder output = new StringBuilder();
		output.Append("(");
		output.Append(left.Output());
		output.Append(" AND ");
		output.Append(right.Output());
		output.Append(")");

		return output.ToString();
	}
}

This class expects 2 operands to be passed in, the only constraint being that they derive from the AbstractWhere class, when the SQL is built in the Output method the ANDWhere class simply delegates off to the left and right operands respectively, We should now have a green tick for the test above.

Full control over SQL produced

I won’t show the implmentation for an ORWhere class as I’m sure you can guess how it can be implemented instead I will show how we can use the classes we have now to produce SQL under our control i.e. nested SQL:

[Test]
public void Should_add_multiple_and_nested_or_where_causes_with_string_values_on_BuildQuery()
{
	string expected = "WHERE ((col1 = 'test') OR ((col2 = 'test2') AND (col3 = 'test3')))";

	SelectQueryBuilder queryBuilder = new SelectQueryBuilder();

	EqualToWhere where1 = new EqualToWhere("col1","test");
	EqualToWhere where2 = new EqualToWhere("col2", "test2");
	EqualToWhere where3 = new EqualToWhere("col3", "test3");
	AbstractWhere andWhere = new AndWhere(where2, where3);
	queryBuilder.AddWhere(new OrWhere(where1, andWhere));

	string output = queryBuilder.WhereSection();
	Assert.AreEqual(expected, output);
}

We can now build up a nice tree of the syntax we would like to use as our SQL without having to worry about levels, What we have from the above can be viewed as:

SQL Syntax Tree Example

Handling other kinds of Operands

In part 1 I made reference to a question that was posted on the SelectQueryBuilder codeproject page in which the poster asked if it would be possible to use functions as part of the where clause:

WHERE DATEDIFF(dd,col1,'20080101') >= 0

One problem is that this function is a SQL server function making the above SQL constrained to SQL server, so before implementing the above into our code we need to keep in mind that we are limiting the SQL generation to SQL server, so if we accept this constraint we could try the following:

[Test]
public void Should_add_single_explicit_function_with_equals_on_BuildQuery()
{
	string expected = "WHERE (DATEDIFF(dd,col1,'20070101') = 0)";
	SelectQueryBuilder queryBuilder = new SelectQueryBuilder();

	FunctionWhere functionWhere = new FunctionWhere("DATEDIFF", "dd", "col1", "'20070101'");
	EqualToWhere equalFunctionWhere = new EqualToWhere(functionWhere, 0);
	queryBuilder.AddWhere(equalFunctionWhere);

	string output = queryBuilder.WhereSection();
	Assert.AreEqual(expected, output);
}

Ok, so lets get that test compiling, firstly we need to add the new FunctionWhere class:

public class FunctionWhere : AbstractWhere
{
	public FunctionWhere(string name, params string[] args)
	{
	}

	public override string Output()
	{
		return "";
	}
}

Next we also need to allow EqualToWhere to accept an overloaded constructor that can accept an AbstractWhere:

public EqualToWhere(AbstractWhere where, object rightSide)
	: this(where.Output(), rightSide)
{
}

This neat we can simply chain the constructor by calling the Output method on the where object passed in.Now to get the test to pass we need to make create the implementation code in FunctionWhere:

public class FunctionWhere : AbstractWhere
{
	private string name;
	private string[] args;

	public FunctionWhere(string name, params string[] args)
	{
		this.name = name;
		this.args = args;
	}

	public override string Output()
	{
		StringBuilder output = new StringBuilder();
		output.Append(name);
		output.Append("(");

		foreach (string arg in args)
		{
			output.Append(arg);
			output.Append(',');
		}

		output.Remove(output.Length - 1, 1); // strip last comma
		output.Append(")");

		return output.ToString();
	}
}

I have gone for the approach of simply passing the function name then passing the arguments in, this could easily be changed to be more specific i.e. using a DateDiffWhere however the FunctionWhere performs what we need to for now.

Closing Notes

I hope this series of articles have provided some help on understanding how we can use patterns to refactor existing code one thign to note is that throughout all these changes I have not once had to re-open the SelectQueryBuilder class all the modifications where made externally which is inline with the OCP coined by Bertrand Meyer.

If your after a great book on refactoring to patterns I recommend Refactoring to Patterns by Joshua Kerievsky, I have recently read it and it is what inspired me to create this series.

Syntax Trees using Composite/Intepreter Pattern Part 2

Part 1

Syntax Trees

In the GOF Design Patterns book you can see an example of using the composite pattern in the interpreter chapter, the way we will implement the syntax tree will be very similar. If you are not familiar with either the composite or interpreter patterns I suggest a trip to the data & object factory patterns site

Achieving what were after

The first step is to begin defining what were after, I write a quick simple unit test to get the ball rolling:

[Test]
public void Should_add_single_where_clause_with_string_value_on_WhereSection()
{
    string expected = "WHERE (col1 = 'test')";

    SelectQueryBuilder queryBuilder = new SelectQueryBuilder();

    EqualToWhere where1 = new EqualToWhere("col1", "test");
    queryBuilder.AddWhere(where1);

    string output = queryBuilder.WhereSection();
    Assert.AreEqual(expected, output);
}

We need to undertake a few steps to get this required result to compile:

  1. We need to add a new class “EqualToWhere”
  2. SelectQueryBuilder needs to be changed to support a AddWhere which can accept the new object
  3. SelectQueryBuilder needs a new method that we can use to check our result “WhereSection()”

Creating the new class

The EqualToWhere class will be what’s called a leaf node and will accept leftside and rightside arguments and will write out SQL via the Output method:

public class EqualToWhere
{
 public EqualToWhere(string leftSide, object rightSide)
 {
 	LeftSide = leftSide;
 	RightSide = rightSide;
 }

	public string Output()
 {
     return "";
        }
}

Changing SelectQuerybuilder’s AddWhere method

private EqualToWhere where;
public void AddWhere(EqualToWhere where)
{
 this.where = where;
}

Adding new WhereSection method to SelectQueryBuilder

I could have chosen not to do this step and instead used the existing BuildQuery method however I didn’t want to start breaking the rest of the object when I’m only interested in the where clauses.

public string WhereSection()
{
 return "WHERE " + where.Output();
}

Getting the test to pass

After we compile and run the tests we get a red cross our next task is to get a green tick, so if we change the Output method in EqualToWhere to this:

public string Output()
{
 StringBuilder output = new StringBuilder();
 output.Append("(");
 output.Append(LeftSide);
 output.Append(" = ");

	switch (RightSide.GetType().Name)
 {
 	case "String": output.Append("'" + ((string)RightSide).Replace("'", "''") + "'"); break;
 	case "DateTime": output.Append("'" + ((DateTime)RightSide).ToString("yyyy/MM/dd hh:mm:ss") + "'"); break;
 	case "DBNull": output.Append("NULL"); break;
 	case "Boolean": output.Append((bool)RightSide ? "1" : "0"); break;
 	case "SqlLiteral": output.Append(((SqlLiteral)RightSide).Value); break;
 	default: output.Append(RightSide.ToString()); break;
 }

	output.Append(")");
 return output.ToString();
}

Once this is compiled and the test ran again we should see green, great success!

Now for the refactoring

From the above we can establish that we need to have a common base class that we can use as the basis for our classes, this will include the standard Output method so we have a common interface no matter if we have a leaf class as in the example or a composite class which will be shown in the next part.

So we use an extract superclass refactoring:

public abstract class AbstractWhere
{
 public abstract string Output();

	protected string FormatForSqlFrom(object input)
 {
 	string SqlValue = string.Empty;

		switch (input.GetType().Name)
 	{
 		case "String": SqlValue = "'" + ((string)input).Replace("'", "''") + "'"; break;
 		case "DateTime": SqlValue = "'" + ((DateTime)input).ToString("yyyy/MM/dd hh:mm:ss") + "'"; break;
 		case "DBNull": SqlValue = "NULL"; break;
 		case "Boolean": SqlValue = ((bool)input ? "1" : "0"); break;
 		case "SqlLiteral": SqlValue = (((SqlLiteral)input).Value); break;
 		default: SqlValue = input.ToString(); break;
 	}

		return SqlValue;
 }
}

I also did a pull up method refactoring as I could predict that in the future derived classes will need to format objects to be used in writing out SQL and therefore this prevents duplication. Once the changes were made I went back to my unit tests to make sure I hadn’t broken anything, green again! We now have the common interface that the classes will use (essentially the Output method that returns SQL), we can now make the SelectQueryBuilder use the common interface rather than a specific derived version i.e. EqualToWhere that it references now, so lets make the change:

private AbstractWhere where
public void AddWhere(AbstractWhere where)
{
 this.where = where;
}

That’s reduced the coupling and given us greater extensibility, we can now pass in any object that adheres to our common interface (AbstractWhere). I do a re-run of the unit tests, still green were well on our way!

Creation of other leaf node classes

Now at the moment we have only 1 leaf node that can create the SQL to an equal operator against 2 operands, lets add another operator object TDD style, and to mix it up we’ll check against a date object on the right operand:

[Test]
public void Should_add_single_lessthan_where_clause_with_date_value_on_WhereSection()
{
     string expected = "WHERE (col1 < '2008/01/21 20:00:00')";
     SelectQueryBuilder queryBuilder = new SelectQueryBuilder();

     LessThanWhere where1 = new LessThanWhere("col1", new DateTime(2008,1,21,20,0,0));
 queryBuilder.AddWhere(where1);

     string output = queryBuilder.WhereSection();

     Assert.AreEqual(expected, output);
}

Spotted a small bug the ToString method for the DateTime formatting needed to be “yyyy/MM/dd HH:mm:ss” rather than “yyyy/MM/dd hh:mm:ss” a clear example of the usefulness of unit testing! ๐Ÿ™‚ Our LessThanWhere class looks like this:

public class LessThanWhere
{
	public LessThanWhere(string leftSide, object rightSide)
	{
		LeftSide = leftSide;
		RightSide = rightSide;
	}

	public override string Output()
	{
		StringBuilder output = new StringBuilder();
		output.Append("(");
		output.Append(LeftSide);
		output.Append(" < ");
		output.Append(FormatForSqlFrom(RightSide));
		output.Append(")");
		return output.ToString();
	}
}

So I re-check my unit test and get it to pass, you can see the only change from the EqualToWhere class being the operator in the output method, this would be an ideal candidate to refactor the duplication with perhaps a BinaryWhere superclass using a template method, I’ll leave that as an optional user exercise ๐Ÿ˜‰

Next steps

I will introduce composite classes that bring together the 2 leaf nodes we have created above to allow us to create logical comparisons, nested expressions and also prove the extensibility of what we have.

Part 3

Syntax Trees using Composite/Intepreter Pattern Part 1

Introduction

Recently I used a really useful library that was posted on codeproject called SelectQueryBuilder I found it a really useful library that helped to build up SQL queries without having to resort to nasty string concatenation, this post should not be seen as me putting down the SelectQueryBuilder in anyway I think it’s a great library and fulfills everything most people will need, the purpose of this post is to show how we can utilise design patterns to refactor a real world application for further changing requirements.

Current Where Clause Usage

WhereClause whereClause1 = new WhereClause();
whereClause1.ComparisonOperator = Comparison.Equals;
whereClause1.FieldName = "col1";
whereClause1.Value = "test";

queryBuilder.AddWhere(whereClause1, 1);

Notice the “1” argument this is used to specify the level that the clause should appear at, for example, if we added another where clause at the same level then these clauses would be AND’ed together however if we added another clause at level 2 then they would be OR’ed together:

WHERE ((col1 = 'test') AND (col2 = 'test2')) -- same level
WHERE (col1 = 'test')  OR (col2 = 'test2') -- different level

Issues

This poses a few of issues the first being that the client has now got to be control of a local level variable and also the builder needs to have protection in place to make sure that a bad index cannot be used.

The second being related to generating SQL how you would like to be written, say I want to have a WHERE statement like this:

WHERE (col1 = 'test')  AND ((col1 = 'test2') OR (col2 = 'test3'))

We can’t get the SQL to produce the above because as-is clauses in different levels are OR’ed and same levels are AND’ed together which is the opposite of the above, now we can get the builder to produce a query that will provide the same logic:

WHERE ((col1 = 'test2') AND (col1 = 'test')) OR ((col2 = 'test3') AND (col1 = 'test'))

However we have now had to repeat the AND clauses for both OR clauses and it is not a readable as the SQL we were after.The third issue is extensibility, the where clause uses the following attributes:

  • Comparison Operator
  • Field Name
  • Value

This is somewhat restrictive one of the questions posted on the codeproject page is about having the builder support functions:

WHERE DATEDIFF(dd,col1,'20080101') >= 0

Given the current Where clause there’s no way we could add this in.

Next Steps

Next I want to get into how we can architect the syntax tree that will resolve the issues above, and reduce the code in the builder for building the WHERE clauses to one line ๐Ÿ™‚

Part 2

Part 3ย