Oi, oi, oi ...

Been awhile. A few months back I started working on some material for some logging-based posts, but got side tracked by work and other stuff too. Still on the radar, but also ramping up on some Silverlight - especially the design aspects!

While searching for Silverlight related tutorials and sites, found this gem .toolbox. It's a clever site with loads of content that I am slowly working through. As a dev, the Expression Blend tutorials are a good way to get familiar with the product. I am especially interested in using Blend to prototype faster so that I may elicit users feedback sooner. I am also enjoying myself with the design sessions, and recommend these to any developer interested in User Interface design work.



.toolbox user avatar for johnny_g



I <3 palindromes ...

w00t! Stack Overflow!

Okay, so maybe not so unique, but pretty cool nonetheless!

Unit testing, how do I mock a signature with ref or out?

So, recently I had the distinct pleasure of mocking a service with method members that contained ref parameters. I thought this rather odd (I have never been a fan of 'in-place' modifications or returns), but not particularly special. That is, not until I came to unit test consumers of the service.

// immutable third-party service interface
public interface ISubmitMessage
{
    // ugly method
    void Submit (string username, string password, ref byte[] message);
}

// our poor poor consumer that has to interface
// with big bad mean service above
public class SubmitAdapter
{
    void Process (ISubmitMessage service)
    {
         // 1. set username and password from configuration
         string username = string.Empty;
         string password = string.Empty;

         // 2. generate byte-encoding of a string message
         byte[] messageBytes = null;

         // 3. invoke service
         service.Submit (username, password, ref messageBytes);

         // 4. inspect messageBytes return value for
         // success\failure
    }
}

Depending on our mocking solution, ref and out method parameters may or may not be supported. From personal experience many mocking frameworks do not support ref and out parameters. My current mock solution of preference is Moq, and with Moq v4.0.10827 (beta) we cannot verify pass-in and pass-out parameters out-of-box.

[Test]
public void Test_Process ()
{
    Mock<ISubmitMessage> service = 
        new Mock<ISubmitMessage> (MockBehavior.Strict);

    string username = "some.username";
    string password = "some.password";
    byte[] messageIn = null;
    byte[] messageOut = new byte[] { 1 };

    // ideally, we would like something similar to this
    service.
        Setup (
        s => s.Submit (
        username,
        password,

        // where we inspect on way in, and define a return value
        ref It.IsRef<byte[]> (d => Equals (d, null)).Returns (messageOut)));

    SubmitAdapter adapter = new SubmitAdapter ();
    adapter.Process (service.Object);

    service.VerifyAll ();
}

Unfortunately, this feature set does not quite exist yet - though as we shall soon see, we do not necessarily need direct support.

The key to mocking ref and out method parameters is understanding our requirements as a consumer and our requirements as a producer - and while disjoint, they can be satisfied via transform. By example, our consumer expects a functional implementation of a specific interface,

public class SubmitMessageStub : ISubmitMessage
{
    public void Submit (string username, string password, ref byte[] message)
    {
        // ??? 
    }
}

our test method expects a contract we can mock,

// a mock-friendly "normalized" version of original interface
public interface ISubmitMessageMock
{
    // similar signature to source method ISubmitMessage.Submit,
    // note new response object,
    SubmitResponse Submit (string username, string password, byte[] message);

    // new response class, a super-set of data expected to be 
    // returned
    public class SubmitResponse
    {
        // contains return message value
        public byte[] Message { get; set; }
    }
}

Jumping back to our consumer, it is a relatively trivial matter to transform from original to mock interface

public class SubmitMessageStub : ISubmitMessage
{

    // simple constructor that accepts a mock-friendly implementation
    private readonly ISubmitMessageMock _mock = null;
    public SubmitMessageStub (ISubmitMessageMock mock)
    {
        _mock = mock;
    }

    public void Submit (string username, string password, ref byte[] message)
    {
        // NOTE: do NOT add any additional parameter checking
        // or verification. our responsibility is to perform a
        // transform and delegate, let expectation testing occur
        // in caller,

        // 1. transform to mock
        ISubmitMessageMock.SubmitResponse response = _mock.
            Submit (username, password, message);
        // 2. reverse-transform
        message = response.Message;
    }
}

Our test setup now looks like

[TestMethod]
public void Test_Process()
{
    Mock<ISubmitMessageMock> serviceMock =
        new Mock<ISubmitMessageMock> (MockBehavior.Strict);

    // proper setup, you can do this with Moq!
    string username = "some.username";
    string password = "some.password";
    byte[] message = null;

    // we control out parameters here through internal
    // response object
    ISubmitMessageMock.SubmitResponse serviceMockResponse =
        new ISubmitMessageMock.SubmitResponse 
        {
            Message = new byte[] { 1, }, 
        };

    serviceMock.

        // we control in parameters here through standard
        // Moq inspection
        Setup (s => s.Submit (username, password, message)).
        Returns (serviceMockResponse);

    SubmitAdapter adapter = new SubmitAdapter ();

    // simply wrap mock-friendly version with our stub!
    adapter.Process (new SubmitMessageStub (serviceMock.Object));

    // and finally verify service interactions!
    serviceMock.VerifyAll ();
}

Unit testing, how do I unit test a Singleton?

Well, as with our WebService client sample, this question is a misnomer. More often than not we are interested in unit testing consumers of a Singleton than unit testing a Singleton's functional set - and depending on implementation, this may not be as straightforward as we think.

Consider

// a fairly typical Data Access Layer implementation (DAL). have actually
// encountered this on-site [shudders].
public static class DatabaseConnectionFactory
{
    public static IDbConnection GetDatabaseConnection ()
    {
        // 1. get connection string from config
        string connectionString = null;

        // 2. create connection
        SqlConnection connection = null;
        connection = new SqlConnection (connectionString);
        connection.Open ();

        // 3. some other custom stuff
        // 4. return connection
        return connection;
    }

    public static IDbCommand GetDatabaseCommand (
        string commandString,
        IDbConnection connection)
    {
        // 1. set custom stuff like transaction and timeout
        // 2. return command
        return new SqlCommand (commandString, connection);
    }
}

// embedded DAL logic in business tier - anyone else vomit in their 
// mouth just a little? - however what is especially offensive is the
// direct calls to data tier via static class DatabaseConnectionFactory
public class AppointmentBusinessObject
{
    public const string CommandString_LoadById_OneParameter = 
@"SELECT * 
FROM Appointments 
WHERE AppointmentId = {0}";

    public void LoadById (long appointmentId)
    {
        // 1. create Sql command string,
        string commandString = 
            string.Format (
            CommandString_LoadById_OneParameter,
            appointmentId);

        try
        {
            // 2. get connection,
            IDbConnection connection = 
                DatabaseConnectionFactory.
                GetDatabaseConnection ();

            // 3. get command,
            IDbCommand command = 
                DatabaseConnectionFactory.
                GetDatabaseCommand (commandString, connection);

            // 4. execute command
            Reader reader = command.ExecuteReader ();

            // 5. read contents
        }
        finally
        {
            // 6. close connection,
            if (connection != null) 
            {
                connection.Close ();
            }
        }

    }

}

Now, there are many reasons why this is a poor design, not the least of which is that we are locked into a single connection and a single implementation. We are forced to duplicate this source if any of these parameters need to change - and believe me this has happened!

If this were not reason enough to contemplate a complete revision, our ability to unit test is also impaired. Specifically, we cannot unit test or verify LoadById without hitting a datastore! In fact, we cannot unit test any class or method that invokes LoadById without hitting a datastore.

So what can we do?

Well the first thing to do is identify the actual problem - and here it appears to be a tight coupling between our business and data tiers. If we were to abstract DatabaseConnectionFactory then the result would be a loosely-coupled and more flexible system. To this end, I would suggest introducing an interface (that defines DatabaseConnectionFactory's existing members) and consuming this wherever possible.

For example,

// a simple data store interface, exposes full functional set
// of existing DatabaseConnectionFactory
public interface IDatabaseConnectionFactory
{
    IDbConnection GetDatabaseConnection ();
    IDbCommand GetDatabaseCommand (
        string commandString, 
        IDbConnection connection);
}

// we want this to be as low-impact as possible, so new
// interface defines members that already exist. sadly, 
// static classes cannot implement interfaces directly, 
// and so DatabaseConnectionFactory must be made an instance 
// class. while this requires a little deft maneuvering, 
// this may be accomplished with *zero* impact to existing 
// consumers! yay!
//
// 1. remove static key word from class declaration, this makes
// class instance-based
public class DatabaseConnectionFactory : IDbConnectionFactory
{

    // 2. remove static key word from members, this simply
    // makes existing method implementations instance-based
    public IDbConnection GetDatabaseConnection () { ... }
    public IDbCommand GetDatabaseCommand (
        string commandString,
        IDbConnection connection) { ... }

    // 3. define *new* static members that delegate to
    // instance-based members, this ensures that existing 
    // consumers do not break
    public static IDbConnection GetDatabaseConnection ()
    {
        // instantiate new factory every call for clarity,
        // potential optimization in declaring a static
        // lazy-loaded instance member, and delegating
        // to that instead
        DatabaseConnectionFactory factory = 
            new DatabaseConnectionFactory ();
        return factory.GetDatabaseConnection ();
    }
    public static IDbCommand GetDatabaseCommand (
        string commandString,
        IDbConnection connection) 
    {
        DatabaseConnectionFactory factory = 
            new DatabaseConnectionFactory ();
        return factory.GetDatabaseCommand (commandString, connection);
    }
}

// embedded DAL logic in business tier - vomitting just a little less -
// less offensive now we reference implementation-independent datastore
// definition
public class AppointmentBusinessObject
{
    // still tightly coupled to Sql-compliant datastore
    // but manageable.
    public const string CommandString_LoadById_OneParameter = 
@"SELECT * 
FROM Appointments 
WHERE AppointmentId = {0}";

    // 1. declare new factory member. this is our *dependency*
    // it *must* be fulfilled for class to operate successfully
    private readonly IDbConnectionFactory _factory = null;

    // 2. expose new parameter constructor, permitting consumers
    // of *this* class to specify an appropriate datastore
    public AppointmentBusinessObject (IDbConnectionFactory factory)
    {
        _factory = factory;
    }

    // 3. expose new parameterless constructor, this preserves
    // existing consumers who may not be "up to speed" regarding
    // this new-fangled connection specification. we also preserve
    // previous operating expectations by defaulting to ... 
    // "default" connection factory, so at worst, we deliver
    // *EXACTLY* same behaviour as before
    public AppointmentBusinessObject ()
        : this (new DatabaseConnectionFactory ())
    {
    }

    // 4. consume!
    public void LoadById (long appointmentId)
    {
        // ...
        try
        {
            IDbConnection connection = 
                _factory.GetDatabaseConnection ();
            IDbCommand command = 
                _factory.GetDatabaseCommand (commandString, connection);
            // ...
        }
        finally
        {
            // ...
        }
    }

}

So, where is the payoff exactly? Well, for one if we now wish to change datastore implementation (say to a MySql, or Oracle, or some other Sql-compliant datastore) we implement a new class and may swap between the two when desired. We also gain the ability to load objects from two or more datastores at the same time!

As a pleasant side-effect, we are also able to unit test LoadById directly, and any consumers that permit datastore specification.

// test business logic without hitting datastore! yay!
[TestMethod]
public void Test_LoadById ()
{
    IDbConnectionFactory mockFactory = null;
    // instantiate mock with expectations
    AppointmentBusinessObject appointment = 
        new AppointmentBusinessObject (mockFactory);
    appointment.LoadById (1024);
    // verify results
}

Unit testing, how do I unit test a WCF Service with dependency OperationContext?

Unit testing WCF Services is a lot like testing Web Services, especially in that they represent remote business logic. However, in some cases testing may not be straightforward, such as when referencing static framework class OperationContext,

// a simple subscription based service.
[ServiceContract (CallbackContract = typeof (ISubscriberCallback))]
public interface ISubscriptionService
{
    // push a subject to service
    [OperationContract]
    void Publish (string subject);

    // subscribe client to publications.
    [OperationContract]
    void Subscribe ();

    // unsubscribe client from publications.
    [OperationContract]
    void Unsubscribe ();
}

// a simple client-side subscription callback.
public interface ISubscriberCallback
{
    // push a subject to subscribers
    [OperationContract]
    void Receive (string subject);
}

// a possible implementation of subscription service
[ServiceBehavior (
    ConcurrencyMode = ConcurrencyMode.Multiple, 
    InstanceContextMode = InstanceContextMode.Single)]
public class SubscriptionService : ISubscriptionService
{

    private readonly object _syncRoot = new object ();
    private readonly List<ISubscriberCallback> _subscribers = 
        new List<ISubscriberCallback> ();

    public void Publish (string subject) { ... }

    public void Subscribe ()
    {
        // hm, static framework references ... 
        ISubscriberCallback subscriber = 
            OperationContext.
            Current.
            GetCallbackChannel<ISubscriberCallback> ();

        lock (_syncRoot)
        {
            if (!_subscribers.Contains (subscriber))
            {
                _subscribers.Add (subscriber);
            }
        }
    }

    public void Unsubscribe () { ... }

}

Unfortunately, if our service implements some sophisticated business logic while initializing callbacks then we risk either losing coverage with no tests or losing time and resources on elaborate environments to support our integration into WCF framework.

Well not to fear, this is strikingly familiar to our Web Service with Context dependency, and I would proscribe a similar solution. Namely,

// a simple abstraction to separate our business from 
// WCF framework. depending on our requirements, could be
// general for re-use elsewhere in our application code,
// or specific if this is a one-off use.
public interface IOperationContext
{
    // add methods as needed, for now we definitely
    // need access to our callbacks!
    CType GetCallbackChannel<CType> ();
}

// a simple wrapper implementation of operation context
public class OperationContextCurrent : IOperationContext
{
    public CType GetCallbackChannel<CType> ()
    {
        return OperationContext.
            Current.
            GetCallbackChannel<CType> ();
    }
}

[ServiceBehavior (
    ConcurrencyMode = ConcurrencyMode.Multiple, 
    InstanceContextMode = InstanceContextMode.Single)]
public class SubscriptionService : ISubscriptionService
{

    private readonly IOperationContext _context = null;
    private readonly object _syncRoot = new object ();
    private readonly List<ISubscriberCallback> _subscribers = 
        new List<ISubscriberCallback> ();

    // inject our dependency on operation context! this
    // decoupling actually makes sense - if we swap 
    // transport stacks we just require an implementation
    // that returns our requested callbacks!
    public SubscriptionService (IOperationContext context)
    {
        _context = context;
    }

    public void Subscribe ()
    {
        // ... ahhhh, that's much better!
        ISubscriberCallback subscriber = 
            _context.
            GetCallbackChannel<ISubscriberCallback> ();

        lock (_syncRoot)
        {
            if (!_subscribers.Contains (subscriber))
            {
                _subscribers.Add (subscriber);
            }
        }
    }

}

Now, if we were so inclined to test Subscribe above,

// test business logic without WCF! yay!
[TestMethod]
public void Test_Subscribe ()
{
    IOperationContext mockContext = null;
    // instantiate mock with expectations
    SubscriptionService service = new SubscriptionService (mockContext);
    service.Subscribe ();
    // verify results
}

Concurrency testing, viva CHESS!

A quick blip on the radar! I was just recently made aware of a (conceptually) awesome framework for exercising concurrent code. I have yet to employ CHESS myself but I am excited by its many promises.

Testing, types, kinds, and flavours

Building on anatomy of a test, I would like to follow up with a few points of note. These points are not based on any academic theory, principle, or practice. Instead, they are based mostly on anecdotal experience - so do not take this as "law", but rather a practical framework for communicating expectations about "testing" in general.

Preface aside, these points may be separated into three independent aspects,

• Types,
• Kinds, and
• Environments,

Types

There are three main types of tests; unit tests, integration tests, and system tests. What differentiates one type of test from another is scope.

A unit test evaluates functional behaviour of a single component - all dependencies are stubbed or appropriately mocked. An integration test evaluates functional integration of two or more components - dependencies outside of a chosen scope are stubbed or mocked. A system test is a special case of an integration test, it evaluates functional behaviour of all components - no dependency is stubbed or mocked.

Kinds

Every test is carried out in one of two fashions, a test is either manual or automated. Admittedly, such a small set hardly warrants discussion, but it is important to recognize this distinction. Even if, as developers, we deal primarily in automated tests it may sometimes fall to us to provide a manual test plan for person-in-seat testers.

In terms of resources required, execution times, and result variance, manual tests may differ significantly from an automated equivalent. As such, it is important to understand these differences, and communicate appropriate expectations regarding the work required by that kind of test.

Environments

Last and not least is the environment in which a test executes. The shape of data used as input may impact veracity and performance* of a test, which is why it is important to implement and maintain different testing stages thoughout a development cycle.

The point and purpose of each stage is twofold. First, each distinct stage we implement should progress from development-ready quality to production-ready quality. Second, each stage isolates itself from the other - and we do not promote code if it does not pass muster.

As an example, we may have several development environments (one for each developer!), a single build environment, a Quality Assurance (QA) environment, a System Integration Test (SIT) environment, a User Acceptance Test (UAT) environment, and a Preproduction Test (PreProd or PPT) environment.

This is certainly not exhaustive, just some of the common stages I have seen in my work.

Perhaps we noticed that one environment is conspicuously absent? That would be Production (Prod). We do not typically test in Prod. In fact, never test in Prod. PreProd is an exact mirror, from hardware to data to security infrastructure. Furthermore, the only distinction between PreProd and Prod is that PreProd is not Prod. This speaks most importantly to the second point and purpose above, isolation. Heaven forbid anything goes wrong at this stage, but if it does then Prod is safe and we dodge the million dollar liability suit/bullet.

Further reading

Wikipedia on testing.




* = hm, some may be confused by my citing "performance" here. Generally speaking, performance testing is treated distinctly from functional testing and we do not mix the two. However, from a perspective of "testing", each sets expectations, invokes a process, and evaluates outcomes to expectations, and so performance is as valid a quality as a functional outcome in this context.