Tweet This Post

For the past three years I’ve been trying to emphasize and build unit tests at Mint.com.  I’ve tried TDD and some other approaches but nothing has kept up pace with an agile-like speed of development.  The article  Is Unit Testing Doomed? talks about the issues that developers have today with all the different philosophies of unit testing, and why so many opt out of doing it.  Instead of forsaking unit testing because I couldn’t find the right methodology to suite my needs, I decided to come up with my own home grown version of it.

My approach is to design the entire project’s front and back-end’s set of interactions, then the data model to represent the back-end.  I then break the design apart into components that I plan on implementing.  After implementing each piece I write some unit tests for the apis I have developed.  I do this for each component or logical unit I am developing.  I use stubs to avoid class dependencies and focus on the class I am testing.

Since I’m operating in an agile-like environment the design is sure to change, in the event that it does I haven’t wasted a whole lot of time by writing test cases up front using TDD.  I can go in and quickly modify a few test cases that are impacted by the new design, and add additional test cases to handle the new design.  They key is that I’m not writing unit tests to fit my design or testing at the very end when I have very little to no time, and there is too much to test.  I’m writing test cases as I’m developing.  This has a couple benefits: first, its faster to spot bugs that new code might be causing in older code because I have test cases for my older code, hence I’m not breaking old functionality.  Second, when I unit test right after I’ve written a new component I’m forced to think about corner cases that might happen in the beginning of the development cycle rather than the end.  This also improves the overall design of my code and the feature.  If I have time prior to a release, I will add test cases as I fix bugs.  This is how I’ve incrementally built up a regressions suite.

I think developers should face the responsibility of unit testing their code, instead of offloading all testing to QA.  While you might not be able to think of 100% of the cases, flushing out simple error cases and thinking through a few corner cases at least handle some of the bugs that will come your way.  Unit testing is not a substitute for integration testing, but given the time for system setup and tear down unit testing at least keeps pace with daily development.  You can focus on integration testing once you’ve got the feature set completed, or also do it as part of your unit testing.  The point is to consider the philosophies of testing, and try to adapt them to meet your needs.  If writing test cases is the primary battle, focus more on meeting your teams developmental needs and less on being dogmatic about the current testing trends.

Tweet This Post

It took me only a few hours to write the test cases I needed using stubs, and with and IDE it’s even easier. But I don’t think its a fair assessment to compare using Mocks to Stubs because each tool is used for a different purpose (behavioral vs. functional).

However, I did notice one very obvious disadvantage to using stubs; if your application is dynamic and the functionality is changing test cases become brittle quickly, and require you to add additional members or functions.

But I would still advocate using stubs in writing test cases, because of the simplicity involved in writing them, being able to abstract away or limit dependent classes.

Tweet This Post

Recently at Mint I noticed I was getting bombarded with bugs everyday I walked into the office. And they were all for features that I had written over 6 months ago! But I hadn’t bothered to test them out, because I was being assigned new feature requests everyday, and writing code for new features was much more glamorous to writing test cases for old features. So one evening I decided enough is enough! I was tired of sitting in my cube every week and devoting more than half a day to fixing and debugging bugs! What was my solution? Unit-testing using stubs. I took each key feature that I was responsible for a created a test suite for them. For each suite I outlined the basic unit test cases, corner cases, and test cases for bugs I had experienced. My next dilemma was how to write the unit test cases without needing a Spring Application Context, and without needing to generate test data or use a database… enter Stubs.

When you have a code base as large as Mint’s you are bound to run into some interdependencies even when unit testing. But the principle of unit testing is to focus on one or a limited number of classes or key functionality. The best way to do this is to use a stub. A stub acts as a substitute for another class. The easiest way to implement it is to extend a parent class or implement an interface, and then provide a basic implementation of the class by specifying simple return values for the methods (i.e. there is hardly any logic in the methods). You then replace the class-under-test’s dependency with a stub class. 

For example, all my Alert classes had a dependence to an AlertDao class (the AlertDao class is the class that stores data base objects). Since I didn’t want to my unit test cases to depend on a data base I abstracted away the AlertDao Class using an AlertDaoStub. The AlertDaoStub contained the same methods as the AlertDao, but they methods had very simple return values. In place of using an AlertDao I used the AlertDaoStub; this change was transparent to the Alert class.

e.g. Implementation of the Stub class

// class-under-test
AlertManager am = new AlertManager();
// stub class that extends a parent class
AlertDao ad = new AlertDaoStub();
// replace AlertDao with AlertDaoStub
am.setAlertDao(ad);

public class AlertManager {
    private AlertDao _myAlertDao;

    public void setAlertDao(AlertDao ad) {
        this._myAlertDao = ad;
    }
}

// This can either be a public class or a private inner class depending on how much it needs to

// be re-used

public class AlertDaoStub extends AlertDao {

    public Set<Alert> getAlerts(User u) {
        return new HashSet<Alert>();
    } 

    public void setAlerts(User u, Set<Alert> alerts) {
        user.setAlerts(alerts); 
    }
}

While this technique required some refactoring, and one long night of coding, I believe in the long run I will have contributed to the quality of the Mint code base, and saved myself countless hours of debugging.

Tweet This Post

1. You must have a decent understand of the use case you are testing; you must know which methods and classes are pivotal to the test case, and which can be abstracted away using expectations.

2. Using expectations can be a time-saver because unlike stubs you can just specify what the return value of the method should be, how many times it should be called, and what if any parameters are expected.  This limits the amount of test code you need to write.  However, the class methods need to be fairly deterministic, meaning they must always return the same result given a certain set of inputs.  

3. Mocks are perfect when it comes to testing classes with limited functionality or that are isolated from other classes.  Interdependencies between classes makes using Mocks a nuisance.  If the given class you are testing has a lot of dependencies amongst other classes then it requires you to create Mocks for those classes as well.  This can bloat your test code, and take a longer time to test, because now you must program for the dependent classes and setup their expected behavior as well.

Tweet This Post

Lately, we’ve noticed that our test cases are brittle, and have poor coverage mostly because of feature growth. My boss proposed that we seek a new approach to how we write and manage our unit test cases that is scalable and easy to practice under time constraints. So today, during one of the most productive and interested meetings I’ve ever had in my career, we discussed the trade-offs between using Stubs versus Mock Objects for unit testing. Our discussion was directed by the following requirements:

1. Little to no use of a database

2. Test logic at the controller level (servlet/ JSP) and at the service level.

3. Simple template that can be re-used to write additional test cases and doesn’t require complete knowledge of back-end architecture.

As a homework assignment for the meeting I read Martin Fowler’s “Mocks Aren’t Stubs”; Mackinnon, Freeman, and Craig’s “Endo Testing: Unit Testing With Mock Objects”; and a few sections from the book “Test Drive Development”. I thought each of these resources presented interesting insights into testing in general.

Fowler’s article introduced me to behavioral testing, which is the concept that in an application engineers should test the interaction amongst classes. Contrast this to functional verification, which is inputing data and then verifying the end results for accuracy.

This is the first time I had been presented with the concept of behavioral testing, and I realize it makes a lot of sense for a scalable web application, because the order and sequence of method invocations is just as important as data accuracy. Running code off of various servers, and across different layers has a huge impact on the data, so its just as important to test the interaction as it is to verify the data values.

Mock objects are the means by which engineers can do behavioral testing, because Mocks are programmed using expectations. Expectations are either a list of methods to call sometimes in a particular sequence, and/or values that a method may return. When a test case is run it simulates the behavior in a system using the Mock objects in place of POJOs, and verifies that the expectations were met.

Thus, Mocks do not contain all the functionality or attributes of a POJO, but serve in place of them by containing enough knowledge in the form of abstractions to test the interactions of the application.

Tweet This Post