I’ll be giving a presentation at Lambda Lounge this Thursday on Behavior Driven Development With Jspec. If you’re in the St.Louis area come on by and learn about BDD and how you can use it to drive your design. I’ll be using jspec to demonstrate how to build a functioning feature for a javascript library driven by small, iterative examples.
Although the cookbook includes an example of using the easyb ant task to run specs and produce reports, I thought I’d try my hand at writing a task to manually run easyb specifications from the commandline. Here’s the result:
task spec <<{
ant.java(classname:'org.easyb.BehaviorRunner', fork:false,
classpath: "${sourceSets.test.runtimeClasspath.asPath}") {
sourceSets.test.java.srcDirs.each { testDir ->
testDir.eachDirRecurse { dir ->
dir.eachFileMatch(~/.*\.specification/){spec->
arg(value:spec.absolutePath)
}
}
}
}
}
Now I can run “gradle spec” from the commandline to write all my easyb specs. Spock is next.
I’ll admit I’ve just recently gotten back into groovy, so I’m still kind of a noob. I keep feeling like groovy has a better way to search for file patterns recursively. 
Over the past four years I have at various times over and over explained to my co-developers why writing tests against setters and getters is bad, dealt with the whole “metrics” argument, and quite frankly I start to get weary repeating myself every three months.
While I was driving into work today, I started thinking about the subject again as I’ve been debating a topic yet again recently that I quite honestly don’t think it warrants any debate and decided that since I haven’t written any blog posts on the subject, I should. So ahead I will describe why test driving setters and getters isn’t only a waste of time, but it is also considered harmful to your design as well!
First off, I’ll point out why I find it a waste of time. I’ve seen developers labor furiously in the past to verify that a behavior that at the simplest is already done for them by their IDE. If your IDE has generated them for you, and you decide to write tests against it, you’re first of all bullshitting yourself that you’re practicing TDD, because you’re really writing tests after the fact. Secondly, what the frak are you testing? That the developers of intelliJ/eclipse/textmate/whatever wrote their setter/getter generation code correctly?
Another myth is that, while the above argument is correct, the setter/getter “might” have some behavior in it, or if you don’t write a test against it and someone goes in and modifies the behavior, there will be no “tests” to catch it. Now, I can’t say for you, but I don’t write tests to “catch bugs”, I do it both to prevent them and to ensure my design is sound. Secondly, I also adhere to concepts like YAGNI (You Ain’t Gonna Need It) and creating “The Simplest Thing That Can Possibly Work.” If I’m going to change the behavior of my setters or getters, I’ll write a test for that behavior before I change it. If you’re trying to prevent developers from being sneaky, well then you have team issues that you need to address, and I don’t think the best way to deal with those team issues is by decreeing that all setters and getters need to be tested.
These are all perfectly good reasons not to test setters and getters in my humble opinion, but now let’s get to the real reason you shouldn’t do it, and why I even considered it harmful to your design. See, when we practice Test Driven Development, there really shouldn’t be so much of an emphasis on “test” but rather on “design” (this is why I really lean toward using the term Behavior Driven Development). I’ve said this before, but let me recap.
We use tests to drive our design. Each test provides an example of functionality that we use to prove the need for the code we’re going to write to exist. With a failing test in place, we then write the code that we need to make that test pass, then on to the next. Using this iterative approach, we allow our system to take part in emergent design, proving we really do need the code we’ve written. If we wrote code we didn’t write an example for, there’s the possibility that we really don’t need that feature (or we do, we just thought too quickly and didn’t write an example). Setters and Getters should get exercised while executing these examples.
Wait, back up. “Setters and Getters should be exercised while executing these examples.” What I mean by this is that when you write a test against a feature that, let’s say returns an object, you should be doing a verification on that return value’s field to see if a value was set to the expected value, in which case in the course of that test the setter and getter get executed. The example for that specific feature proves that you need that field to hold some value.
A simple example:
@Test
public void shouldAddDiscountIfTenOrMoreItems(){
Order order = new Order();
order.setItems(add(10).items().thatCost(ITEM_PRICE));
calculator.calculateDiscount(order);
assertEquals((ITEM_PRICE*10)*0.20, order.getTotal();
}
Now in this example, we exercise various setters and getters in Order, because we need them. That’s right, because we need them… our feature that our business cares about needs them in order to work.
Now we lead ourselves to why TDD’ing setters/getters is harmful… by writing tests against setters and getters, we’re saying that we need these properties on an object, but we’re not explaining why. This is akin by starting a project by hitting the ground running and designing the database without no real understanding of how the data will be used. At the least you’ll possibly be defining objects and properties you might night use… at worst, you’ll create a design of objects that aren’t grouped within the domain correctly, which will show itself overtime when you have domain objects and services that, although they work, don’t make a lot of sense.
This is a good middle ground for people concerned with metrics. Although I think that high code coverage is a side effect and not the end goal, if you follow this practice you should achieve optimal code coverage without succumbing to wasting your time testing field names.
UPDATE: Pat Maddox posted a link on the TDD mailing list to an old (but rather insightful) article on why getter and setter methods are evil. I’ll have to say I agree with this quite a bit!
Earlier I spoke about specifications at the story level, now it’s time to explore them at the unit level. Story level examples explore how the feature in it’s entirety works, demanding code to be produced to make the example work… at the unit level, we apply the same concept, we provide examples that prove the need for code in our class to exist and then write code to make that example work.
The basis for this are Exemplars, which we saw earlier in story level specifications, but just to recap they are somewhat equivalent to scenarios consisting of the structure
Given (Some Context/Precondition)
When (Event Happens)
Then (Outcome)
You might recognize this structure from an existing TDD practice of “Arrange, Act, Assert.” Hypothetically I like to think of this as the same format and we should aim to keep each three phases of the exemplar as small as possible. In JUnit I might write an exemplar somewhat like this (using an example from the BDD wikipedia page):
@Test
public void shouldNotBeEmptyWhenListHasMoreThanOneItem(){
List<String> list = new ArrayList<String>();
list.add("foo");
assertFalse(list.isEmpty());
}
In the above example, the given/when/then would be
Given a new list
When an item is added
Then the list should not be empty
From there I’ll create more examples of how the API should work, refining and adding examples in an attempt to “break the model”, that is attempt to define examples that break my assumptions on how the API should work and even provides better insight on what roles/collaborations my object might have. Now this doesn’t mean wasting time by passing null to every method call or writing fifty exhaustive examples with every possible input known to man, but rather finding examples that could prove that my domain logic is incorrect… by doing so, I’m able to reflect on my current API design and refine my model.
The main process at work here that ties this together is something I first saw at a Real Options based presention (as well as in the comic I mentioned in my earlier post): David Kolb’s learning styles model which I think accurately reflects the development cycle that one often uses when applying BDD style practices. To break the process down into Kolb’s Cycle:
I know that sounds like quite a bit of work, but I often find that this process occurs naturally and can be pretty rapid, building each example off of the previous examples, seeing patterns and refactoring, and even eliminating bogus examples. Especially if you’re pair programming it helps if you are doing “ping pong” style pair programming: writing the first example, your peer writes just enough code to make the example work, then uses his concrete experience to write the next example for you to write code to work.
I think the biggest point to convey here is what the difference is between conventional “unit testing” and a BDD mindset… in the earlier, you might be concerned with testing your code but what you should do is be focusing on providing examples of how your code works and using those examples to prove the need for your code. That mindset coupled with writing examples in the Given/When/Then format keeping them simple and to the point helps you drive the design of your API and even provide some developer readable documentation on how the API works.
Again, I’m only barely scratching the surface… there’s a lot more to talk about, and a lot more to delve into in how best practices in TDD are present in BDD. I haven’t even covered any BDD specific spec frameworks, opting to provide examples only in JUnit! There’s also more to discuss on Contexts, Test Doubles, etc. Stay tuned!
For those of you who didn’t know, I’m currently contracted at a workplace that is still stuck using java 1.4 (I heard they might ring in 2010 by upgrading, but we’ll see). This has create a number of difficulties, chief amongst them (in my view) that I can’t use Mockito and most people use EasMock… version 1.2 (yeah, the one with that MockControl nonsense).
Not to be foiled, I used retrotranslator to translate mockito to 1.4 so I can use it for TestCases… it is now available on the Mockito project site.
This is pretty much a direct port, and I added the class MockitoTestCase which basically has all the methods you’d normally statically import off of the Mockito base class. Here’s a quick demo of usage:
public class CustomerServiceTest extends MockitoTestCase{
private CustomerDao customerDao;
private CustomerService customerService;
public void setUp(){
customerDao = (CustomerDao) mock(CustomerDao.class);
customerService = new CustomerService();
customerService.setCustomerDao(customerDao);
}
public void testShouldReturnTheNumberOfActiveAccounts() throws DataFormatException{
List customers = Arrays.asList(new Customer[]{new Customer(3), new Customer(4)});
when(customerDao.getActiveCustomers()).thenReturn(customers);
assertEquals(7, customerService.getTotalActiveAccounts());
}
public void testShouldSaveCustomerIfHeHasAccounts(){
Customer customer = (Customer) mock(Customer.class);
when(customer.getNumberOfAccounts()).thenReturn(new Integer(1));
customerService.saveCustomerWithAccounts(customer);
((CustomerDao) verify(customerDao)).save(customer);
}
}
Good Luck!
Continuing a trend of Mockito related articles, I thought I’d blog about using another feature I recently discovered while trying to figure out how to implement fine grained behavior in mockito.
First, let me set the stage… I had a method call I was writing examples for that was a typical Castor marshal operation, which often looks something like this:
StringWriter w = new StringWriter(); marshaller.setWriter(w); marshaller.marshal(obj); channel.send(w.toString());
Yeah I know… it looks like shit. But sometimes APIs force that on you. Anyway, I wanted to write an example illustrating that the channel received a message that was the result of whatever marshaller wrote to the output stream, but this was a little difficult to do (but easy to do with hard coded subclasses and lots and lots of boilerplate code).
The answer comes with, well, the Answer interface. To illustrate this, let me set up an example. Given I have started a unit test that looks like this:
@RunWith(MockitoJUnitRunner.class)
public class MessageServiceTest {
@Mock private Marshaller marshaller;
@Mock private Channel channel;
private MessageServiceImpl messagService;
@Before
public void before(){
messagService = new MessageServiceImpl();
messagService.setChannel(channel);
messagService.setMarshaller(marshaller);
}
@Test
public void shouldWriteMarshalledDataOutToChannel(){
}
}
We want to be able to verify that the channel (which is a simple interface with a send(String arg) method) receives whatever String the marshaller wrote to the provided Writer. Now, because the writer is constructed in the class itself, it’s difficult to stub this out. So in order to make the writer have SOMETHING written to it, we set up an answer to be invoked both when marshaller.setWriter(..) and when the marshaller.marshall(..) method is called:
private class MarshalAnswer{
private Writer writer;
public Answer<Writer> respondToSetWriter(){
return new Answer<Writer>() {
public Writer answer(InvocationOnMock invocation) throws Throwable {
Writer w = (Writer) invocation.getArguments()[0];
writer = w;
return null;
}
};
}
public Answer<Object> respondToMarshall(final String writeThis){
return new Answer<Object>(){
public Object answer(InvocationOnMock invocation)
throws Throwable {
writer.write(writeThis);
return null;
}
};
}
}
Bleh… lots and lots of boilerplate code because of the interfaces, but oh well. Here’s the revised test case in action:
@RunWith(MockitoJUnitRunner.class)
public class MessageServiceTest {
@Mock private Marshaller marshaller;
@Mock private Channel channel;
private MessageServiceImpl messageService;
private Invoice invoice;
private MarshalAnswer answers;
@Before
public void before(){
messageService = new MessageServiceImpl();
messageService.setChannel(channel);
messageService.setMarshaller(marshaller);
invoice = new Invoice();
answers = new MarshalAnswer();
}
@Test
public void shouldWriteMarshalledDataOutToChannel() throws Exception{
doAnswer(answers.respondToSetWriter()).when(marshaller).setWriter(any(Writer.class));
doAnswer(answers.respondToMarshall("<someresult/>")).when(marshaller).marshal(invoice);
messageService.send(invoice);
verify(channel).send("<someresult/>");
}
...
I could have just created the respondTo methods in the TestCase itself, but I wanted to keep it separated by putting it in a helper class.
So now we run the test and we should see the following:
Fair enough… lets add the following code to the method we’re describing and rerun:
public void send(Invoice invoice) throws Exception{
StringWriter writer = new StringWriter();
marshaller.setWriter(writer);
marshaller.marshal(invoice);
channel.send(writer.getBuffer().toString());
}
We rerun and sure enough, green bar! The way it works is pretty simple… whenever setWriter gets invoked, the Answer object gets called and the argument is available within it (in this case the writer) which we can then store to use for the marshal call. When marshal gets called, our Answer we defined gets called and writes whatever we told it to.
While this does provide us with a useful tool when requiring some slightly complex behavior in our collaborator interactions (in this case, we essentially needed to capture what was passed to the writer, which was acting as a hidden output variable) I’d still recommend avoiding them when you can.
Yesterday during our morning huddle one of the developer’s mentioned that on the story he was working on he had achieved 97% code coverage and this got me thinking a bit about something that’s been on the tip of my mind for awhile. Now, I cannot speak for the developer at my huddle because I haven’t seen his test and I can assume that they’re well written, but I often have heard through my career reports of good code coverage and have seen instances where code coverage was used as a bad metric in the worst possible way.
Recently I was BDD’ing a class, and the behavior I needed to describe happened to be an object created internally (a simple value object) populated with a few attributes and passed to a collaborator. The old way I might be tempted to do this is just create a concrete implementation of the collaborator that stores the argument passed to it, then get it back and verify some outcome on it.
But I wondered if there was a way to do it with Mockito without it looking crappy. And apparently there is a way to verify values on an argument to a collaborator using ArgumentCaptor.
Subscribe in a reader