Archive for December 2009
Earlier I spoke about specifications at the story level, now it’s time to explore them at the unit level. Story level examples explore how the feature in it’s entirety works, demanding code to be produced to make the example work… at the unit level, we apply the same concept, we provide examples that prove the need for code in our class to exist and then write code to make that example work.
Examples, or Exemplars
The basis for this are Exemplars, which we saw earlier in story level specifications, but just to recap they are somewhat equivalent to scenarios consisting of the structure
Given (Some Context/Precondition)
When (Event Happens)
Then (Outcome)
You might recognize this structure from an existing TDD practice of “Arrange, Act, Assert.” Hypothetically I like to think of this as the same format and we should aim to keep each three phases of the exemplar as small as possible. In JUnit I might write an exemplar somewhat like this (using an example from the BDD wikipedia page):
@Test
public void shouldNotBeEmptyWhenListHasMoreThanOneItem(){
List<String> list = new ArrayList<String>();
list.add("foo");
assertFalse(list.isEmpty());
}
In the above example, the given/when/then would be
Given a new list
When an item is added
Then the list should not be empty
From there I’ll create more examples of how the API should work, refining and adding examples in an attempt to “break the model”, that is attempt to define examples that break my assumptions on how the API should work and even provides better insight on what roles/collaborations my object might have. Now this doesn’t mean wasting time by passing null to every method call or writing fifty exhaustive examples with every possible input known to man, but rather finding examples that could prove that my domain logic is incorrect… by doing so, I’m able to reflect on my current API design and refine my model.
The main process at work here that ties this together is something I first saw at a Real Options based presention (as well as in the comic I mentioned in my earlier post): David Kolb’s learning styles model which I think accurately reflects the development cycle that one often uses when applying BDD style practices. To break the process down into Kolb’s Cycle:
- Concrete Experience: seek challenge and immediate experience - I like to think of this as coming up with the first example and writing code to make that example work
- Reviewing the Experience: ‘stand back’, gather data, ponder and analyse – We wrote an example with passing code, what have we learned from this experience? I also like to think this is the point where we might refactor, not only to clean code up, but to refactor toward deeper insight.
- Concluding from the Experience: think things through in logical steps, assimilate disparate facts into coherent theories - we’ve analyzed the example and solution, refactored towards deeper insight, now we have several ideas of where to go next
- Planning the next steps: seek and try out new ideas - this connects us back to step one, we try out successive examples and new examples we’ve come up with, and we’re not shy to try an example out and discard it if it’s incorrect
I know that sounds like quite a bit of work, but I often find that this process occurs naturally and can be pretty rapid, building each example off of the previous examples, seeing patterns and refactoring, and even eliminating bogus examples. Especially if you’re pair programming it helps if you are doing “ping pong” style pair programming: writing the first example, your peer writes just enough code to make the example work, then uses his concrete experience to write the next example for you to write code to work.
So What’s the Real Point?
I think the biggest point to convey here is what the difference is between conventional “unit testing” and a BDD mindset… in the earlier, you might be concerned with testing your code but what you should do is be focusing on providing examples of how your code works and using those examples to prove the need for your code. That mindset coupled with writing examples in the Given/When/Then format keeping them simple and to the point helps you drive the design of your API and even provide some developer readable documentation on how the API works.
Next?
Again, I’m only barely scratching the surface… there’s a lot more to talk about, and a lot more to delve into in how best practices in TDD are present in BDD. I haven’t even covered any BDD specific spec frameworks, opting to provide examples only in JUnit! There’s also more to discuss on Contexts, Test Doubles, etc. Stay tuned!
No tags
Recently I gave a presentation on Test Doubles at a client site and while fielding questions at the end, one person asked how they could use Mockito (which I used to demonstrate Test Doubles) when autowiring fields with Spring. The problem he had was he was autowiring the fields, which were private, and which had no setters to use for injection. Although he could use reflection and cheat, I stated what I believe to be philosophically correct; dependencies should be expressed in your implementation (I often don’t express them in interfaces unless they are an argument based dependency) and as such you should always provide an injection method for them (constructor or setter) to realize that dependency to those who might use the concrete implementation.
So, rather than doing this:
public class SomeServiceImpl {
@Autowired
private Dao dao;
}
I’d prefer to do this:
public class SomeServiceImpl {
private Dao dao;
@Autowired
public void setDao(Dao dao){
this.dao = dao;
}
}
Although one can argue that the setter is only used for substitution in tests/examples and there’s no client code at all that will inject the dependency besides Spring, I’d argue that your tests/examples are clients themselves… I’d also argue it’s just plain good design to express the external dependencies that your class requires. 
No tags
-
Harness the power of Niko Niko charts for retrospectives through twitter.
No tags
22
Code Doesn’t Exist Unless It’s Checked In
3 Comments · Posted by James Carr in Uncategorized
Being back in a solo development environment after many years in an agile environment with required pair programming can be quite a change. I’ve been quite used to pair programming and influencing my partner to check in on each passing example. In a solo environment this can be a little trickier, and even I have succumbed a few times to letting the sun set on un-checked in code.
And being in a solo development environment has really made this easy to let slide… I found myself deleting code and starting over a few times in the past month because I let myself do work that I didn’t check in before the end of the day… mostly because the old adage is true… when you let yourself do work and not check it in for multiple days, the code in version control will tend to drift in a way that makes check-ins difficult and leaves you feeling like you’re not getting stuff done because you have uncommitted code on your machine.
So it all still holds true for me… code just does not exist at all if you haven’t checked it in yet. 
No tags
22
On BDD: From Acceptance Tests to Story Level Specifications
4 Comments · Posted by James Carr in Uncategorized
Behavior Driven Development is best described as taking an “Outside-In” approach, meaning you define your feature from the outset and describe business value, and work your way in a small piece at a time. That being the case, I thought it’d be best to cover one stage of evolution that I’ve had over the years… from Developer Facing Acceptance Tests to Customer Facing Story Level Specifications.
The Start: Developer Facing Acceptance Tests
These were the starting point… where I started at 5 years ago. I would write rather long winded integration style tests in an xUnit framework that would exercise the end to end functionality of a feature. It worked because it provided an automated test that would verify that the feature actually worked, and provided a nice safety net when doing wide ranging system re-structuring.
These also would at times consist of “data driven testing,” that is, it’d exercise the system with different input data to verify different outputs and behavior. There would often be quite a bit of setup and tear down of inserting/removing data. Setting up test records would often create a lot of noise.
The problem here is that at the time the stakeholders were not involved in the process other than to hand off requirements and be around for the devs to “hand off” completed work to QA. The “acceptance tests” could barely even be called that… as far as anyone was concerned, they were for “developer eyes only” and usually consisted of very very large test methods that, to be honest to God were very painful to read and quite brittle. Truth be told, this didn’t work well and even the smallest of these so called acceptance tests barely conveyed any domain knowledge whatsoever. Fail. 
A Little Better: Developers Writing Acceptance Tests In FIT
From this point I arrived at an organization that had started using fitnesse for acceptance tests. This was quite a bit better than writing mammoth JUnit Tests with lots of test data setup as one could now just create a test with a great big table of input data, run through the full system, and verify the outputs. The nice thing here is one could write acceptance tests for each feature and re-use a few ColumnFixtures and RowFixtures to setup and verify the system.
The problem however is that when acceptance testing is approached from a developer point of view, again much domain knowledge is lost through clever re-use and treating the tests written in fitnesse as “test scripts.” It wasn’t uncommon to see pages of fitnesse tests filled to the brim with xpath expressions or the like. I felt a little disheartened… reading the FIT book I got the impression it was a tool to aid stakeholder and developer communication, but even the QA were mortified at the sight of the fitnesse tests the developers wrote.
To help ease things a bit, we tried to take a “QA matrix” of a huge spreadhseet of real records found in the database and whip up a fixture to parse and execute it. We had an “executable QA Matrix” but there was still a lot to be desired… looking back at one of those fitnesse tests a year later would definitely leave one scratching their head in confusion as to what domain logic was being verified.
I recall at onetime attending a presentation by a team that had bragged about full customer involvement that had their primary stakeholder actually writing fitnesse tests for them and was quite intrigued… this was EXACTLY the kind of customer/developer interaction I had been seeking, although there was a disclaimer that their stakeholder had some technical knowledge. When I looked at the fitnesse tests, they were filled with JDBCFixtures running raw SQL queries and using xpath queries in HtmlFixture to verify what should be displayed in the UI. Far from the nirvana I was seeking.
Now let me say that FIT isn’t a bad tool… in fact, I’d say it’s a rather good Story Level Specification BDD framework! We were just using it wrong, and I think 70% of FIT users misuse it too. At this stage I think we managed to accomplish all five ways to misuse FIT that James Shore once blogged about. He put it exactly the way it was,
If you aren’t using Fit for communication and collaboration, you’re just writing automated tests. In HTML! With tables!
We would desperately try to make our tests more readable… DoFixture worked quite well in this regard by allowing us to map actual sentences to fixtures to perform work and left us with tests that were very readable and easy to understand, at times we were even able to pull them up during a meeting and show them to our stakeholders and they would correct us on some of the logic or agree that it accurately describes how the feature should work.
We were close, but a lot was left to be desired. For the most part, these were still automated test that, although readable, were still developer centric and barely decipherable by the stakeholders. Some attempts might sometimes be made to make so that an actual requirements document can be copied in (text, tables, and all) and made executable, but I think this really tries to cover too much at once, misses a lot, and encourages stakeholders to continue churning out requirement documents to developers instead of engaging in hands on collaboration in user story writing sessions.
What We Were Learning
Although what I have described thus far isn’t remotely close to an ideal working environment, these experience had taught us several key points:
- Collaboration is IMPORTANT! We need to keep the stakeholder’s in the loop as much as possible
- Plain text works well in expressing features
- We should treat our pages in fitnesse as an executable wiki, a kind of living documentation
- Acceptance Tests in FIT typically involve setting up some kind environment for the scenario, an execution of the System Under Test, and a verification of the behavior
- They should do whatever they can to describe the domain
At this time period, we had finally gotten our stakeholders to the point where some were quite comfortable with writing user stories and would often copy the user story into a new Fitnesse page, and take conditions of satisfaction off of the card and transform each of them into scenarios. They were usually a bullet point describing some simple rule (something like “If it’s older than 10 days, don’t display it”) and would usually be expressed pretty nicely in the fitnesse test.
With this environment in place, it was quite a bit easier to use our acceptance tests to communicate better and clearer with our stakeholders… sometimes they would even dive in and write a few scenarios here and there for us. What struck us here is that the most effective approach is to have user stories that describe a feature well and provides some very concrete conditions of how that feature should behave.
Bridging the Gap: Behavior Driven Development
Around this time is when I discovered Dan North’s post What’s In A Story, which outlined two notable concepts: Feature Injection and scenarios expressed as Given, When, Then. I’ll touch more on Feature Injection later, but what I really want to focus on here is the scenario format. Specifically, scenarios fall into the format:
Given (some precondition or context)
When (an event occurs)
Then (outcome)
Having customers start writing their conditions of satisfaction in this format made them much more concise… and often we’d find that we’d build on top of them after thinking about each one. In BDD, we call these Scenarios, but we also call them Application Examples. Which is what they are… the customer is describing an example of how the application should behave in different instances for this feature.
So… how does this fit into acceptance tests? Well, the moral is you have to stop thinking of them as “tests” … imho such thinking is what helps teams wind up with some monster test that generates data for the 500 possible scenarios, execute for two hours, then verify the results. We’re not testing… we’re describing the examples of how the system behaves for that particular feature in an executable way… reaching green isn’t so much of passing a test but rather an indication of feature completeness.
How Do You Do It?
Any framework can meet the needs of working with your customer to specify application examples at the story level… it really just depends on what tool you and your customer feel comfortable with. Remember it’s not so much the framework, but rather the philosophy… you and your customer are not writing tests, but application examples. I just really want to make sure I drill that point home.
That being said, there’s quite a few. Fitnesse/Slim is pretty good… especially when coupled with the plain text FitLibrary styles and GivWenZen fixture.I’ve also tried concordion out, which is nice if you have your customer use some html editor (or if they are content with html). Cucumber and JBehave are equally nice and I like them because they use just plain text… you can even use a tool like Pickler to sync stories from Pivotal Tracker to execute in cucumber. There’s quite a bit more, the moral is do your own exploring and stick to whatever tool you and your stakeholders like the most.
The next question I often hear is at what level is appropriate for writing story level specifications? Should they run through the GUI? Although some BDD proponents do believe you should run them through the GUI since that is the top most level, I disagree. From experience, I find that examples ran through the GUI lose a lot of domain terms and they are quickly replace with terms like “click, enter text, sees on page” … sure, it does describe the application’s visible behavior, but in my opinion causes examples to miss the point of how the domain works. Additionally, I find that the GUI is often quite volatile and slow, which may work as an obstacle to writing as many application examples as are needed.
I usually wire my application examples through the underlying mechanism… either through the controller type object that might be in place or even through the direct services that the controller might use… essentially, the fixture becomes a stand-in for the controller or point of entry the application might normally have. I might also substitute external dependencies as well to provide speed, but again this is entirely up to you.
I like to start with an Application Example or two in place, then work at getting each one to pass, writing unit level specifications for each object I need, and using test doubles for each component I haven’t built yet, iteratively replacing each stand in for the real thing. I’d like to hear the approaches others take, so let me know.
Scratching the Surface
I’ve only scratched the surface with this post… and a lot of the topics I briefly touched on can be delved into at a much deeper level. Topics like Feature Injection and how we can use it and Application Examples to help us out in analysis and gain a deeper understanding of the domain process. I’ll try to dig deeper in future posts, but I think Chris Matts‘ comic on the subject covers it quite well.
No tags
Okay, let me get something straight… for the past several MONTHS I’ve been trying as much as possible to distill my thoughts on Behavior Driven Development and each time I tried I wound up with a train wreck of a blog post that I would promptly delete. While commuting to work today I finally realized why I kept having so much trouble: I was trying to cover too much in one post.
Therefore this is simply going to serve as an introductory post for several small posts that describe my philosophy around different aspects of BDD and most of my practices that grew naturally through several years of experience developing software test first. A lot of emphasis will be on switching focus from “Test Everything That Can Possibly Break” to “Describe How This System Should Work.”
In the days and weeks to come, you’ll definitely see a variety of posts detailing my experiences that led me to a BDD mindset and I hope to use each post to describe each aspect of BDD in high detail. I’ll update this post with links to each successive post for future reference.
Cheers!
No tags
No tags
No tags
