Accidental DSL
Something kind of funny happened on a project that I am currently wrapping up at work. We had been doing acceptance testing with fitnesse, and written many custom fixtures that were specific to the domain problem (DoFixture is great for this). Further, there were quite a few RowEntryFixtures, SubsetFixtures, and RowFixtures that provided quick and dirty access to the underlying DAOs for adds/updates/deletes that allowed for quick modification of data, and each of our tests tried to mimic a user story as much as possible with minimal technical details.
Anyhow, during our daily stand up meeting with our customer, they asked if we could put something together so they could quickly create scenarios for them to test manually, and interestingly enough this didn’t take much time as we were able to show them to use the existing fixtures to interact with the system to do whatever environment setup they needed for their tests.
I think this kind of earmarks a good acceptance testing harness… one that mimics a DSL so much that you can even use your test harness for other purposes to interact with the application. Even more important, when you have an acceptance test that reads like a DSL, it means it is more customer friendly and conveys information in a useful manner. When tests are clear, concise, and easily readable by domain experts, it is much more easier to tighten the feedback loop and make changes very very quickly.
The only drawback with a testing DSL is that the initial setup can sometimes be burdensome… it may not seem worthwhile, and it may even require a little devotion to get going in the beginning and require a little maintenance to be useful. One problem we have had with our new fangled group of fixtures and tests is that development was rather quick in the beginning, but then we reached a plateau where the tests became a bit fragile and required some extra hard work to maintain and improve. Another smell that arose was that our original DoFixture, which contained sentences and statements representative of the domain, soon grew to over 400 lines and felt like a God Object. 
However, continuous maintenance and perfection improves quality, as with anything. Test DAOs solely used in the fixtures were refactored out and replaced with the real ones from the system, with a dash of “in memory” DAOs to inject when desired. The DoFixture was broken up a little bit to seperate fixtures represetnative of the different aspects of the domain being tested. And overall, to improve testability sometimes some redesign of the system was required to allow for better inspection of the system under test, which lead to a more solid API.
The final reasult? Although a little dedication was required up front and in the middle stages, writing new tests is a piece of cake. One of our customers called yesterday complaining about a missing feature that was part of a story we recently finished, but seemed not to work. How come? The tests are green. So I worked with him, and he mentioned the product code he was using was ZZZ, which was something slightly different… our test was using product code CCC, which to my knowledge represented the product needed for the story. “Oh… CCC is the product code for that product, but so is ZZZ. ZZZ is a subset of CCC, but in the context of [proprietary domain] they are essentially the same thing.” In only about 10 seconds I was able to add the product code and verify that the expected behavior didn’t take place, then my partner and I quickly fixed, ran tests, and released.
Writing new tests with a test DSL in place is so much faster, it felt like we were writing many tests quickly in later iterations after getting past the plateau of broken or inconsistent fixtures. Perhaps the only problem we have right now is our customers don’t run the tests… they just like to look at them to see what test coverage we have, which has been good because they can see at a glance what we are doing and often make corrections and comments in daily stand ups. They call our acceptance tests our “Test Plan”. 
I’ll try to write more on Testing DSLs… I keep thinking about other possibilities outside of FIT/Fitnesse, perhaps with tools like JBehave or even with some of the languages for creating DSLs like ANTLR or OGNL.
If you're new here, you may want to subscribe to my RSS feed. Thanks for visiting!
