James Carr

Looks like I’ve already accomplished one of my new year’s resolutions by doing some polygot programming at work today embedding javascript within java. And it happened completely by accident.

Lately we’ve had a handful of Selenium Remote Control based fitnesse fixtures crop up at work… there’s a nice generic one, and several custom ones designed to tackle specific front end domain problems. On our latest project we have your run of the mill tabbed layout, you know… click on tabs and and you see different content, nothing special. What made things interesting, however, is that marketing wanted to track some specific usage information when each tab is clicked. Fine enough, just whip up some ajax and register some event handlers for the click event, and we’re ready to go. Well, hold on… what about testing?

Everything on the server side was unit tested and functionally tested to a T… in fact I am rather proud of the partner I worked with and myself on how well the tests are written and how many scenarios are covered. However, I still wanted to test the front end… I want to be sure that each time someone promotes, they didn’t do something to, as our team’s QA member would say, “jack it up.”

So I quickly wrote some selenium tests in fitnesse (note: this violates everything I believe FIT should be for, but fitneese is a convenient poor man’s application server) to get to the vehicle history report and test it. Speed bump number one. The interface that lets users run reports uses a target attribute on the form to open the report in a new window, and apparently selenium can’t focus on a window unless you explicitly use window.open to open the window. Drats… foiled. Or was I?

After a bit of searching, I found out that selenium remote control has a method called getEval that will simply evaluate any javascript you pass into it, which is handy for checking variable values for example. From there, you can access the runner via this.browserbot, which provides a whole slew of awesome commands you can use for manipulating the runner, including accessing the current window. And that’s where it sinks in… I can do anything in the world (almost) with javascript, and if I can access the window, I can make things happen. Bypassing the earlier speed bump was a simple matter of opening a new window with the same name as the target, and then focusing on it once the form is submitted. Piece of cake.

Since we have a lot of forms like this, I wrote a simple script to open new windows for any forms with a target on it using the name of the target, and wrapped in a java method. So that was step one… I now had a java method in a fitnesse fixture with javascript code inside a string indented all nicely… and evaluated on demand.

Now. at the new tabbed page is where things got immensely interesting… I wanted to intercept xmlHttpRequest calls, capture them, and be able to access them later. Redefining jQuery’s XHR methods were a piece of cake, the code was something like this (and whipped up in about 5 seconds):

var _d = this.browserbot._caughtValues = []
this.browserbot.getCurrentWindow().jQuery.post = function(a,b,c){
	_d[_d.length] = b;
}

the second parameter here was the data packet being passed in, which is what I was interested in the most. Different meta data is fired off on different tab clicks.

Then topping this off was a simple matter. I created a method that would return a ListFixture (because I wanted to loop over the meta-data packets in the order they were fired, and also because it is vastly supperior to RowFixture) and got to work scraping the values out, which became a mix of BOTH java and js, together.

Building a list of java objects from the captured values was a matter of:

1. Getting the length of the _caughtValues array and converting it to a java int (getEval only returns strings)
2. iterate a number of times equal to the length to visit each value on the array
3. do an eval against _caughtValues with an index specified to get the json packet:
   

   String packet = getSelenium().getEval("this.browserbot._caughtValues["+index+"]");
                  

4. parse it into a java.util.Map by making a call to a nice JSON parsing library
5. populate a java object representing the packet with the map, add it to a list, and return a new ListFixture containing that list.

I wrestled a little to get it work, but I’ll have to admit it felt oddly great when I saw the table of expected packets light up green… and then red for the last 4 rows (good to know it was able to catch the invalid packets). There’s also a sense of joy in seeing it run consistently… for a session it worked, but would randomly fail.

It’s pretty interesting, but I think my coworkers might hate me for the embedded javascript code inside of java. So far they either think it’s really cool, or really insane.

Via James Shore’s blog:

before pulling weeds
perhaps I should remember
what the plants look like

How very true. One common action I like to take when undertaking a refactoring is to comment out sections of the code I am about to refactor and run the unit test… if nothing breaks, proceed no further until you have better tests in place!

Uncle Bob recently made an interesting post on TDD and test generators, and I couldn’t agree more! I’ve always despised test generators myself, mostly for the same reasons Martin makes.

In my opinion, desiring to use a test generator on fresh code is an attempt to cut corners and bypass testing, which really is counterproductive. From my experience thus far, writing code test first has helped me write more loosely couple code, while both legacy code and code written test last usually end up being a tightly coupled mess. Even if you design nice, loosely coupled code and generate the tests for it, there is still the problem that the test generator is not really going to be smart enough to figure out your domain logic, and TDD is a helpful tool for expressing domain logic in the form of both test inputs/outputs and object behavior.

Like Bob mentions, it’s helpful to use generators to provide coverage for a legacy code base (especially when refactoring… I once spent a couple days writing tests around a legacy system to prepare it for refactoring!) but if you’re using them for a fresh code base, you’re of course not doing TDD… and you lose all the benefits you could reap from practicing TDD!

As an aside, I noticed he mentioned that 33% of the codebase for fitnesse was unit tests… I’d go further and argue that one should aim for 50-60% of their codebase being composed of unit tests! I’m just extreme like that.

InfoQ reports that JUnit 4.4 has been released. I have not had a chance to download it and try it out, but it looks like it has added a very good and useful feature, assertThat.

This is a very welcome addition. For a long time I have actually been using RMock in my TestCases solely for the assertThat method that it provides. For example, let’s say I want to assert that a string contains “foo” or “bar”, I can do this in RMock like this:

assertThat(result, is.containing("foobar").or( is.containing("bar") ));

Which is better than that assertTrue nonsense with a little bit of code jammed inside it to return true or false. JUnit4.4 now does something similar:

assertThat(result, either(containsString("foo")).or(containsString("bar")));

Cool stuff… I’ll have to start using this new version.

Something kind of funny happened on a project that I am currently wrapping up at work. We had been doing acceptance testing with fitnesse, and written many custom fixtures that were specific to the domain problem (DoFixture is great for this). Further, there were quite a few RowEntryFixtures, SubsetFixtures, and RowFixtures that provided quick and dirty access to the underlying DAOs for adds/updates/deletes that allowed for quick modification of data, and each of our tests tried to mimic a user story as much as possible with minimal technical details.

Anyhow, during our daily stand up meeting with our customer, they asked if we could put something together so they could quickly create scenarios for them to test manually, and interestingly enough this didn’t take much time as we were able to show them to use the existing fixtures to interact with the system to do whatever environment setup they needed for their tests.

I think this kind of earmarks a good acceptance testing harness… one that mimics a DSL so much that you can even use your test harness for other purposes to interact with the application. Even more important, when you have an acceptance test that reads like a DSL, it means it is more customer friendly and conveys information in a useful manner. When tests are clear, concise, and easily readable by domain experts, it is much more easier to tighten the feedback loop and make changes very very quickly.

The only drawback with a testing DSL is that the initial setup can sometimes be burdensome… it may not seem worthwhile, and it may even require a little devotion to get going in the beginning and require a little maintenance to be useful. One problem we have had with our new fangled group of fixtures and tests is that development was rather quick in the beginning, but then we reached a plateau where the tests became a bit fragile and required some extra hard work to maintain and improve. Another smell that arose was that our original DoFixture, which contained sentences and statements representative of the domain, soon grew to over 400 lines and felt like a God Object.

However, continuous maintenance and perfection improves quality, as with anything. Test DAOs solely used in the fixtures were refactored out and replaced with the real ones from the system, with a dash of “in memory” DAOs to inject when desired. The DoFixture was broken up a little bit to seperate fixtures represetnative of the different aspects of the domain being tested. And overall, to improve testability sometimes some redesign of the system was required to allow for better inspection of the system under test, which lead to a more solid API.

The final reasult? Although a little dedication was required up front and in the middle stages, writing new tests is a piece of cake. One of our customers called yesterday complaining about a missing feature that was part of a story we recently finished, but seemed not to work. How come? The tests are green. So I worked with him, and he mentioned the product code he was using was ZZZ, which was something slightly different… our test was using product code CCC, which to my knowledge represented the product needed for the story. “Oh… CCC is the product code for that product, but so is ZZZ. ZZZ is a subset of CCC, but in the context of [proprietary domain] they are essentially the same thing.” In only about 10 seconds I was able to add the product code and verify that the expected behavior didn’t take place, then my partner and I quickly fixed, ran tests, and released.

Writing new tests with a test DSL in place is so much faster, it felt like we were writing many tests quickly in later iterations after getting past the plateau of broken or inconsistent fixtures. Perhaps the only problem we have right now is our customers don’t run the tests… they just like to look at them to see what test coverage we have, which has been good because they can see at a glance what we are doing and often make corrections and comments in daily stand ups. They call our acceptance tests our “Test Plan”.

I’ll try to write more on Testing DSLs… I keep thinking about other possibilities outside of FIT/Fitnesse, perhaps with tools like JBehave or even with some of the languages for creating DSLs like ANTLR or OGNL.

While thumbing through the Xunit Test Patterns book tonight, I came across a rather interesting pattern named “Dependency Lookup.” The premise was simple, and I had actually used it on occasion… you delegate object retrieval/creation to a broker rather than access it directly (i.e. factory), and then provide a mechanism to set the object you want to have returned (good for overriding dependencies with mocks/stubs). Normal stuff, I know… but what interested me was one implementation that used a Singleton, but then the test extended that Singleton and overrode the sole instance in order to return test data.

What follows is my late night, 10 minute hack to exercise the idea.
Read More »

Writing a test that checks for static text on a page or dialog that will never change based on domain logic or system behavior and then writing the text to make it pass…

That’s not test driven development. It’s just a complete waste of time and absolutely stupid. By the kind of contrived logic such tests are based on, I should be writing tests for each of my blog posts.

There’s one important aspect of XP (and I think any agile software development process, if not any intelligent process) is to automate all tests. No ifs or buts about it. “But it can’t be done!” you say. Bullcrap. Anything can be automated if you try hard enough, and there are hundreds, if not thousands of tools out there to choose from.

“But, my customers won’t let me do that!”

Who are your customers to say you can’t write automated tests? What they SHOULD be concearned with is that you are delivering them a quality product in a timely matter. Quality being the keyword here… if you waste your development time manually testing the application (which can take quite a bit of time) you are already likely to miss a lot of different scenarios, and each time you need to retest, you’re going to spend the same amount of time manually testing it again… and again, you will probably miss something because let’s be honest, wouldn’t you rather be coding than manually testing stuff?

But then I’ll have to spend x amount of time writing tests, causing something that can be done in a day to be done in 3 days!

But if that extra time spent writing automated tests lets you test all of the scenarios in your application with the click of a button, is it really time wasted? If manually testing something takes 2 hours of your time, and you automate it so that you just click “test” and it does the same thing in even 5 minutes, and now you can repeat that test to your heart’s desire, isn’t that time well spent? If you can sit there and rerun your tests 10 times in 30 minutes, that’s possibly 20 hours of manual testing (which would never happen, so you won’t get the coverage).

The perception that it can take a lot more time to automate tests is concearning… because by automating tests we improve the quality of our product, and that improves our bottom line. So quit holding yourself down… quit thinking it can’t be done… and quit thinking that your customers won’t let you do it. Because all you need to do is tell your customer that you are doing this to ensure that they get top quality, and they’ll buy in.

Anyhow, some linkage to end this quick post.

Automating “All” Tests

While putting together some ideas for the TDD Anti-Pattern wiki tonight, I got to thinking… “Whatever happened to that XUnit Test Patterns book?” (yes, in case you were wondering, I do think with words hyperlinked like that). You know, the one I blogged about almost a year ago when I accidentally stumbled across the Amazon.com page and the author’s site. At a glance, I was enthusiastic, and even joined a mailing list for the book, which I realized I have not seen much activity on in ages.

And it was a good thing I decided to look into it. I mozzied on over to the xunitpatterns site and, not only has it been largely updated,the author states it that it is currently in “Typesetting” and close to release! I got excited, but then read the next sentence…

If all goes well, the book should go to the printers in March and be in stores in April!

Awww… it is May now, and I know it’s not out, otherwise I’d have it in my hands reading it right now.

However, I really look forward to seeing it, and in the meantime the site is an EXCELLENT source, and provides some handy patterns for unit testing. Definitely worth a look!

There’s one post and one post only on my blog that gets the largest amount of attention: the TDD Anti-Patterns entry. This was an entry to kind of precede an article I had prepared for the IEEE special issue on TDD, however that article never materialized (mainly due to last minute writing on my part).

Since this particular blog entry has been pretty popular, I am going to launch a wiki for it soon. Hopefully it can serve as not only a catalog of the smells, but allow for community contributions, examples, and steps to improve and move away from those smells.

So.. stay tuned, and let me know if you are interested in helping out.