Comments on: When Not to Automate http://blog.james-carr.org/2007/09/13/when-not-to-automate/ Rants and Musings of an Agile Developer Sat, 22 Nov 2008 05:55:43 +0000 http://wordpress.org/?v=2.6.1 By: James Carr http://blog.james-carr.org/2007/09/13/when-not-to-automate/#comment-31716 James Carr Fri, 14 Sep 2007 12:56:16 +0000 http://blog.james-carr.org/2007/09/13/when-not-to-automate/#comment-31716 Peter, I agree... my point is I have come across UI tests that ONLY test that form fields are present. However, I have written tests that do fill out form fields, submit them, and check the output (dao, email, whatever) of the process and maybe the page the user ends up on. As I said: <blockquote> On the other hand, a test that fills out those form fields with different combinations of data, submits them, and verifies the expected outputs is a VERY worthwhile test. </blockquote> I just take an issue with automated tests testing such things like images being present, static text, etc. Of course, this is at discretion... I say it's dumb to test images, but what if it's part of something like a map renderer? Like always, context matters. ;) Thanks, James Peter,

I agree… my point is I have come across UI tests that ONLY test that form fields are present. However, I have written tests that do fill out form fields, submit them, and check the output (dao, email, whatever) of the process and maybe the page the user ends up on. As I said:

On the other hand, a test that fills out those form fields with different combinations of data, submits them, and verifies the expected outputs is a VERY worthwhile test.

I just take an issue with automated tests testing such things like images being present, static text, etc. Of course, this is at discretion… I say it’s dumb to test images, but what if it’s part of something like a map renderer? Like always, context matters. ;)

Thanks,
James

]]> By: Peter Zsoldos http://blog.james-carr.org/2007/09/13/when-not-to-automate/#comment-31715 Peter Zsoldos Fri, 14 Sep 2007 12:48:48 +0000 http://blog.james-carr.org/2007/09/13/when-not-to-automate/#comment-31715 James, Your statement that "IMHO a test to make sure form fields are present is a waste of time" is without context. If you are using a form builder class/module, then there is definitely a point to making sure that the expected fields are present in the form. If it is a static, manually edited form, you might be right. However, I don't see that it would be too much work to automate it. I would even argue that writing the test for it and running the test is much faster than manually verifying it. In addition to that, having such test in place could point you to the source of error pretty fast should that field be accidentally deleted, saving you some time hunting for bugs in the wrong places. James,

Your statement that “IMHO a test to make sure form fields are present is a waste of time” is without context.

If you are using a form builder class/module, then there is definitely a point to making sure that the expected fields are present in the form.

If it is a static, manually edited form, you might be right. However, I don’t see that it would be too much work to automate it. I would even argue that writing the test for it and running the test is much faster than manually verifying it. In addition to that, having such test in place could point you to the source of error pretty fast should that field be accidentally deleted, saving you some time hunting for bugs in the wrong places.

]]> By: Guillaume http://blog.james-carr.org/2007/09/13/when-not-to-automate/#comment-31714 Guillaume Fri, 14 Sep 2007 12:40:04 +0000 http://blog.james-carr.org/2007/09/13/when-not-to-automate/#comment-31714 I kinda agree with the general point, but I feel that you're missing one key benefit of automated tests: backup for refactoring. Even when a test seems dumb or unneeded, you never know when it can suddenly become the indicator that something went wrong in a refactoring session. Simply put, the more tests I have in my project (even silly ones sometimes), the more confident I am when hacking and slashing in my code refactoring everything. I just run my full build, and if everything still passes, then I can commit to the source control and sleep well at night. Granted, the above does not apply to "static text" testing, obviously. Such tests are really silly. I kinda agree with the general point, but I feel that you’re missing one key benefit of automated tests: backup for refactoring. Even when a test seems dumb or unneeded, you never know when it can suddenly become the indicator that something went wrong in a refactoring session.

Simply put, the more tests I have in my project (even silly ones sometimes), the more confident I am when hacking and slashing in my code refactoring everything. I just run my full build, and if everything still passes, then I can commit to the source control and sleep well at night.

Granted, the above does not apply to “static text” testing, obviously. Such tests are really silly.

]]> By: Sammy Larbi http://blog.james-carr.org/2007/09/13/when-not-to-automate/#comment-31709 Sammy Larbi Fri, 14 Sep 2007 11:34:38 +0000 http://blog.james-carr.org/2007/09/13/when-not-to-automate/#comment-31709 James, I think it can be worthwhile to verify that a dynamically built form is build according to expectations, but I can see why you think it would be silly to verify that a statically-typed-in-by-the-programmer form would meet expectations. All you are doing is requiring them to type it twice - once for the test and once for the site/application, which would just turn into copy and paste. It might be nice to automate testing that it is valid markup though. Even an existence test can be worthwhile for the times someone messes with it and removes a field that should be there (such as when you build a form, client has access to change it, and then removes a field that should always be there). Regards, Sam James,
I think it can be worthwhile to verify that a dynamically built form is build according to expectations, but I can see why you think it would be silly to verify that a statically-typed-in-by-the-programmer form would meet expectations. All you are doing is requiring them to type it twice - once for the test and once for the site/application, which would just turn into copy and paste.

It might be nice to automate testing that it is valid markup though. Even an existence test can be worthwhile for the times someone messes with it and removes a field that should be there (such as when you build a form, client has access to change it, and then removes a field that should always be there).

Regards,
Sam

]]>