Ivan’s agile thinkings

I’ve had this one sat in my FireFox tabs for months and only just got around to reading it.  Gojko Adzic lists the outcome of an openspace session at CITCON Eurpose:  http://gojko.net/2009/09/24/top-10-reasons-why-teams-fail-with-acceptance-testing/

Most of the reasons rung true with me too, especially at the start.  It’s been two years now since we adopted FIT as our acceptance test tool.  We’ve improved greatly since then, but we still have challenges.

1. No Collaboration
   Initially we had developers writing tests and proving the usefulness of the approach.  Unfortunately, the same developers were then tasks with writing a battery of tests against the existing buggy code.  It was a between-project slack period and there wasn’t anything better for them to do.  Unfortunately #2 and #10 came in to play with junior and intermediate devs mirroring the existing system in tests, rather than describing the business requirements.
2. Focusing on how, not on what
   We definitely had a lot of this going on.  We have far more procedural ‘this then this then this’ tests that have a huge amount of setup actions than I like to admit.  We have definitely turned this around somewhat more recently with more abstract ‘business requirement’ tests rather than ‘technical requirements’: the tests don’t change just because the implementation does.
3. Tests unusable as live documentation
   This is my biggest chagrin with the tests, lack of prose around the FIT test tables explaining WHY.  When there are comments they describe what we’re doing with the ‘this then this then this’ style.  Again, we have turned the corner recently but it’s a bugger to retro-fit that kind of knowledge into the tests.
4. Expecting acceptance tests to be a full regression suite
   I think I can own the blame on this one.  I pushed for this to be the case.  In my defense I did push for quick, localized acceptance tests that could run quickly and give the required feedback.  As you may guess, that’s not what we ended up with.  Rather, we got a whole slew of System tests.  They are still very useful tests, they just take longer to run that I want.  In a way it’s a reflection of the system we’re testing: very coupled system begets very coupled tests (maybe).
5. Focusing on tools
   Not really something we struggled with too much.  If anything it was the opposite – few tools in our price range.  We settled on FIT and went forward.  We also use the FitPro Eclipse plugin though it’s limited Mac and Linux capabilities are still a bit of a challenge.  We did look at FitNesse for a while and knowing what I know now I would probably have chosen that as it is more actively developed.  Our biggest perceived challenge was one of integrating with Subversion, in the end it was a non-issue.
6. Not considering acceptance testing as value added activity
   We are doing so much better than before with QA/BA developing the first example FIT tests but we’ve still got a ways to go.
7. “Test code” not maintained with love
   Another sore point for me.  Developers treat our test code as an afterthought and seem to think that good design principles and refactoring don’t apply to test code.  I couldn’t disagree strongly enough.  I am always looking for ways of making my tests simple, robust and expressive.  I often spend a lot longer on my (unit) tests than my production code.  I have hear developers whinge and complain about the test code and how “it’s hard to do such and such”.  I swear that in the time they’ve spent complaining they could have improved the test code ten-fold – but they don’t treat it the same as production code!
8. Objectives of team members not aligned
   We’re actually doing pretty well at this one, we’re a million miles from where we once were.  Our QA guys now sit with the developers and BA’s and there is a true collaborative spirit at times.  I was utterly stunned (shocked, awed, caught-out) recently when a developer said to me “I’ll just commit it and QA can find the bugs”.  I hadn’t realized that that attitude still lived on within our teams.  I guess old habits die hard!
9. No management buy-in
   This one isn’t a problem – I’ve seen too many bugs and mis-understood/implemented features and also seen the benefits of automated acceptance tests to ever go back (so’s my boss).
10. Underestimating the skill required to do this well
    As mentioned above, we definitely struggled with this one initially.  Things are improving as we get better and better with this.  I think the biggest drag was the large amount of tests and fixtures that were written without too much thought that were then copy/pasted or at least aped.

Despite our challenges I think one reason we didn’t fail was that there were several people who held the opinion that ‘failure wasn’t an option’.  We needed to a rigor to our development process: Automated Acceptance Driven Development was a large part of the added rigor.  Yes it takes longer to develop initially, but it pays for itself over and over once you get over the hump.

It’s not often that I get to do much coding these days.  Normally I wander around the office offering advice and guidance to others: aka, generally interfering (aaka, managing by walking around).  But today was a pleasant change, being a Saturday.  I remained cooped up and happy hacking away at a Circuit Breaker implementation inspired by Michael Nygard’s book Release It!  That book has enabled me to look deeper into the lives of Ops guys: the poor suckers who have to run the software that developers write, for years on end!  Release It! is a truly fabulous book and I recommend every single professional software developer to give it a good hard read (perhaps you could use the SQR3 technique).

Anywho, I got to code and I really enjoyed it.  I had written some of this code a while back, but without any tests.  I was allowed to do this because it was just a spike – it wasn’t intended to be production code.  But today I was to turn it into proper code.  Here’s what happened:

1.  Removed extraneuous code I had added as an experiment into component transparency – exposing this as an JMX MBean.
2. Removed other code that wasn’t going to be used in the first cut of this component.
3. Create a JUnit 4 test case.
4. Unit test #1: test that the delegate call works as expected – just what should happen when the circuit breaker is in the Closed state.
5. Unit test #2: test that the circuit breaker trips when a single failure occurs (with the failure threshold set to 1).
6. Refactor the test to make them more concise.
7. Unit test #3 and #4.
8. Refactor the tests, refactor, refactor.
9.  #5, refactor, #6, refactor, #7….. continue testing various aspects of the circuit breaker.
10.  Refactor the test cases and again and again.

I had previously written the code as a spike and tested informally using a browser within the webapp.  But now I wrote the unit tests to prove that I hadn’t coded any amusing bugs.  I wrote each of them so they would fail first, but I was testing existing code – not quite following the word of TDD but living the spirit.

But the thing I brought away from this was the amount of refactoring of the test case I did – I barely changed the existing code which was pair programmed originally.  The amount of test code refactoring reminded me of some work I had done previously about how a specific featureworked.  During my investigation I wrote a set of unit (missing) tests that got refactored and refactored and refactored until it made sense to someone who might care: not just a developer.  This took a good chunk of time to develop, but in the end I was able to publish a literal test as a Javadoc example that would make sense to your Grand Mother!

I did a similar job on the circuit breaker testing – pushing down on the test until it was concise and expressed my intent.  I squished and squashed until I had some really neat assert* statements that made sense for this component:

• assertDelegateExceptionThrown
• assertCircuitBreakerFailsFast
• assertDelegateCallSucceeds

I had previously spent a chunk of time thinking about how this component should function and I reveled in the time I was forced to take in making the tests expressive.

I hope to add the ability to monitor the circuit breaker through JMX tomorrow.  This time it’s going to be TTD.

Mmmmmmm, coding…….

Whenever I play in a new language one of the first things I do is to install the appropriate xUnit framework to get feel for the language.  For my Java code I have been using JUnit for many years.  My use of JUnit hasn’t really changed over the years, but some things have been bugging me for a while.

One of the things that has bugged me is naming the test class for the class under test.  That is, if the class under test is Banana your test class would be called BananaTest.  There are obvious advantages to this pattern:

1. You know where to look for the test code for a class
2. You can find all the test classes very easily – using filename pattern matching.

However, there are several disadvantages.  One is that it ties you to having a single test class for each class which can cause problem.  I have often found myself writing or maintaining large test classes that combine testing different aspects of the class’ behaviour.  Some of these classes would exceed Checkstyle’s line length limit for a method.  One solution to this is to blindly split the class into two classes.  What then to call the new class?  Banana2Test – not very descriptive or helpful.  Maybe we should name the test for the aspect of things we are testing, e.g. BananaSlippynessTest or BananaFlavourTest.  But now the test isn’t following the original naming convention.  This isn’t a big problem, but is there maybe something we can learn from this.

Apparently there is.  Yesterday I attended an Agile Vancouver workshop – “BDD and TDD code off”.  Our group chose to use TDD which is still fairly new to me.  I was lucky though that Peter, who was in our group, was an experienced TDD’er.  It was an interesting experience and the code we produced was a little different to the code I would normally produce.  I don’t yet know if the code was better or worse, for now it was just ‘different’.

That workshop drove me to learn a little more about BDD, so I watched the Google Talk of David Astels introducing BDD and he mentioned how BDD tests are typically structured – around the different expected behaviours.  One example he gave was testing a List class.  You would likely have a test class called EmptyList which would contain the specifications for an empty list.  For example: Ensure list has size = zero, Ensure list iterator has no items.  This test class would contain the specifications for just empty lists.  I assume you would then add a test class for a non empty list, and perhaps another to ensure the ordered-ness of list, and possibly several more.

I’m not sure if I’ll be adopting this in my current project at work just yet.  Doing something different without hard and fast rules can confuse people and can put people off testing, which I really don’t want to happen.  However, in my pet projects and fiddlings I think I’ll be experimenting a little more.