Ivan’s agile thinkings

I had the good fortune to attend the latest VanDev meetup on EAI tools versus custom build solutions.  David Dossot gave a high-level overview of Enterprise Application Integration tools and how they compare to the ad-hoc chunks of scripting and home-grown glue that are used to integrate systems.  David covered several aspects that might affect whether you pick on or the other.  His presentation is online at Prezi (which I think is a very fancy presentation system).

Update:  David actually blogged on this too: http://blogs.mulesoft.org/presentation-eai-when-tools-can-help/

The big thing for me about this presentation was that it finally tripped a switch in my brain: Enterprise Application Integration patterns don’t have to be about asynchronous queues and such, it’s about all the different ways systems integrate.  That includes simply copying files around, for example.  I honestly wish I’d realised this a few months ago, it would have given me the insight to stop a strange project using rails and leverage an existing EAI engine/framework.  Despite the learning curve we would now have a less custom, and more tested, system.  Better late than never?

I’m also currently on a mild clojure kick.  Having done the Bowling Game Kata in Groovy and then followed it up with a clojure version (thanks to this link for some hints) I thought I’d try to get a simple Apache Camel example up and going in clojure.  The first hit on Google proved very useful for guidance.

Simply trying to get that (and the bowling) example up and running made me have to dig into so many different things:

• leiningen - a clojure wrapper for maven
• Polyglot Maven - a wrapper for maven in several different languages so XML can be shunned
• test-is – a simple unit testing library in clojure-contrib
• AOT – ahead-of-time class creation, amongst other things this allows clojure to run in a binary form (no need for source in production).
• Counterclockwise - an Eclipse plugin that adds support for clojure editing and running (with very pretty, and effective, rainbow bracketing)
• Clojars.org - a maven repo for clojure jar files
• Cloduino can be used to allow clojure to interact with Arduino boards
• Getting started with clojure - this link has a bunch of detail on getting started with clojure.
• labrepl - a learning environment for clojure
• I fired up Idea for the first time in ages and install La Clojure just to try it out.

Obviously, I have only scratched the surface of these things, but at least I know about them now and have a better overall handle on clojure and EAI.

I really do love learning new things and the VanDev meetup triggered more of this than normal.  So I encourage you all to come to VanDev or find a similar meetup near you which covers things that you really enjoy (I’ve got my eye on the local Erlang group next).  

InfoQ is one of my favourite techy sites on the interweb.  It’s full of really great presentations, interviews and articles about many different aspects of software development.Here’s an example where Marc de Graauw writes about not needing reliable messaging.  If the business operation needs to be once-and-once-only or the operations need to be in-order then the mechanics should be pushed down into the business message/processing.  Don’t leave this to the transport layer, create appropriate idempotent operations and most of the problems go away.If you don’t already have InfoQ on your feed list then I can’t recommend adding it in enough

I like to think that one of my strengths is my willingness to pro-actively learn new things.  Recently I have been trying to learn about learning by reading the excellent Pragmatic Thinking and Learning by Andy Hunt of The Pragmatic Programmer fame (which remains one of my favourite books of all time).

A lot of the content in PTL wasn’t that new to me, as I have previously read several of the books that Andy has read and references from the text.  That said, I definitely learned new things and deepened my understanding about certain areas.

Constant learning is definitely a incredibly valuable thing, I look for it in every potential new hire I interview.  But continually learning in the same habitual way may not be the optimal strategy.  If you apply the agile development approaches to how we learn it suggests that you should retrospect at what works, what doesn’t and that we should be prepared to try different things to see if they work better (rinse and repeat).

Though I’m not a novice at learning, I am a novice at learning using new tools and techniques: when was the last time you learned how to learn in a new way?  So how do I move froward from here?  According to the book, what I need is some novice/advanced beginner handrails to hold whilst I improve.  Different to the other books that I have read around this subject, PTL brings a consolidated set of next-steps in the context of the life of a software developer.

This is the initial list of things I am going to start:

• Read more deliberately using the SQR3 technique, then blog about what I have learned.
  • Read PTL again, this time, read it deliberately.
  • Re-read Domain-Driven Design deliberately.
• Increase my blogging frequency – hopefully doing more often will make me quicker.
• Review my blog posts once in a while – do I still agree? what have I learned since?
• Try mindmaps again having deepened my understanding about how they might help me learn/understand.
  • Redraw mindmaps every now and then – let some mental mushing happen – see what my brain has subconsciously come up with.

Ever more things will come up as I continually learn and continually learn to learn.  I’m still on that journey that started the best part of 33 years ago and one that I hope doesn’t stop for a very long time yet.

On the way back from a long-weekend on Vancouver Island we had a little time to kill whilst waiting for our ferry home.  There was a Chapters store nearby, so that was an easy mascre of half and hour.  After browsing for a while and noting possible future purchases I purchased Objective-C Pocket Reference.  At 122 small pages it was a quick read and covered he basics of Objective-C’s syntax and detailed various fundamental aspects of the Cocoa and GNUStep and the various differences betwen them.

I got from it pretty much what I was hoping: an easily digestible introduction into Objective-C.  I had previously read a bunch on the web, but I always find reading books, rather than reading a screen, to be far more effective for myself.  I had many different questions answered from an attempted iPhone coding session a few months ago.

I have been nudging some of the guys at work to start an Objective-C/iPhone/Mac programming study group. This book will be very useful for the novices in the group (myself included) to quickly look up the fundamentals and work through the syntax impedance as they try and push a new language into their Java brains – this was definitely a sticking point with the Erlang study group we had for a while.

In summary, a great, concise reference – just like the title suggests.  It appears that the reviewers on Amazon also agree.

A while back I attempted to create a visualization of the relations that OpenJPA creates between the Java classes that it enhances – the intent was to help better understand a particularly intertwined persistent object graph and see if I couldn’t figure out why we were getting a few strange problems.

The initial approach I took was a hilarious trip into the land of sed, perl and graphviz.  It kinda looked like it worked, but didn’t actually

Time passed……more time passed……I learned more……more time passed…….and then I had the wacky notion of tying directly into the Enhancer that OpenJPA uses to create the persistence enabled classes.  I haven’t done the visualization yet, but this is what I learned about actually tying in an AuxiliaryEnhancer.

1. Create the AuxiliaryEnhancer

The following code is the minimal you will need.  Enter this and then compile – you will need OpenJPA and serp on your classpath.

package com.ij;

import org.apache.openjpa.enhance.PCEnhancer.AuxiliaryEnhancer;
import serp.bytecode.BCClass;
import serp.bytecode.BCMethod;
import org.apache.openjpa.meta.ClassMetaData;

public class Grapher implements AuxiliaryEnhancer {

    public void run(BCClass bc, ClassMetaData meta) {
        System.out.println("Running Enhancement");
    }

    public boolean skipEnhance(BCMethod m) {
         System.out.println("skipEnhance?");
         return false;
    }
}

2. Register the enhancer using jar based services

This is something I was only barely aware off – registering concrete implementations of services through jar file meta-data.

• Create a file in a META-INF/services/ directory called org.apache.openjpa.enhance.PCEnhancer$AuxiliaryEnhancer
  • Note the dollar sign! That one caught me out for a while: I couldn’t decide whether the dot notation or the dollar notation should be used to reference the nested class.
• This file should simply contain the fully qualified name of the Enhancer you wrote: com.ij.Grapher
• Jar this directory and the class file up and you should end up with a jar listing like this

ivan-jensens-computer:jar ivanjensen$ jar tvf ../grapher.jar
     0 Tue Dec 02 21:56:42 PST 2008 META-INF/
    60 Tue Dec 02 21:56:42 PST 2008 META-INF/MANIFEST.MF
     0 Tue Dec 02 21:54:28 PST 2008 META-INF/services/
    16 Tue Dec 02 21:30:44 PST 2008 META-INF/services/org.apache.renamed.openjpa.enhance.PCEnhancer$AuxiliaryEnhancer
     0 Tue Dec 02 21:55:16 PST 2008 com/
     0 Tue Dec 02 21:55:38 PST 2008 com/ij/
   771 Tue Dec 02 21:55:38 PST 2008 com/ij/Grapher.class
   465 Tue Dec 02 21:54:46 PST 2008 com/ij/Grapher.java

3. Add the new jar file to the enhancer classpath

I did this by adding my jar file to the classpath element of my ant script – pretty simple.

4. Re-enhance your classes

Re-enhance your classes and you will see the System.out.println statements all over your console.

 [openjpac] skipEnhance?

 [openjpac] skipEnhance?

 [openjpac] Running Enhancement

 [openjpac] skipEnhance?

 [openjpac] skipEnhance?

 [openjpac] skipEnhance?

 [openjpac] skipEnhance?

 [openjpac] skipEnhance?

 [openjpac] skipEnhance?

 [openjpac] skipEnhance?

One thing I noticed about this: I had to delete my persistent classes and re-enhance, only then would the ‘run’ method be called.  The skipMethod is called whether I delete the classes or not.  I don’t know what that means yet, but I’m sure I am about to find out.

Notes:

I was actually using a patched version of OpenJpa 1.0.1 (for various, non-amusing, reasons), but the patches shouldn’t affect this tutorial.

I’m constantly reading, watching or listening to stuff that makes me learn an I recently found the following very useful and timely for the work I’m doing at the moment (or in the near future).

• Dave Syer from Spring Source talking about Spring Batch – extracting the essential elements of batch processing Spring style.  If you have to write batch jobs in Java you could do a lot worse than use Spring Batch as a base.
  http://www.infoq.com/presentations/syer-introducing-spring-batch
• Mishkin Berteig on how using (incredibly short) 2 day Scrum iterations acted as a catalyst to make a business become more collaborative and better prioritise the work for their (rather small) IT department.  Eventually determining that the IT department shouldn’t have been working on any projects other than one that would make the company more money than the rest of the projects put together!
  http://www.infoq.com/presentations/Short-Iterations-Mishkin-Berteig 
• A friend of mine sent me a link to a very interesting talk by Mitch Lacey about how a Scrum project he was leading went very wrong – what happens when the business just doesn’t ‘get’ agile development.

  http://www.infoq.com/presentations/A-Story-of-Project-Failure-Mitch-Lacey

• Randy Shoup on the architecture of eBay – learn that data consistency doesn’t have to be 100% but you have to trade-off with availability and data partitioning – extremely illuminating.
  http://www.se-radio.net/podcast/2008-09/episode-109-ebay039s-architecture-principles-randy-shoup