Archive for the ‘Articles’ Category

Scrum Shock Therapy, Part 2

December 17, 2008

So we have a recipy for the team. But this is just not enough. We need to make sure that management and everyone else is on board. The second part of the Scrum Shock Therapy presents a bootstrapping recipe on how you can do this.

The first part of this series can be found here: Scrum Shock Therapy, Part 1.

The Management Recipe

The checklist above is only a part of what it takes to go through Shock Therapy. I often find myself at new companies and need to work with a bigger picture than focusing solely on the team. As is evident from the polls and surveys above, we need to address management as well. The word shock is perhaps more applicable here. If it has learnt anything from the past it is that you have to “drive” development – delegate, follow up, check, adjust, hand out, recheck…
Telling them to trust the very group that has not delivered before is asking for a leap of faith.

Setting up a recipe for this group is tricky. You have to adapt to management style, the corporate culture and so forth and the things you ask for are softer. These can be very hard indeed to change! In plain English; The management has to take some more real tough decisions for a Scrum to work efficiently. If you need to kill a document standard or process step then do so! If this is impossible (for now) then go for “barely enough”.

The Management

  • Attend the Scrum training session with everyone else
  • Hands off during 3 iterations
  • Attend:
    • Some Daily Scrums – be quiet
    • All Sprint Reviews
  • Start to work on waste – now!
  • Management by walking, asking and listening, i.e. practice facilitation.
  • Make the first step easy for the team

When learning something new you have to do it several times just to make sure you are not totally confused. I have found that three times a charm. Try your best to get management to abide by this “rule”. After the three iterations everyone has better picture to judge the value of the effort and Scrum.
Eliminating waste is vital on any type of project or endeavor. In a transition it makes even more sense, you have to make space to learn something new.

Finally, management has to ask the people that will perform these miracles what they need to do just this. Have they? Have you?

The Management Exit

  • One successful team
  • Removed impediments and studied results
  • Change in perception at “ground level”
  • A good agile reason to change the rule

Having one successful team is sort of self-evident. This takes patience when the ride is anything but smooth. Being a part of this process is important. Management will have the best ability to change ambition level, removing impediments etc. Get cracking and be that good example.

If there is a change at ground level on how they perceive management, then you’re probably on to something.

Is this recipe the final version – probably not? Your mileage will vary, adapt to your management.

The Organization Recipe

Many of our customers are just starting out doing Scrum. Therefore we typically come into play as an external effect. It is important for us to understand how the agile effort has started and its nature within our client’s organization. We can basically divide this efforts into to two major types – “bottom-up” and “top-down”. The typical traits for a bottom-up style is one where the programmers et al have decided to go agile. The management might be aware of this and even letting it run. Then again this might be a stealth and an under the cover job.

I have found the bottom-up approach to be the most common case. Programmers usually like to work in an agile way. In these type of setups my work usually revolves around questions like “How do we fit this to a waterfall context?” etcetera. This can be tough work, but I find it to be the more easy of the two – the people who do the day to day grinding are already on board.

The top-down effort of introducing Scrum comes with its own set of problems. Any methodology or decision that involves change tend to meet resistance from the organization. We need both experience and sideways effort to the transition.

The Top-Down Scrum Organization

  • Get a Scrum Sensei
  • Inject a Agile Senior Programmer into the teams
  • Make sure there is organizational transparency

The Scrum Sensei is an experienced Scrum Master who’s been there before. Much in agile and Scrum is common sense, but there is no need in making the simple mistakes. With a battle-trained mentor the whole organization get a sounding board (Swedish: bollplank) helping out to implement Scrum. This part is important to the Shock Therapy where he or she will be the contract owner enforcing the startup and approving exit. In some sense this person could act as a “bad cop” in a good-bad ScrumMaster setup.

Or in the words of Nanny McPhee – a children’s movie where a nanny answers the seven ne’er-do-well children on how long she will stay:
– When you need me, but do not want me, then I will stay. When you want me, but do not need me, then I have to go.
The Scrum Sensei needs to work in similar way to bootstrap the team and management.

A top-down effort always risks resistance from the organization. My experience tells me that a side way force is needed for translation on the factory floor. Moreover, such a person has the experience on a technical level to do the hands on tools and practices that is a vital part of any agile projects. A seasoned programmer with agile experience knows what works and how to translate the management vision into bits and bytes.

“Self-evident” sometimes reads “evident to myself and no one else”. If we want to affect the organization we also have to make sure that practices, experiences and results are easy to come by. Walking down a corridor should yield interesting information about ongoing projects. If there is a pilot project doing Scrum then visibility is high on the wish list. Not only does it mean commitment for the Scrum team, but also a pull factor for other teams to get on board. Organizational transparency comes in many flavors. Apply liberally!

Summing it up

There is no denying that projects have problems getting started with Scrum. But more often than not I believe that we can do something about it. Why not try some nice Shock Therapy?
It might be the sweetest hard deal around!

/Björn Granvik, Jayway

MySpace Therapy:

Scrum Poll on Nokia Test Practice:

VersionOne Report: State of Agile Development Survey:

Situational Leadership:

Nanny McPhee:

Originally published at Jayway Team Blog.

Scrum Shock Therapy, Part 1

November 17, 2008

Scrum consists of a straightforward process, half a bunch of roles and a few artifacts. Sounds simple enough, but according to an online poll 3 out of 4 projects that call themselves Scrum fail to implement even the simplest parts. To make matters worse, most of the mistakes are on the simple side of things. Is there a way to bootstrap the project? Might there be a sweet hard deal that we can use?

Scrum and Agile development has started to rock the world within IT in various ways. A simple, but powerful process that seems to deliver. Developers and customer alike are happy. What’s not to like about it? Still, there are signs of problems. Projects that claim to use Scrum do not reap the benefits, management looses patience with this new method and team members struggle to adapt an agile approach to their everyday work life. Why is this? And more importantly, what can we do about it?

The Therapy: Get off to a good start by directing the team with a careful set of good practices and a strict agreement that leaves little or no choice. The team can then, over a couple of iterations, gradually take command themselves.

Easy, but still so hard

Why then is Scrum so hard? Well, first of all it is fail-fast, meaning that any problems will be quickly apparent. There are no artifacts or process steps to hide behind. This is never easy. To this list we can add even more pitfalls that are just plain hard to deal with whatever method we are using.

Still, it does not explain why there seems to be so many simple mistakes with such a simple framework. One of the best examples is the failure to keep backlog that is prioritized and up to date.
– We’re doing Scrum, but we’re too busy to work on the product backlog!

That one has always short-circuited my brain. What can be more important than to decide what is important? This pattern of falling short and missing basic parts of Scrum even has a name – “ScrumButt”. If you look in the sentence above you see why.

Nokia created a test to make sure the projects are “Scrum enough”. These plain questions form a lithmus test that can be used to analyze most agile projects.

The Nokia Test
Iterative Development?

  • Iterations – time boxed, less than 4 weeks.
  • Software features – tested and working at the end of each iteration.
  • Iteration must start before specification is complete.

Scrum (in Nokia’s opinion)?

  • You know who the product owner is.
  • There is a product backlog prioritized by business value.
  • The product backlog has estimates created by the team.
  • The team generates burndown charts and knows their velocity.
  • No project managers (or anyone else) disrupting the work of the team.

The questions seem to be on the easy side. It turns out that this might not be the case.
One online poll, Nokia Test by Practice , about the ability to achieve the Nokia Test level gave some interesting answers. In essence, only one out of four give themselves a full mark. Half of the groups report that management interferes, not directs but interferes.

Informal queries by Jeff Sutherland among his audiences around the world give an even worse number – something like 1 out of 10 pass the Nokia Test. Hand on your heart: Does your project live up to this?

A survey by VersionOne tries to dig deeper into the current state of the Agile affairs. Under the headline “Top barriers to adopting Agile” we read the top four answers:

  1. Organisational culture, 45%
  2. General resistance to change, 44%
  3. Lack of people with experience, 42%
  4. Lack of management support, 32%

Take a long look at these barriers. No technical stuff. Nothing specific to agile. Just people.

What do we know?

Learn to walk before you run. Greek Proverb.

We could be a bit closer to today’s reality than these ancient words of wisdom. The Situational Leadership offers an approach to management that has been around since the 60’s. These theories say that there is no single leadership style that is optimal for all people in every situation. Situational leadership talks about four different leadership styles that need to be matched with the follower. Basically, when leading someone who is passionate, but have no experience, you should adapt and give clear directives. As the follower evolves his or her ability to execute, so should the leader using less directives and more encouragement. In the end the follower is able to perform with only a stated goal and little or no directives.

If this holds true for people, could it not be true for teams as well?

The Shock

So we have problems making Scrum and Agile efforts to run. What is the conclusion? Could it be that we adapt our leadership style to the group? Make them walk before running?

This is counter intuitive, at least to me: To go agile, we first need to control. We should not let our teams and managements start off without some kind of directives. Give them a good deal, some practices that make Scrum work. Otherwise, we are just repeating the same old mistakes as the next-door agile project. Set up an agreement with both management and team so that they can evolve and experience Scrum for real. Just reading a book (or article) or listening to some good speaker alone does not help you out there in real life. You have to experience it.

Shocking as it may feel: We need to exercise some control over the Scrum project start so that it can evolve and self organize! But, and this is a big but, this has to be done with compassion and genuine involvement, otherwise we stand the risk of destroying what we are trying to build.

The Therapy

So what can we do? We need to work the team, management and often the whole organization.

The Team Recipe

So how do we get started? Here’s a recipe for the team as it used at MySpace:
The Team

  • Scrum training session for everyone
  • Sprint 1 week long
  • Definition of Done:
    • Feature Complete
    • Code Complete
    • No known defects
    • Approved by the Product Owner
    • Production Ready
  • Story Points
  • Physical Task Board
  • All-in-one Sprint planning meeting.
  • No Multi-tasking, work in priority order.

Scott Downey, Agile Coach at MySpace

The idea of this recipe is to set a good practice right from the start. Instead of debating the details on how to work in an agile way, we can focus on getting going.

Everyone has to have a common Scrum understanding to start from, hence the training session. One-week iteration sounds really tight for any reasonably sized projects. The thinking behind this is to get as quick iterations as possible – the faster that the group gets going the better. Short sprints help in this aspect. We learn faster with more turnarounds.
This in turn affects the meetings where MySpace has elected to combine them all, except the morning meeting, into one single occasion. The time is just too short to in a weekly sprint.

MySpace’s definition of done is a good and basic list. However, as a programmer I lack for instance continuous integration, pair programming and so on. These are not mandatory to have, but I see them as enablers for Scrum and supporting your agile initiative.

The next two on the list – Story Points and Physical Task Board – are really effective. If you haven’t tried them out then please do. They work. Finally we have something that is a no-brainer to me even though I have great problems avoiding it, “No Multi-tasking, work in priority order”. Context-switching is expensive, just avoid it. The priorities are in order, you just have to follow them. In the Reference box you find a link to Scott Downey’s description of their therapy.

When the team matures, they should evolve and handle things more on their own and no longer need directives. At MySpace the group can “prove” that they are ready for prime time by reaching the following goals.

The Team Exit

  • Hyper-Productive (>240%)
  • Three successful Sprints consecutively
  • Good business reason to change the rule

The Result?

So how does this work out? At MySpace all groups achieved exit. All, but one, improved after that. One group even achieved a whopping 1,650% improvement after just four months (16 sprints). At Jayway one of the teams used such a bootstrap technique, primarily on the technical side, and reached 800% after 3 months.

The second part of this blog can be found here: Scrum Shock Therapy, Part 2 where I will look into how to handle management and organisation with some recipes.

/Björn Granvik, Jayway

Originally published at Jayway Team Blog.

Interview: Dr Jeff Sutherland, father of Scrum

February 2, 2008

Jeff Sutherland has piercing blue eyes – friendly, but probing – and a frequent smile. They are a bit like a distant uncle who wants to know what you have made of yourself.

After several CTO assignments at various companies, Jeff Sutherland found himself at Easel corporation where he created Scrum together with Ken Schwaber. Ken later presented this methodology at the OOPSLA conference in 1995.

A dozen years later I find myself in search for a good spot where I can concentrate on my interview. Jeff and I sit down at a café at the Øredev conference. My computer has been acting up going blue screen on me twice that day. Time is short and I wonder if it’s an omen.

Photo of Dr Jeff Sutherland.

Photo of Dr Jeff Sutherland.

Dr Jeff Sutherland. Photo by Mai Skou Nielsen, Trifork

Q: I hope I’ve done my reading. As far as I understand you’ve been a fighter pilot in Vietnam, an assistant professor in math and medicine and the CTO at nine different high tech companies. Moreover, you travel the world to teach Scrum. Does it take a resumé like this to invent a process like Scrum?

The key part of what makes Scrum work is “complex adaptive systems” and you need some understanding of this. Also you need to understand systems theory and how to optimise a system as a whole.

You do not have to be smart, but you have to thoughtful.

Q: How did Scrum start and from where does it draw its heritage?
Scrum and Lean both derive from complex adaptive system theory. Constraint theory is a narrow subset of complex adaptive systems thinking. Lean implementations of these theories have been created at companies like Toyota and Honda. Honda was actually more aggressive at this in the nineties, but then Toyota was more successful in the market. They also were more formalised and better known.

Q: Why does it work?
The first reason for all of this is important – super intelligent performance out of ordinary developers. In order to do that we need we need to set up a simple framework so that people will self-organize. The second reason is that there is no central point of control. The manager needs to back off. That is critical.

The third principle is behaviour that emerges in the team based on self-organizing in order to get max production.

The fourth reason is that the production is far higher for the group than any single developer. Groups actually achieve more than any lone bright developer.

These are the basic principles of a complex adaptive system. Any organization is a complex adaptive system. Software production is an organizational phenomenon. It is an empirical process that must absorb significant change in requirements, technologies, priorities, and people.

Q: You talk about managers, why is it important for them to back off?
A company needs to stop sub-optimizing locally. That is the thing that Toyota figured out, largely due to the efforts of Edward Deming and other Americans who trained the Japanese in WWII industry processes. Most managers just care about local performance. Managers have to work as a team to look across the whole spectrum. It is difficult for them. They’re paid and given bonuses on a single level, not the company as a whole.

For instance, look at the General Motors manufacturing plant in Fremont, California. GM shut down that plant 1982 due to several problems, among them quality. Two years later it reopened as a joint venture between Toyota and GM. Toyota needed a plant in the US in order to avoid import restrictions. In an agreement with the labour union they offered reemployment to the former employees.

Two years later the NUMMI plant was producing over 200 000 cars per year and was twice as productive as the typical GM plant. The thinking they applied is the same as “lean production” where Toyota makes the team responsible to figure out the best way to solve the problems and achieve the goals. NUMMI later won several awards for quality.

“Your battery is running low”, declares my machine solemnly. Jeff and I make a run for a power outlet and find one on the second floor. My power chord is too short so we stack up real close to the wall. It will do. More than that actually, it feels like we really are working together on this interview.

Q: Some parts of Scrum are frightening, like making the group totally responsible for an implementation. How come people in management buy this?

The best answer I can give you, is the discussion I had with my first CEO when I introduced the ideas of Scrum. We had a critical project with a hard deadline. It was a question of survival.

The CEO expected a Gantt chart [ed: a timeline specifying activities and their dependencies, showing the progress of a project].

  • How many Gantt charts have you received that actually worked? I asked the CEO.
  • None, was the short answer. In my 25 years I haven’t received a single one that was correct. I asked him if a 100% failure rate wouldn’t make it foolish of me to give him one.
  • What are you going to give me, he asked?
  • Working software. Decide for your self to if we’re on track. In return I made him promise not to disrupt the team before the deadline. At this point he started to worry.

I said that at the end of the month he could change everything he wanted, but not during that month.

  • If you do this, you will get significant more code with better quality.
  • I can live with that. Let’s do it.

Scrum was born in that minute.

It’s a basic contract between management and team. The team delivers twice as much stuff with twice as good quality (minimum) and the management supports the team in their efforts to self-organize and self-manage. If either party does not fulfil this contract then it will fail.

Q: Doesn’t Scrum fail?

Scrum itself is a set of rules that causes self-organization if certain things are put in place. It’s like the rules of soccer. Does soccer fail?
With Scrum you will fail significantly less.

The Standish Group \[ed: a company that specializes in IT studies\] reports that project in the range 3-6 million dollars with a traditional project plan fail more than 85% of the time. Scrum does a lot better than that!

Q: What are your top three tips on using Scrum?

Bas Vodde was the Lead trainer for Scum Master at Nokia Siemens Networks. He has the shortest set of questions to identify a dysfunctional Scrum team.

You must have iterations of 30 days or less with working software tested at system level at end of each iteration. You need an agile specification, just enough for programmers to start for that iteration.

You need a product owner who orders activities by business value of the feature set. The team estimates the product backlog.

And lastly, the team needs to monitor progress with a burn down chart and know its velocity at the end of the iteration.

Q: Finally, what is in the future for Scrum?

Hyper productivity on an economic and marketing level where companies consistently out perform their competitors. Scrum is a disruptive technology that will alter markets. Companies doing Scrum well will grow rapidly in both size and profitability and dominate their waterfall competitors. There is already a brain drain going on where the better developers are fleeing from waterfall companies to Agile companies. This will hasten the demise of traditional projects.

Q: Is this a shameless plug?
Oh no, we have achieved it in several departments at my company \[PatientKeeper\]. It is doable. Other companies I work with right here in Scandinavia are doing it as well.

Going away from this fast paced interview, I still have the feeling that I could have done more with my life. If Jeff can do it, then so can I …I hope

Originally published in JayView.

Interview: Martin Fowler – man in the know

February 2, 2007

I am in search of an empty room at the Øredev conference. Normally this is an easy task, but I’ve got Martin Fowler on my tail. My mind is still blank. What on earth can I ask him that he hasn’t already written himself?

Finally, an empty room, well almost. Another speaker, Erik Dörnenburg, is sitting half way into his screen and mutters.
– What’s up, I ask.
– I’ve updated my machine and my demo doesn’t work. I’ve got 45 minutes until the presentation.
We sat down next to him. Do not disturb a developer while he’s coding…

So I got a man who has coined phrases like Dependency Injection and POJO in front of me. What next? Martin is easily recognizable both in accent and appearance, a frequent and brilliant speaker.
He has an excellent web site,, which contains loads about his work. Articles and references abound. That is when it suddenly hits me – who is he as a programmer and person?

Q: When was the last time you coded?
Well, I do code my own website. But it’s been a while since I had any paying customers. I’ve been pairing quite recently though. A real delivery? That was some time ago. I’m actually afraid to lose contact with code, but I have smart people around me.

Q: But what makes you tick?
I enjoy trying to figure out new techniques – to organize knowledge. I see myself as a conduit [ledning] for transferring knowledge, to process what is out there and make some kind of structure out of it. Brian Foot actually described me as an “intellectual jackal with a good taste in carrion” [intellektuell schackal med god smak för kadaver].
I look around for interesting stuff and try to make sense of it.
The “Refactoring” is a good example. I figured out how to describe it and wrote a book that came out when it could make a difference and move the area forward.
I also enjoy writing a lot, that’s a big thing. I’m better now at speaking, but that’s not what makes me tick.

Q: You’ve written quite a few books – how do they compare?
Out of the five, “Uml Distilled” sold more copies than the others put together. Usually you can’t make a living out of your books, I guess I could though.
All of the books had their good sides, but I would have to say that it was fun to write with Kent Beck [red: wrote “Extreme Programming”, created JUnit etc]. We were in tune and could support each other through the dull bits.
I would have to say though that I’m proudest of “Refactoring”. It’s an important technique and didn’t get the attention it should have received – the book helped.

Q: How did you start out?
I was an independent consultant for many years. Giving talks was a good way of getting jobs. Articles same thing – it got my name known.
Also, I write something because I don’t understand a certain area or technology. It’s a good way to learn.
Erik is now on the phone with California. We calculate that time is roughly 6:30 there – in the morning.

Q: Then what? How come you started working for Thoughtworks (TW)?
I’ve been there for six years and done a lot of consulting. I never wanted to work for a company, but there was something about TW that made me interested.
Get the work done and tons of bright people. But more importantly is that it is a sort of social experiment. A notion that good people makes a difference.
I hope we can affect IT, which is a difficult and skilled exercise at best.

Q: What is the most difficult part of being a celebrity?
I’m not an extrovert person. I’m not good at the “person to person”. I get emails with questions like “I got a problem on…what is the magic trick”. They worked for months on it and I can only point to a book. That clearly wasn’t an answer they liked. It’s frustrating.
However, celebrity is also a nice thing – it opens a few doors. I can email people like Rod Johnson [red CEO of Interface 21 that created the Spring framework] if I have a question about something. And he will answer.
People tend to think I’m an ingenious programmer. I’m not. I’m pretty good, but not necessarily that great.
Erik suddenly spits out:
– F—!…ok the demo will be shorter.

Q: Looking forward, what’s next?
Oh, there is tons of stuff to write about. The design patterns area for instance. I’m also interested in DSL [domain specific languages] and agile development. But in agile there are too many writers and I don’t like competition. There are too many smart people in agile development.
My strategy is to look for topics that no one has written about. Basically I don’t foretell the future.

Q: What are your top three pieces of advice to a programmer?
My first advice must be to learn to collaborate with the user or purchaser. The really good ideas usually come from them. You don’t have to be an expert to do this. This I found to be a good general advice.
Secondly, it would be “continuous learning”. It’s like running up a downwards-moving escalator – you have to keep running.
The third one is difficult…
“Buy lots of books by good authors” would be it.
Erik suddenly releases a big:
– Yes!
I saw Erik’s demo some twenty minutes later – it was really good.
As for Martin, our discussions continued well into the debate panel and beyond. He would frequently forget his back pain and sip into some extra energy pack. I wonder how he did that.

Originally published in JayView.

Neo – a netbase

February 1, 2007

Neo is a network-oriented database for semi-structured information. Too complicated, let us try again. Neo handles data in networks – nodes, relationships and properties – instead of tables. This means entirely new solutions for data that is diffi cult to handle in static tables. It could mean we can go agile all the way into the persistence layer.

The relational database represents one of the most important developments in the history of computer science. Upon its arrival some 30 years ago, it revolutionized the way the industry views data management and today it is practically ubiquitous.

In fact, it is so taken for granted that we, as an industry, have stopped thinking. Could there be a better way to represent and store our data? In some cases the answer is – yes, absolutely. The relational database is showing its age. Some telltale signs:

  • The mismatch between relational data and object oriented programming.
  • Schema evolution – updating your data model when the domain model changes – is just so much manual labor.
  • Semi-structured or network-oriented data is diffi cult to handle.

The Neo Database

Neo is a database that is built with a different philosophy. Where the relational database squeezes all data into static tables, Neo uses a fl exible and dynamic network model. This model allows data to evolve more naturally as business requirements change. There’s no need for “alter table…” on your production databases after you introduce a new version of the business layer and no need to rewire and migrate your O/R mapping confi gurations. The network will evolve along with your business logic. This spells agility.

Neo is an embedded persistence engine, which means it’s a small, lightweight and non-intrusive Java library that is easy to include in your development environment. It has been designed for performance and scalability and has been proven to handle large networks of data (100+ millions of nodes, relationships and properties). Neo is a newly founded open source project, but the software is robust. It has been in commercial production in a highly demanding 24/7 environment for almost four years and has full support for enterprise-grade features such as distributed ACID transactions, confi gurable isolation levels and full transaction recovery. But so much for sweet talk, let’s cut to some code!

Model and Code


In the Neo model, everything is represented by nodes, relationships and properties. A relationship connects two nodes and has a well-defi ned, mandatory type. Properties are key-value pairs that are attached to both nodes and relationships. When you combine nodes, relationships between them and properties on both nodes and relationships they form a node space – a coherent network representing your business domain data.
This may sound fancy, but it’s all very intuitive. Here is how a simple social network
might be modeled:

Figure 1

Figure 1: An example of a social network from a somewhat famous movie. Note the different type on the relation between Agent Smith and his creator The Architect.

Note how all nodes have integer identifi ers and how all relationships have a type (KNOWS or CODED_BY). In this example, all nodes have a “name” property. But some nodes have other properties, for example, an “age” property (node 1) or a “last name” property (node 3). There’s no overall schema that forces all nodes to look the same. This allows Neo to capture so-called semi-structured information: information that has a small amount of mandatory attributes but many optional attributes. Furthermore, the relationships have properties as well. In this example, all relationships have an “age” property to describe how long two people have known each other and some relationships have a “disclosure” property to describe whether the acquaintance is secret.
Working with nodes and relationships is easy. The basic operations are as follows:

Figure 2

This is an intuitive representation of a network and probably similar to many other implementations that want to represent a network of data in an object-oriented language. It’s worth noting, however, that relationships in this model are full-blown objects and not just implicit associations between nodes. If you have another look at the social network example, you’ll see that there’s more information in the relationships between nodes than in the nodes themselves. The value of a network is in the connections between the nodes and Neo’s model captures that.

Creating a Node Space

And now, finally some code. Here’s how we would create the Matrix social network
from figure 1:

Transaction tx = Transaction.begin();
EmbeddedNeo neo = ... // Get factory
// Create Thomas ’Neo’ Anderson
Node mrAnderson = neo.createNode();
mrAnderson.setProperty( ”name”, ”Thomas Anderson” );
mrAnderson.setProperty( ”age”, 29 );
// Create Morpheus
Node morpheus = neo.createNode();
morpheus.setProperty( ”name”, ”Morpheus” );
morpheus.setProperty( ”rank”, ”Captain” );
morpheus.setProperty( ”occupation”, ”Total bad ass” );
// Create a relationship representing that they know each other
mrAnderson.createRelationshipTo( morpheus,
   MatrixRelationshipTypes.KNOWS );
// Create Trinity, Cypher, Agent Smith, Architect similarly

As you can see in the code above: It is rather easy to construct the node space for our Matrix example. And, of course, our network is made persistent once we commit.

Traversing a Node Space

Now that we know how to represent our domain model in the node space, how do we get information out of it? Unlike a relational database, Neo does not support a declarative query language. Instead, Neo provides an object-oriented traverser framework that allows us to express complex queries in plain Java. Working with the traverser framework is very straight-forward. The core abstraction is, unsurprisingly, the Traverser interface. A Traverser is a Java Iterable that encapsulates a “query” – i.e. a traversal on the node space such as “give me all Morpheus’ friends and his friends’ friends” or “does Trinity know someone who is acquainted with an agent?”. The most complex part of working with a Traverser is instantiating it. Here’s an example of how we would create a Traverser that will return all the (transitive) friends of the “Thomas Anderson” node of the example above:

// Instantiate a traverser that returns all mrAnderson’s friends
Traverser friendsTraverser = mrAnderson.traverse(
    Direction.OUTGOING );

Here we can see that traversers are created by invoking the traverse(...) method on a start node with a number of parameters. The parameters control the traversal and in this example they tell Neo to traverse the network breadth-first (rather than depth-fi rst), to traverse until it has covered all reachable nodes in the network (StopEvaluator.END_OF_NETWORK), to return all nodes except the fi rst (ReturnableEvaluator.ALL_BUT_START_NODE), , and to traverse all OUTGOING relationships of type KNOWS. How would we go about if we wanted to list the output of this traversal? After we’ve created a Traverser, working with it is as easy as working with any Java Iterable:

// Traverse the node space and print out the result
for ( Node friend : friendsTraverser )
    System.out.println( friend.getProperty( “name” ) + “ at depth “ +
        friendsTraverser.currentPosition().getDepth() );

Running the traversal above on the Matrix example would yield the following out-

$ bin/run-neo-example
Morpheus at depth 1
Trinity at depth 1
Cypher at depth 2
Agent Smith at depth 3

As you can see, the Traverser has started at the “Thomas Anderson” node and run through the entire network along the KNOWS relationship type, breadth fi rst, and returned all nodes except the fi rst one. “The Architect” is missing from this output since the relationship connecting him is of a different type, CODED_BY. This is a small, contrived example. But the code would work equally well on a network with hundreds of millions of nodes, relationships and properties. Now, let’s look at a more complex traversal. Going with our example, suppose that we wanted to fi nd all “hackers of the Matrix,” where we defi ne a hacker of the Matrix as any node that you reach through a CODED_BY relationship. How would we create a Traverser that gives us those nodes? First off, we want to traverse both our relationship types (KNOWS and CODED_BY). Secondly, we want to traverse until the end of the network and lastly, we want to return only nodes which we came to through a CODED_BY relationship. Here’s the code:

// Instantiate a traverser that returns all hackers of the Matrix
Traverser hackerTraverser = mrAnderson.traverse(
    new ReturnableEvaluator()
        public boolean isReturnableNode( TraversalPosition pos )
            return pos.getLastRelationshipTraversed().
                isType( MatrixRelationshipTypes.CODED_BY );
 Direction.OUTGOING );

Now it’s getting interesting! The ReturnableEvaluator.ALL_BUT_START_NODE constant from the previous example was actually a convenience implementation of the ReturnableEvaluator interface. This interface contains a single method and you can supply a custom implementation of it to the traverser framework. It turns out that this is a simple but powerful way to express complex queries. Setting aside the anonymous inner class cruft surrounding the code in bold, we basically pass in a snippet of code that checks whether we traversed a relationship of type CODED_BY to get to the current node. If this statement is evaluated to “true” then the current node will be included in the set of nodes that is returned from the traverser.
When executed with a simple print loop, the above code prints the following:

$ bin/run-neo-example
The Architect

StopEvaluators work the same way. In our experience, writing custom evaluators
is very easy. Even the most advanced applications we have developed with Neo – applications that traverse extremely large and complex networks – are based on
evaluators that are rarely more than a few lines of code.


Neo is not a silver bullet and some areas needs to improve, for instance tools, standardizing the model and a query language. However, if your data is naturally ordered in a network or is semi-structured or you just need to go truly agile, give the Neo database a run for your money. We hope you find it, as we do, to be an elegant and fl exible alternative that is both robust and fast.

Emil Eifrém, Neo Technology
Björn Granvik, Jayway


Neo specification

Originally published in JayView.

JavaOne 2006 – almost in hindsight

October 1, 2006

JavaOne 2006 might be only a few months old, but a summer feels a like lot longer than that. What were the trends and are they still hot?

JavaOne conferences seem to come in three different flavours – announcement, middle and final. Several “new” technologies made their finale this year; a graduation year. EJB 3, Java EE 5 etc. were all more or less announced two years ago. Even the examples were in some cases identical to then: mature and perhaps a bit boring. However, EJB3 will play a role in many projects so “mature” is a good sign.
There were also some surprises and rejuvenation, mostly on the open source side even though Sun managed to squeeze in some surprises.
What happened in the Java SE and EE area? Let us check out some trends and
see what we can make of them.

AOP – still stillborn?

Sun has had a very cautious approach to aspect oriented programming. Two years ago on JavaOne, a large, attentive audience tried to understand what Sun had to say on the subject. The answer from the panel on aspects was like something taken from an old police station series on TV:
– Let’s be careful out there!
Well, the “fireside chat” this year with Sun aluminaries and, of course, James Gosling proved to be more of the same.
The EJB 3 gang also had something similar to say during the conference – along the lines of: “why aspects, when interceptors will do the job”. Interceptors could be described as poor man’s aspects using interfaces and proxies to catch calls to methods. Those of you with good memory banks will recall the Spring folks doing the same rhetorical twist a couple of years ago. This was of course before they saw the light and got the AspectJ guy on their payroll. Smart move. My guess is that we will hear Sun speaking more on using aspects instead of interceptors some two years from now. In the mean time, Spring will lead the way on aspects and real life programming.


In Java 6 there will be JVM support for hooking in your choice of scripting language through the standard Java Scripting API. The contestants are already many and include such names as JavaScript, Ruby, Groove, BeanShell and a bunch you probably haven’t heard of. The driving force behind this is twofold; first the amount of glue code needed in Java. It is sometimes just too much code for simple stuff or should I say stuff that should be simple. Secondly, the competition in the language area is heating up. Ruby on Rails is a good example of powerful and fast programming for setting up a website. Their 15-minute video does compete with successful TV ads. You know the ones that make you buy something just because it is such a good deal.
My forecast is that we will see a scripting case where the language and solution just makes sense. Which one? Who knows. Unfortunately we might see a lot of bad examples before the right one comes along.

Java 7 – Dolphin

The Mustang (Java 6) might not be on the loose just yet, but I had to check out the next mammal in the Java sequel – Dolphin. Java 7 was apparent in several sessions, but the common denominator was the vagueness. The speakers used phrases like “thinking about”, “might” etc. Still it was interesting and some of the features are needed.

XML in Java

For instance, what about supporting XML in Java more directly where the XML would be a data type?

Current way

String name = “Björn”;
XML xml =
      <name>{ name }</name>
      <age>{ 25 }</age>

Alternatively the XML could be expressed as, less verbose:

Possible new way

XML xml = #person
           #name { name }
           #age { 25 };

Less verbose but will it catch on? I find XML so mind numbing that I am ready to try just about anything.

Super Packages

In large projects the notion of packages does not scale. How can we use a library of classes without exposing internal classes best left in the dark? There are several aspects to this problem, from distribution format to lifecycle. JSR 294 addresses the modularity issue even if it includes a certain NIH factor (Not Invented Here). OSGi, for example, already has a similar mechanism. But for now let us just examine how we can handle classes and their visibility, i.e. “super packages”.
Here’s a code snippet on what a definition for a super package would contain.

super package com.sun.myModule {
  // Classes visible outside of this super module.
  export com.sun.myModule.myStuff.*;
  export com.sun.myModule.yourStuff.Interface;
  // Not visible outside

As you can see super packages is essentially a package of packages. This way we can only expose classes on a need to know basis. Solving this with current Java syntax is tricky. Some even resort to removing any JavaDoc on the “internal” classes to obscure their usage.

Java Module System

The next big thing is aimed at the JAR file format that, let’s face it, doesn’t scale and has poor dependency enforcement especially when it comes to versioning. What is needed is a system for resolving references between “Java modules” where each module has meta-data about its version and related resources. This is exactly what JSR 277 defines and yes, it sounds very much like shared assemblies in .NET. However, combining super packages with Java modules forms a powerful way of distributing, versioning and managing components. I like.
These three examples on what we might see in the next-next version of Java are likely to change. Nevertheless I found them interesting.

Ajax, Ajax …oh and Ajax

If there ever was a message being touted on the rooftops of Moscone Center in San Francisco, it is Ajax. The intro sessions were packed, standing room only even on the repeat sessions. Ajax is simple at its lowest level – some JavaScript to query the backend and voilà, you’re done. At the face of it, this is deceptively easy. However, once you have done that simple first try, you quickly get into more questions. Where should we put retrieved data on the client? How do we handle lots of Ajax calls?
Will someone mesh up my data?
Performed correctly though, you do get neater web clients and that is actually an attractive notion. The flip side of this, though, is all the hype. Everything from the Colgate inspired salesmen – yes, they were there with their surreal suntan – to the former Sun execs doing the IPO dance. “Next episode: Will they get cash in before the expiry date?”
Ranting aside, Ajax is here and we all better start to learn it because it does matter. And it does put an amicable face on your web site.

Far out Java container

Two years ago the chosen Java compatible container was …a cool BMW. This year it was an autonomous dune buggy named Tommy with software all in Java. It was entered in a challenge to cross a patch of the American desert – without a driver. It looks a bit like Flash Gordon with four tires instead of rockets, built in somebody’s garage. I must say Tommy spurred my interest more than the BMW, but then again you would not look too cool driving it.
Sun to ground control…


It was quite evident that Sun has a new version of NetBeans out and that it does support the latest Java EE version. And, yes, it does look good. And the price tag is quite right – zero. I just wish that it wasn’t being plugged at most every Sun keynote etc. The audience did get the message and we will check it out. Boy, that IDE market is tough!

Java 6

“Java 6 is here” was one part of the message and “compatibility matters” was the other. Sun CEO Jonathan Schwarz has been going the open source route a lot and many applications are being made OS. Even Java itself could be made open some day. The “When” is still open though. Personally, this isn’t such a big deal for me. Sun has been, for the most part, good at stewarding Java. They have also managed to keep the code running, mostly, between versions. Opening it could be a problem, but also very interesting. Making compatibility a part of the arguments makes sense in such a scenario. Let us hope it stays just that – compatible.

“We want your help”

Making your software open source does change your way of thinking. Sun has promoted a lot of its stuff into the OS arena. Good, I say! It is only logical that participation, from you and me, is seen as a great opportunity. Many of the applications going open, Project Glassfish etc, often ended their sessions with a “we need you”. Maybe Sun’s bad economy has a role in this? To me it doesn’t matter. I like it and I am eagerly waiting to see what the future might bring.

Summing up

JavaOne is a lot more than you can cram into an article such as this, but let us not be stopped by that when summing up. EJB 3 has come of age. This was their prom night and things were basically just as they said they would be two years ago. Keep an eye on it. It will play a role in many projects. Spring will deliver aspect oriented programming, not Sun, and it will go from hot to “fun and here”.

Ajax is everywhere and you will use it. You might as well pick up a good book on this subject and start hacking.

Java 6 will deliver lots of goodies. Will we get to use it? Or is your project stuck in Java JDK 1.4.2 unable to upgrade? Let us hope not. Check out scripting! It will change with Java 6. Who can tell which language will affect it big time? Someone will. Check out your local Javaforum to see if you can predict the winner.
Keep coding and may your code be with you.

Originally published in JayView.

Javaforum Malmö – now alive and kicking!

March 4, 2006

The Javaforum Malmö (re)opened at Jayway’s new office at Malmö one afternoon in March. The meeting set a new record for that conference room and filled it with ease.

Some forty people checked in for this, the first Javaforum event in Malmö for several years. The audience was a mosaic of around fifteen companies. The forum handled subjects like project methodology Scrum, application servers like T4 and Geronimo, news of the Spring framework and finally sensory networks using tiny Java thingies. The sessions covered the small and the big, from the sublime to the surreal – and all in Java.

Feel like more Java? Check for up to date information.
Welcome next time around!

Originally published at Jayway Team Blog.

The scent of freshly cut code

March 4, 2006

I just had to be there. My friend was starting out anew. A fresh idea, a start up – I was hoping it was going to be good. I was not disappointed.

I could smell it straight away when I en- tered the small office – the fresh and raw scent of newly cut code. Two men sat in front of their computers. One of them smiled at me.
Jon Hauksson and I had hacked code, back to back, against a fast moving dead- line in a distant project. He had given up his daytime job as a consultant at a respectable firm. Now, he was shoulder deep into his new venture, bringing au- dio books to the mass market of mobile phones.

Jon and Andreas in a classical image – happy men with computers.

What had he learned? What is it like to develop a business to consumer app running on all those different Java mo- biles? A lunch later we were heading back into his office at Ideon – an incu- bator for new companies. Mobiles were strewn across their tables. Blank walls. Shelves were more or less empty. It all breathed start up and focus – get that app out trough the door!
We sat down and Jon pulled an SQL- statement to calculate the last 24 hours of commerce.
– Yeah, rising, he said and smiled.

It has been said that developing a Java ME solution for all those different mo- biles is difficult and time consuming. Is it that hard?
Yes… but it has improved, especially for Nokia and Sony Ericsson. But some other brands can be trickier. We usually code for the Sony Ericsson emulator and then bring the code over to the Nokia one. Usually there aren’t any problems here. From here on we test the other brands. If we hit a problem we either try to solve it or we have to program our way around it. It used to be more difficult. The situation has improved in the last year. Especially Sony Ericsson is working hard to make it better.

Does this mean you have different sets of your code?
We have four variants of the result- ing jar/jad file and that is enough for us. Fortunately NetBeans has pre-process- ing that makes life easier and allows us to only work with one code base.

Without any time to think, what are the top three hard to learn lessons that you can you share with us?
1) Testing, testing, testing is extremely important. Many unforeseen things can happen in areas like operator handling, memory management, the mobile network and different quality issues – a lot can happen and usually does.

Bokilur’s application playing a book.

2) The Java settings are often not in- stalled correctly on the terminal from start. The user can download the set- tings for Java from the operator but sometimes these settings are not cor- rect. We’ve tried to add a guide that pops up when this happens but fiddling at this low level is not a pleasant experience for the customer. Some call our support number and we try to help them, but it is still a problem.
3) Streaming solution
The network, GPRS or 3G, has gaps and this affects our application. We’ve had to develop a streaming solution with a five-minute buffer so that you can listen to our books while driving. We’ve even tried it on trains, on the X2000 for instance, which reportedly has its share of problems when it comes to mobile network connections. But to our relief it worked there as well.

Would you do use Java again?
For sure, it is easy to work with and even though it has problems, it is getting better.
Although, we’ve chosen to imple- ment a number of our own frameworks even though there might be a good open source solution available. It is im- portant to us to keep our code size to a minimum and we usually don’t want the whole nine yards.

Any examples of when you’ve had to roll your own solution?
Well, we implemented a GUI frame- work a bit like J2ME Polish. This way we can target our application’s needs and keep control. We’ve also developed an RMI like protocol to interact with the servers. There are, however, a lot of open source solutions to try out.
We also have a simple debug solu- tion where the phone sends back debug information to the server so it can be logged. Very handy. And of course simple stuff like text han- dling.

Would you do anything differently now when you look back? How long has it been anyway?
One and a half year. Ha-ha, it feels a lot more! We are trying to improve the user friendliness of the entire browsing and purchasing process in the applica- tion. That’s user interaction, but I can’t think of anything technical. Keeping the program easy and intuitive is important to us.

Any practical hints on starting a prod- uct company in the mobile space?
Get the operators as partners. For a data rich service like ours, you need to get a deal without traffic charges. Other- wise it will be hard to develop a pricing model that is attractive to your targeted customers. Besides, the operators have a strong interest to bring in attractive services into their portals. They need more than just games and ring tones.
Oh, and one more thing, keep track of the top selling mobile phone list. That way you know which models to target during testing.

What about the future?
Our business idea is to be the best mobile literary service both here and abroad. You can expect us to go after the Nordic countries.

Has the “market” for new companies changed?
Yes. During the last year more of my former colleagues have started compa- nies of their own! It’s even come to the point where venture capital has a prob- lem finding something to invest in…

During our interview Bokilur’s other programmer, Andreas Melvanius, kicks in.
– We have a problem.
It turns out that a single Nokia phone is thrown into silence. A quick round of questions and they agree on some new tests.
After a while Jon asks whether I’d like a tour of the ”pavilion”. Walking in the corridors, we find a new com- pany behind every door. I cannot help but being impressed, such diversity. We stumble upon solutions for detecting schizophrenia, simulation of combus- tion engines, cameras that can see in fog and darkness and so on. I leave the barracks feeling elated – I too want to start up my own company.

PS. Curious about the Bokilur applica- tion? Check out
PPS. Bokilur later changed their name to
Originally published in JayView.

Bert Rubaszkin, the quote master

March 3, 2006

Bert R working on his sauna raft.

Bert Rubaszkin is the Chief Technologist, CTO to us mere programmers, for Sun Sweden. He is often quoted in such computer mags as Computer Sweden. Questions cover a lot of ground, from Sun servers being tossed from tall buildings to the future of some new Sun technology.

Bert comes across as the wise ol’ programmer who has seen most of what there is to see. Always in a good mood, I find it hard to see him loosing his cool despite being probed with difficult questions. But first, let’s start off with a simple question.

What’s it like being asked to comment on just about everything under the Sun?
There are two problems. If you have worked long enough, people expect you to be an expert at everything. The truth is that you only know a fraction of what is needed, but I do know who to ask.
Another major problem are magazines. They are mainly interested in either new information or mud raking. Some are seriously interested in what has happened, others… they only want to talk about things that don’t work. But it is my job. I just try and answer the question. Sometimes it’s actually flattering that people think I know so much.

Open Source has become quite important to Sun of late. As I understand it, you’re going to open source all of your code. Why?
It is correct that we plan to release all our source code, but I would like to point out that there is no time plan saying that at a certain point in time all the code will be released. The most important open source project for us now is OpenSolaris. The cat is, so to speak, out of the bag. We have been discussing this for several years. Once we took the decision, we needed to wash the code and make sure we had the copyrights to do this.
The effect is quite positive. We now have a community around OpenSolaris 10, and version 11 will contain features that wouldn’t otherwise have made it in there without the community. And bug tracking of course. Plus it defused the Linux versus Solaris debate. We can now discuss things with our customer in a much more interesting way. Someone at Sun said “It’s not about the code stupid, it’s about the dialogue!”
Internally it has also been a vitamin injection. Involvement is a lot higher. Everything we wished for has happened.
There was a suspicion that we would only let go of the old stuff. Instead we released innovative bits like DTrace etc. This has the positive side effect of code going cross operating systems. For example: a group of people are working on a distribution of FreeBSD, which will include DTrace. That’s one example of code becoming de facto standard outside the GPL arena.
Other high priority Open Source projects from Sun are OpenOffice and Netbeans. The end result of all of this is that we get more competitive. We get better tools and the risk of stagnation disappears.

What is the story with CDDL (pronounced “cuddle”), Common Development and Distribution License? Wasn’t the Mozilla open source license good enough for Solaris?
I’m no expert on licensing but CDDL is based on Mozilla except for the one paragraph that states that the license can be changed at any given moment. It doesn’t even require any approval. Our big clients don’t accept this single difference.

Most readers of JayView already know about the resurgent Javaforum. Apart from that, what is the status of Java in Sweden? Can you spot any trends?
Trends are meta trends and so… What I see from my angle is that Java still dominates the server side. Microsoft probably would have hoped that it would have made a more visible dent in the server side of things. We see very little impact from Dotnet among our customers.
The meta trend is SOA and web services and how to interact in a mixed environment such as Java and Dotnet. We work together with Microsoft on this and more is going to happen. J2EE dominance on the server is not threatened. From a customer point of view, it’s good to be able to encapsulate web services in the different worlds and that there is a consensus on how to interact between the two.
Looking on Java end-to-end, we’ve only started. There have been problems with limited midp hardware and different profiles between small devices. These mobiles will be more powerful. The problem of coping with differences between small platforms will be forgotten in 5 years.
We’re not there yet… but the APIs will be the same and so on.

Q: Sun has had this ambiguous relationship where it both owns Java, but also lets it play with the programmers on the block. It’s not hard too find examples of changing times. Take for example the Apache Harmony project (an open source implementation of Java 5 SE) or the interest in dynamic languages and the effort to provide support for these in the JVM in Java 6 SE. Are Sun’s dual roles over Java changing?
There is a concept within Sun which refers to “church and “state” as far as Java is concerned. Some work within the state it’s no more difficult than that. Java is a big investment for Sun and we want to see it spread, to complete its potential. It’s important that standards are kept and that nobody can corrupt them. Journalists typically want to make a big thing out of this. Our customers think JCP (Java community process) works. Maybe it is true that Sun has been able to keep the two roles apart and at the same time have products.
One strategy that we have followed is that every time there has been a split view, a 50-50 vote, on any subject within a JCP committee, we haven’t used our deciding vote. We’ve actually let the other point of view “win”. I believe that off the record even IBM will agree to this. The common denominator for those who complain, is that they haven’t actually looked into the JCP. It’s rather quick work nowadays. In the beginning some smaller companies complained, but that has been resolved. It’s rare to hear someone say “Damn Sun”.

Java is more of an ecosystem than just a language. What’s the status of that system today? And where is it going?
A flora of tools and frameworks has grown up, but doesn’t necessarily run within the JCP. Java is sort of the centre and further out are things like Ajax. You can go far out from the middle of this system and that is a good thing. James Gosling himself thinks this is very good. He was asked what happens if something can be done smarter in a different language to Java.
“I’m all for it!” That goes for most Sun folks. There is no single person in the world that can answer all the questions within that field – that’s good.

At the far end of that ecosystem I actually place Microsoft and C#. Some of the recent improvements in Java were inspired by the “other camp”, just as ideas and open source flow in the other direction. To me this is “competing symbiosis”. Take away one and the other would stagnate, much like Internet Explorer vs. Netscape. What’s your view on this?
It’s good with competition, but I’m critical of the fact that Microsoft with a budget exceeding some countries’ GNP cannot produce anything better than a pure Java clone, C#, with some minor improvements. Pitiful! But then again, competition is good. The Java community has a keen ear for improvements.
I have to give Microsoft credit for building tools that are easy to use. But this is partly at the expense of power and has meant that some things have not been developed. For instance, I’m totally convinced that Excel is a threat to mankind. People make decisions based on small errors that are impossible to debug.
If we at Sun can maintain the power and at the same time increase ease of use.

What is the best/worst thing that has happened in Java?
The best thing that is happening right now it is that we succeeded with Javaforum. It’s an embryo for an independent organisation. Tremendous. On a more technical level there are lots of good things happening. One thing that I like, but I would not classify it as “tremendous”, is annotation in Java 5.
The worst thing that has happened in Java was Microsoft signing an agreement, only to break it again immediately after. That wasn’t nice – not funny at all. But what was funny was them being made to pay.

Looking into your crystal ball, where is Java heading? Is there a device that will _not_ hold a JVM?
Different times place different demands on tools. Java’s dominance will be broken sooner or later. For instance, there are more network-oriented languages at academic level. Old languages don’t die. Look at how much Cobol code there is.
Java was the fist programming language for the net and shows no signs of old age. I would guess that Java will have a lifecycle similar to Cobol. It will take hundreds of years before it disappears – ha ha.

It has been said that Java will be the Cobol of the future. I dare you to make a guess at when that will be!
20 years from now it will be legacy. Not within five.

I once managed to make James Gosling speechless. He was taking questions from the audience and I managed to sneak in _”What’s the meaning of Java, as in ‘what’s the meaning of life’?”_ What’s your take on that?
It was the first programming system built for the net, just as NFS was the first file system for the net. That was the meaning.

Dang, I thought that at least the last question would stomp Bert.

Originally published in JayView.

Jonas Bonér, your regular AOP guy

February 4, 2006

Jonas is still standing when I meet him at the Öredev conference to discuss an interview. Even though a cold is doing its best to bring him to his knees. He still has a ‘roundthe-globe’ trip to Tokyo to do and the following sprint to reach JavaOne in Japan another one of those Swedes going places.

Jonas Bonér is the founder of AspectWerkz, an aspect oriented framework without the need for special compiler. It later joined forces with AspectJ, which could be described as the Father Christmas of aspect oriented programming in the Java world. He used to work for BEA in Sweden, but has recently joined Terracotta, San Francisco.

Jonas you’re a long term AOP advocate. When and how did you start with aspects and, perhaps most importantly, why?
I started using AspectJ back in 2001 and I was immediately very fascinated by the concepts of AOP, its power, beauty and ability to solve really hard problems in such an elegant way. I have always valued clean design and simplicity and in AOP I found a way of letting design and concepts map directly to actual code.
I started by using ‘policy enforcement aspects’, e.g. AspectJ’s declare error/warning, logging and tracing aspects, as part of the build process. After a while I gradually started to use it for more advanced things, such as security, transaction demarcation, persistence, caching, “unit of work”, design patterns etc. for better modularity, reusability, maintainability and simplicity.

Why did AspectWerkz and AspectJ merge?
The short answer is that the users needed a standard. But let me elaborate a little bit. During recent years the AOP landscape for Java has flourished. When a technology is new and immature it is usually a good thing to have many frameworks/products competing, since they can inspire and push each other and the technology even further, but when the technology matures it starts to become a problem. For example, the users might end up not knowing which framework, syntax and semantics to start using, since they do not know which ones will even be around a year later. If you see this happen a lot, like it did with AOP, then there is a need for standardization.
During 2004, AspectWerkz and AspectJ slowly converged, but the two projects were still focused on different areas and had complimentary strength. We had some discussions and both teams saw the need for standardization and felt that both projects as well as the user community would benefit from unifying the projects, roadmaps and teams instead of competing. So we decided on the merger that was announced in the beginning of 2005 and was completed with the release of AspectJ 5 at the end of last year.

So where do you think we’re heading for in the future, as far as AOP is concerned?
I am confident that AOP will play an important role in the future of enterprise application development. Many vendors are already starting to embrace AOP as an enabling technology for separation of concerns, transparency, simplicity etc. For example Terracotta DSO, JBoss EJB 3, TopLink, Spring etc.
These are all examples of vendors using AOP to layer infrastructure services on the user’s domain models in a transparent fashion. That is great, but I hope that it won’t stop with that, my wish is that users will see the value of using AOP in their application specific code, in order to achieve more modular design, code that is more easily written, understood, reused and maintained. In some ways AOP had a healthy backlash over the last year, which has removed most of the hype around it. It has matured, has a standard in AspectJ 5 and is ready for real-world deployments.

Tool wise it seems that we have several things in place in Eclipse/AspectJ/ AJDT to get a good grip on aspects. Examples: being able to see what code an aspect affects, diff engines to compare changes to an aspect etc. Do we have all we need?
The AJDT team has done a tremendous job and I think that we have already come a long way, even though more work can of course be done and needs to be done in this area. But while this is true for static views of the system, like AJDT is showing, there is almost no support for monitoring and introspecting the runtime view of the system, something that is getting more important now when the usage of dynamic AOP is becoming more widespread, which has the ability of redefining the behavior of the system at runtime. This is an area in which I hope that the academic community will contribute more in the future.

Can you see anything that needs to be fixed or added before we see more widespread adoption of AOP?
I think that the most important thing now is education trainings, books, articles etc. We need to try to reach beyond the group of “expert” developers. Another thing is a standard library. We need an ADK (Aspect Development Kit) in the same sense that Java has its JDK. I am involved in an attempt to gather and develop a set of library aspects for AspectJ 5, and we’ll see where that leads. Finally I hope to see more best-practices, patterns as well anti-patterns emerge.

Terracotta, the new company you work for, has a distributed cache product. What’s so cool about it?
It is completely transparent. Meaning that zero code changes are required to cluster an arbitrary J2EE application. Our only requirement is that you have to write a correct multi-threaded application. We then take that multi-threaded single VM application and a tiny bit of XML configuration to declaratively specify what should be shared, lock semantics etc., and turn it into a multithreaded multi-JVM application. These are the features I find most interesting:
* True transparency.
* Noserialization.We only send the actual changes to an object graph (the delta) as well as keep track of who is referencing who, so we can send these changes only to the nodes that need it, e.g. lazily.
* Object identity is preserved. We maintain Java’s “pass-by-reference” semantics, so regular object references work, which means that you can use your domain model the way you have designed it, as well as traditional OO design patterns etc. without a need to think about distribution, caching or clustering. • Coordination and memory manage-ment. We maintain the Java Memory Model transparently throughout the cluster, including distributed coordination, for example wait(), notify(), synchronized() etc., distributed garbage collection etc. We work as a virtual heap, meaning that you can for example run an application with 200G heap on a machine with 4G of RAM in which we then page in and out the heap on a demand basis.

How does it work?
We use AOP technologies to adapt the application at class load time. In this phase we extend the application in order to ensure that the semantics of Java are correctly maintained across the cluster, this includes object references, thread coordination, garbage collection etc. For example, we extend the semantics of a regular synchronized block, or actually maintain them across the cluster, by taking a global lock for the object instance that you are synchronizing on right before entering the block and releasing it right after exiting the block. You can declaratively define the exact semantics for the block, e.g. read, write, concurrent etc.
Another example is the invocation of notifyAll(). Here we turn this call into a cluster wide notification, which will open up for all nodes to contend for the lock.
The architecture is hub and spoke based, meaning that we have a central server which manages the clients, we use TCP/IP so the server just needs to be somewhere on the network. The client in this case is simply your regular application together with the Terracotta JAR. People always ask whether the server is not a single-point of failure, but we have a SAN-based solution to support fail-over in an active-passive manner. You can have an arbitrary number of (passive) servers waiting in line and upon failure the selected one will pick up right where the master server that has crashed left off.

Finally, as an active open source contributor, do you have anything up your sleeve for the future?
I have some ideas, but nothing I can talk about now. Stay tuned…

Something tells me that we will hear more from Jonas Bonér.
His cold? Well the trip actually made it go away. He made JavaOne with half an hour to spare. I guess it’s that Viking blood.

Originally published in JayView.