Posts Tagged ‘java’

Neo – a netbase

February 1, 2007

Neo is a network-oriented database for semi-structured information. Too complicated, let us try again. Neo handles data in networks – nodes, relationships and properties – instead of tables. This means entirely new solutions for data that is diffi cult to handle in static tables. It could mean we can go agile all the way into the persistence layer.

The relational database represents one of the most important developments in the history of computer science. Upon its arrival some 30 years ago, it revolutionized the way the industry views data management and today it is practically ubiquitous.

In fact, it is so taken for granted that we, as an industry, have stopped thinking. Could there be a better way to represent and store our data? In some cases the answer is – yes, absolutely. The relational database is showing its age. Some telltale signs:

  • The mismatch between relational data and object oriented programming.
  • Schema evolution – updating your data model when the domain model changes – is just so much manual labor.
  • Semi-structured or network-oriented data is diffi cult to handle.

The Neo Database

Neo is a database that is built with a different philosophy. Where the relational database squeezes all data into static tables, Neo uses a fl exible and dynamic network model. This model allows data to evolve more naturally as business requirements change. There’s no need for “alter table…” on your production databases after you introduce a new version of the business layer and no need to rewire and migrate your O/R mapping confi gurations. The network will evolve along with your business logic. This spells agility.

Neo is an embedded persistence engine, which means it’s a small, lightweight and non-intrusive Java library that is easy to include in your development environment. It has been designed for performance and scalability and has been proven to handle large networks of data (100+ millions of nodes, relationships and properties). Neo is a newly founded open source project, but the software is robust. It has been in commercial production in a highly demanding 24/7 environment for almost four years and has full support for enterprise-grade features such as distributed ACID transactions, confi gurable isolation levels and full transaction recovery. But so much for sweet talk, let’s cut to some code!

Model and Code


In the Neo model, everything is represented by nodes, relationships and properties. A relationship connects two nodes and has a well-defi ned, mandatory type. Properties are key-value pairs that are attached to both nodes and relationships. When you combine nodes, relationships between them and properties on both nodes and relationships they form a node space – a coherent network representing your business domain data.
This may sound fancy, but it’s all very intuitive. Here is how a simple social network
might be modeled:

Figure 1

Figure 1: An example of a social network from a somewhat famous movie. Note the different type on the relation between Agent Smith and his creator The Architect.

Note how all nodes have integer identifi ers and how all relationships have a type (KNOWS or CODED_BY). In this example, all nodes have a “name” property. But some nodes have other properties, for example, an “age” property (node 1) or a “last name” property (node 3). There’s no overall schema that forces all nodes to look the same. This allows Neo to capture so-called semi-structured information: information that has a small amount of mandatory attributes but many optional attributes. Furthermore, the relationships have properties as well. In this example, all relationships have an “age” property to describe how long two people have known each other and some relationships have a “disclosure” property to describe whether the acquaintance is secret.
Working with nodes and relationships is easy. The basic operations are as follows:

Figure 2

This is an intuitive representation of a network and probably similar to many other implementations that want to represent a network of data in an object-oriented language. It’s worth noting, however, that relationships in this model are full-blown objects and not just implicit associations between nodes. If you have another look at the social network example, you’ll see that there’s more information in the relationships between nodes than in the nodes themselves. The value of a network is in the connections between the nodes and Neo’s model captures that.

Creating a Node Space

And now, finally some code. Here’s how we would create the Matrix social network
from figure 1:

Transaction tx = Transaction.begin();
EmbeddedNeo neo = ... // Get factory
// Create Thomas ’Neo’ Anderson
Node mrAnderson = neo.createNode();
mrAnderson.setProperty( ”name”, ”Thomas Anderson” );
mrAnderson.setProperty( ”age”, 29 );
// Create Morpheus
Node morpheus = neo.createNode();
morpheus.setProperty( ”name”, ”Morpheus” );
morpheus.setProperty( ”rank”, ”Captain” );
morpheus.setProperty( ”occupation”, ”Total bad ass” );
// Create a relationship representing that they know each other
mrAnderson.createRelationshipTo( morpheus,
   MatrixRelationshipTypes.KNOWS );
// Create Trinity, Cypher, Agent Smith, Architect similarly

As you can see in the code above: It is rather easy to construct the node space for our Matrix example. And, of course, our network is made persistent once we commit.

Traversing a Node Space

Now that we know how to represent our domain model in the node space, how do we get information out of it? Unlike a relational database, Neo does not support a declarative query language. Instead, Neo provides an object-oriented traverser framework that allows us to express complex queries in plain Java. Working with the traverser framework is very straight-forward. The core abstraction is, unsurprisingly, the Traverser interface. A Traverser is a Java Iterable that encapsulates a “query” – i.e. a traversal on the node space such as “give me all Morpheus’ friends and his friends’ friends” or “does Trinity know someone who is acquainted with an agent?”. The most complex part of working with a Traverser is instantiating it. Here’s an example of how we would create a Traverser that will return all the (transitive) friends of the “Thomas Anderson” node of the example above:

// Instantiate a traverser that returns all mrAnderson’s friends
Traverser friendsTraverser = mrAnderson.traverse(
    Direction.OUTGOING );

Here we can see that traversers are created by invoking the traverse(...) method on a start node with a number of parameters. The parameters control the traversal and in this example they tell Neo to traverse the network breadth-first (rather than depth-fi rst), to traverse until it has covered all reachable nodes in the network (StopEvaluator.END_OF_NETWORK), to return all nodes except the fi rst (ReturnableEvaluator.ALL_BUT_START_NODE), , and to traverse all OUTGOING relationships of type KNOWS. How would we go about if we wanted to list the output of this traversal? After we’ve created a Traverser, working with it is as easy as working with any Java Iterable:

// Traverse the node space and print out the result
for ( Node friend : friendsTraverser )
    System.out.println( friend.getProperty( “name” ) + “ at depth “ +
        friendsTraverser.currentPosition().getDepth() );

Running the traversal above on the Matrix example would yield the following out-

$ bin/run-neo-example
Morpheus at depth 1
Trinity at depth 1
Cypher at depth 2
Agent Smith at depth 3

As you can see, the Traverser has started at the “Thomas Anderson” node and run through the entire network along the KNOWS relationship type, breadth fi rst, and returned all nodes except the fi rst one. “The Architect” is missing from this output since the relationship connecting him is of a different type, CODED_BY. This is a small, contrived example. But the code would work equally well on a network with hundreds of millions of nodes, relationships and properties. Now, let’s look at a more complex traversal. Going with our example, suppose that we wanted to fi nd all “hackers of the Matrix,” where we defi ne a hacker of the Matrix as any node that you reach through a CODED_BY relationship. How would we create a Traverser that gives us those nodes? First off, we want to traverse both our relationship types (KNOWS and CODED_BY). Secondly, we want to traverse until the end of the network and lastly, we want to return only nodes which we came to through a CODED_BY relationship. Here’s the code:

// Instantiate a traverser that returns all hackers of the Matrix
Traverser hackerTraverser = mrAnderson.traverse(
    new ReturnableEvaluator()
        public boolean isReturnableNode( TraversalPosition pos )
            return pos.getLastRelationshipTraversed().
                isType( MatrixRelationshipTypes.CODED_BY );
 Direction.OUTGOING );

Now it’s getting interesting! The ReturnableEvaluator.ALL_BUT_START_NODE constant from the previous example was actually a convenience implementation of the ReturnableEvaluator interface. This interface contains a single method and you can supply a custom implementation of it to the traverser framework. It turns out that this is a simple but powerful way to express complex queries. Setting aside the anonymous inner class cruft surrounding the code in bold, we basically pass in a snippet of code that checks whether we traversed a relationship of type CODED_BY to get to the current node. If this statement is evaluated to “true” then the current node will be included in the set of nodes that is returned from the traverser.
When executed with a simple print loop, the above code prints the following:

$ bin/run-neo-example
The Architect

StopEvaluators work the same way. In our experience, writing custom evaluators
is very easy. Even the most advanced applications we have developed with Neo – applications that traverse extremely large and complex networks – are based on
evaluators that are rarely more than a few lines of code.


Neo is not a silver bullet and some areas needs to improve, for instance tools, standardizing the model and a query language. However, if your data is naturally ordered in a network or is semi-structured or you just need to go truly agile, give the Neo database a run for your money. We hope you find it, as we do, to be an elegant and fl exible alternative that is both robust and fast.

Emil Eifrém, Neo Technology
Björn Granvik, Jayway


Neo specification

Originally published in JayView.

JavaOne 2006 – almost in hindsight

October 1, 2006

JavaOne 2006 might be only a few months old, but a summer feels a like lot longer than that. What were the trends and are they still hot?

JavaOne conferences seem to come in three different flavours – announcement, middle and final. Several “new” technologies made their finale this year; a graduation year. EJB 3, Java EE 5 etc. were all more or less announced two years ago. Even the examples were in some cases identical to then: mature and perhaps a bit boring. However, EJB3 will play a role in many projects so “mature” is a good sign.
There were also some surprises and rejuvenation, mostly on the open source side even though Sun managed to squeeze in some surprises.
What happened in the Java SE and EE area? Let us check out some trends and
see what we can make of them.

AOP – still stillborn?

Sun has had a very cautious approach to aspect oriented programming. Two years ago on JavaOne, a large, attentive audience tried to understand what Sun had to say on the subject. The answer from the panel on aspects was like something taken from an old police station series on TV:
– Let’s be careful out there!
Well, the “fireside chat” this year with Sun aluminaries and, of course, James Gosling proved to be more of the same.
The EJB 3 gang also had something similar to say during the conference – along the lines of: “why aspects, when interceptors will do the job”. Interceptors could be described as poor man’s aspects using interfaces and proxies to catch calls to methods. Those of you with good memory banks will recall the Spring folks doing the same rhetorical twist a couple of years ago. This was of course before they saw the light and got the AspectJ guy on their payroll. Smart move. My guess is that we will hear Sun speaking more on using aspects instead of interceptors some two years from now. In the mean time, Spring will lead the way on aspects and real life programming.


In Java 6 there will be JVM support for hooking in your choice of scripting language through the standard Java Scripting API. The contestants are already many and include such names as JavaScript, Ruby, Groove, BeanShell and a bunch you probably haven’t heard of. The driving force behind this is twofold; first the amount of glue code needed in Java. It is sometimes just too much code for simple stuff or should I say stuff that should be simple. Secondly, the competition in the language area is heating up. Ruby on Rails is a good example of powerful and fast programming for setting up a website. Their 15-minute video does compete with successful TV ads. You know the ones that make you buy something just because it is such a good deal.
My forecast is that we will see a scripting case where the language and solution just makes sense. Which one? Who knows. Unfortunately we might see a lot of bad examples before the right one comes along.

Java 7 – Dolphin

The Mustang (Java 6) might not be on the loose just yet, but I had to check out the next mammal in the Java sequel – Dolphin. Java 7 was apparent in several sessions, but the common denominator was the vagueness. The speakers used phrases like “thinking about”, “might” etc. Still it was interesting and some of the features are needed.

XML in Java

For instance, what about supporting XML in Java more directly where the XML would be a data type?

Current way

String name = “Björn”;
XML xml =
      <name>{ name }</name>
      <age>{ 25 }</age>

Alternatively the XML could be expressed as, less verbose:

Possible new way

XML xml = #person
           #name { name }
           #age { 25 };

Less verbose but will it catch on? I find XML so mind numbing that I am ready to try just about anything.

Super Packages

In large projects the notion of packages does not scale. How can we use a library of classes without exposing internal classes best left in the dark? There are several aspects to this problem, from distribution format to lifecycle. JSR 294 addresses the modularity issue even if it includes a certain NIH factor (Not Invented Here). OSGi, for example, already has a similar mechanism. But for now let us just examine how we can handle classes and their visibility, i.e. “super packages”.
Here’s a code snippet on what a definition for a super package would contain.

super package com.sun.myModule {
  // Classes visible outside of this super module.
  export com.sun.myModule.myStuff.*;
  export com.sun.myModule.yourStuff.Interface;
  // Not visible outside

As you can see super packages is essentially a package of packages. This way we can only expose classes on a need to know basis. Solving this with current Java syntax is tricky. Some even resort to removing any JavaDoc on the “internal” classes to obscure their usage.

Java Module System

The next big thing is aimed at the JAR file format that, let’s face it, doesn’t scale and has poor dependency enforcement especially when it comes to versioning. What is needed is a system for resolving references between “Java modules” where each module has meta-data about its version and related resources. This is exactly what JSR 277 defines and yes, it sounds very much like shared assemblies in .NET. However, combining super packages with Java modules forms a powerful way of distributing, versioning and managing components. I like.
These three examples on what we might see in the next-next version of Java are likely to change. Nevertheless I found them interesting.

Ajax, Ajax …oh and Ajax

If there ever was a message being touted on the rooftops of Moscone Center in San Francisco, it is Ajax. The intro sessions were packed, standing room only even on the repeat sessions. Ajax is simple at its lowest level – some JavaScript to query the backend and voilà, you’re done. At the face of it, this is deceptively easy. However, once you have done that simple first try, you quickly get into more questions. Where should we put retrieved data on the client? How do we handle lots of Ajax calls?
Will someone mesh up my data?
Performed correctly though, you do get neater web clients and that is actually an attractive notion. The flip side of this, though, is all the hype. Everything from the Colgate inspired salesmen – yes, they were there with their surreal suntan – to the former Sun execs doing the IPO dance. “Next episode: Will they get cash in before the expiry date?”
Ranting aside, Ajax is here and we all better start to learn it because it does matter. And it does put an amicable face on your web site.

Far out Java container

Two years ago the chosen Java compatible container was …a cool BMW. This year it was an autonomous dune buggy named Tommy with software all in Java. It was entered in a challenge to cross a patch of the American desert – without a driver. It looks a bit like Flash Gordon with four tires instead of rockets, built in somebody’s garage. I must say Tommy spurred my interest more than the BMW, but then again you would not look too cool driving it.
Sun to ground control…


It was quite evident that Sun has a new version of NetBeans out and that it does support the latest Java EE version. And, yes, it does look good. And the price tag is quite right – zero. I just wish that it wasn’t being plugged at most every Sun keynote etc. The audience did get the message and we will check it out. Boy, that IDE market is tough!

Java 6

“Java 6 is here” was one part of the message and “compatibility matters” was the other. Sun CEO Jonathan Schwarz has been going the open source route a lot and many applications are being made OS. Even Java itself could be made open some day. The “When” is still open though. Personally, this isn’t such a big deal for me. Sun has been, for the most part, good at stewarding Java. They have also managed to keep the code running, mostly, between versions. Opening it could be a problem, but also very interesting. Making compatibility a part of the arguments makes sense in such a scenario. Let us hope it stays just that – compatible.

“We want your help”

Making your software open source does change your way of thinking. Sun has promoted a lot of its stuff into the OS arena. Good, I say! It is only logical that participation, from you and me, is seen as a great opportunity. Many of the applications going open, Project Glassfish etc, often ended their sessions with a “we need you”. Maybe Sun’s bad economy has a role in this? To me it doesn’t matter. I like it and I am eagerly waiting to see what the future might bring.

Summing up

JavaOne is a lot more than you can cram into an article such as this, but let us not be stopped by that when summing up. EJB 3 has come of age. This was their prom night and things were basically just as they said they would be two years ago. Keep an eye on it. It will play a role in many projects. Spring will deliver aspect oriented programming, not Sun, and it will go from hot to “fun and here”.

Ajax is everywhere and you will use it. You might as well pick up a good book on this subject and start hacking.

Java 6 will deliver lots of goodies. Will we get to use it? Or is your project stuck in Java JDK 1.4.2 unable to upgrade? Let us hope not. Check out scripting! It will change with Java 6. Who can tell which language will affect it big time? Someone will. Check out your local Javaforum to see if you can predict the winner.
Keep coding and may your code be with you.

Originally published in JayView.

Javaforum Malmö – now alive and kicking!

March 4, 2006

The Javaforum Malmö (re)opened at Jayway’s new office at Malmö one afternoon in March. The meeting set a new record for that conference room and filled it with ease.

Some forty people checked in for this, the first Javaforum event in Malmö for several years. The audience was a mosaic of around fifteen companies. The forum handled subjects like project methodology Scrum, application servers like T4 and Geronimo, news of the Spring framework and finally sensory networks using tiny Java thingies. The sessions covered the small and the big, from the sublime to the surreal – and all in Java.

Feel like more Java? Check for up to date information.
Welcome next time around!

Originally published at Jayway Team Blog.

The scent of freshly cut code

March 4, 2006

I just had to be there. My friend was starting out anew. A fresh idea, a start up – I was hoping it was going to be good. I was not disappointed.

I could smell it straight away when I en- tered the small office – the fresh and raw scent of newly cut code. Two men sat in front of their computers. One of them smiled at me.
Jon Hauksson and I had hacked code, back to back, against a fast moving dead- line in a distant project. He had given up his daytime job as a consultant at a respectable firm. Now, he was shoulder deep into his new venture, bringing au- dio books to the mass market of mobile phones.

Jon and Andreas in a classical image – happy men with computers.

What had he learned? What is it like to develop a business to consumer app running on all those different Java mo- biles? A lunch later we were heading back into his office at Ideon – an incu- bator for new companies. Mobiles were strewn across their tables. Blank walls. Shelves were more or less empty. It all breathed start up and focus – get that app out trough the door!
We sat down and Jon pulled an SQL- statement to calculate the last 24 hours of commerce.
– Yeah, rising, he said and smiled.

It has been said that developing a Java ME solution for all those different mo- biles is difficult and time consuming. Is it that hard?
Yes… but it has improved, especially for Nokia and Sony Ericsson. But some other brands can be trickier. We usually code for the Sony Ericsson emulator and then bring the code over to the Nokia one. Usually there aren’t any problems here. From here on we test the other brands. If we hit a problem we either try to solve it or we have to program our way around it. It used to be more difficult. The situation has improved in the last year. Especially Sony Ericsson is working hard to make it better.

Does this mean you have different sets of your code?
We have four variants of the result- ing jar/jad file and that is enough for us. Fortunately NetBeans has pre-process- ing that makes life easier and allows us to only work with one code base.

Without any time to think, what are the top three hard to learn lessons that you can you share with us?
1) Testing, testing, testing is extremely important. Many unforeseen things can happen in areas like operator handling, memory management, the mobile network and different quality issues – a lot can happen and usually does.

Bokilur’s application playing a book.

2) The Java settings are often not in- stalled correctly on the terminal from start. The user can download the set- tings for Java from the operator but sometimes these settings are not cor- rect. We’ve tried to add a guide that pops up when this happens but fiddling at this low level is not a pleasant experience for the customer. Some call our support number and we try to help them, but it is still a problem.
3) Streaming solution
The network, GPRS or 3G, has gaps and this affects our application. We’ve had to develop a streaming solution with a five-minute buffer so that you can listen to our books while driving. We’ve even tried it on trains, on the X2000 for instance, which reportedly has its share of problems when it comes to mobile network connections. But to our relief it worked there as well.

Would you do use Java again?
For sure, it is easy to work with and even though it has problems, it is getting better.
Although, we’ve chosen to imple- ment a number of our own frameworks even though there might be a good open source solution available. It is im- portant to us to keep our code size to a minimum and we usually don’t want the whole nine yards.

Any examples of when you’ve had to roll your own solution?
Well, we implemented a GUI frame- work a bit like J2ME Polish. This way we can target our application’s needs and keep control. We’ve also developed an RMI like protocol to interact with the servers. There are, however, a lot of open source solutions to try out.
We also have a simple debug solu- tion where the phone sends back debug information to the server so it can be logged. Very handy. And of course simple stuff like text han- dling.

Would you do anything differently now when you look back? How long has it been anyway?
One and a half year. Ha-ha, it feels a lot more! We are trying to improve the user friendliness of the entire browsing and purchasing process in the applica- tion. That’s user interaction, but I can’t think of anything technical. Keeping the program easy and intuitive is important to us.

Any practical hints on starting a prod- uct company in the mobile space?
Get the operators as partners. For a data rich service like ours, you need to get a deal without traffic charges. Other- wise it will be hard to develop a pricing model that is attractive to your targeted customers. Besides, the operators have a strong interest to bring in attractive services into their portals. They need more than just games and ring tones.
Oh, and one more thing, keep track of the top selling mobile phone list. That way you know which models to target during testing.

What about the future?
Our business idea is to be the best mobile literary service both here and abroad. You can expect us to go after the Nordic countries.

Has the “market” for new companies changed?
Yes. During the last year more of my former colleagues have started compa- nies of their own! It’s even come to the point where venture capital has a prob- lem finding something to invest in…

During our interview Bokilur’s other programmer, Andreas Melvanius, kicks in.
– We have a problem.
It turns out that a single Nokia phone is thrown into silence. A quick round of questions and they agree on some new tests.
After a while Jon asks whether I’d like a tour of the ”pavilion”. Walking in the corridors, we find a new com- pany behind every door. I cannot help but being impressed, such diversity. We stumble upon solutions for detecting schizophrenia, simulation of combus- tion engines, cameras that can see in fog and darkness and so on. I leave the barracks feeling elated – I too want to start up my own company.

PS. Curious about the Bokilur applica- tion? Check out
PPS. Bokilur later changed their name to
Originally published in JayView.

Bert Rubaszkin, the quote master

March 3, 2006

Bert R working on his sauna raft.

Bert Rubaszkin is the Chief Technologist, CTO to us mere programmers, for Sun Sweden. He is often quoted in such computer mags as Computer Sweden. Questions cover a lot of ground, from Sun servers being tossed from tall buildings to the future of some new Sun technology.

Bert comes across as the wise ol’ programmer who has seen most of what there is to see. Always in a good mood, I find it hard to see him loosing his cool despite being probed with difficult questions. But first, let’s start off with a simple question.

What’s it like being asked to comment on just about everything under the Sun?
There are two problems. If you have worked long enough, people expect you to be an expert at everything. The truth is that you only know a fraction of what is needed, but I do know who to ask.
Another major problem are magazines. They are mainly interested in either new information or mud raking. Some are seriously interested in what has happened, others… they only want to talk about things that don’t work. But it is my job. I just try and answer the question. Sometimes it’s actually flattering that people think I know so much.

Open Source has become quite important to Sun of late. As I understand it, you’re going to open source all of your code. Why?
It is correct that we plan to release all our source code, but I would like to point out that there is no time plan saying that at a certain point in time all the code will be released. The most important open source project for us now is OpenSolaris. The cat is, so to speak, out of the bag. We have been discussing this for several years. Once we took the decision, we needed to wash the code and make sure we had the copyrights to do this.
The effect is quite positive. We now have a community around OpenSolaris 10, and version 11 will contain features that wouldn’t otherwise have made it in there without the community. And bug tracking of course. Plus it defused the Linux versus Solaris debate. We can now discuss things with our customer in a much more interesting way. Someone at Sun said “It’s not about the code stupid, it’s about the dialogue!”
Internally it has also been a vitamin injection. Involvement is a lot higher. Everything we wished for has happened.
There was a suspicion that we would only let go of the old stuff. Instead we released innovative bits like DTrace etc. This has the positive side effect of code going cross operating systems. For example: a group of people are working on a distribution of FreeBSD, which will include DTrace. That’s one example of code becoming de facto standard outside the GPL arena.
Other high priority Open Source projects from Sun are OpenOffice and Netbeans. The end result of all of this is that we get more competitive. We get better tools and the risk of stagnation disappears.

What is the story with CDDL (pronounced “cuddle”), Common Development and Distribution License? Wasn’t the Mozilla open source license good enough for Solaris?
I’m no expert on licensing but CDDL is based on Mozilla except for the one paragraph that states that the license can be changed at any given moment. It doesn’t even require any approval. Our big clients don’t accept this single difference.

Most readers of JayView already know about the resurgent Javaforum. Apart from that, what is the status of Java in Sweden? Can you spot any trends?
Trends are meta trends and so… What I see from my angle is that Java still dominates the server side. Microsoft probably would have hoped that it would have made a more visible dent in the server side of things. We see very little impact from Dotnet among our customers.
The meta trend is SOA and web services and how to interact in a mixed environment such as Java and Dotnet. We work together with Microsoft on this and more is going to happen. J2EE dominance on the server is not threatened. From a customer point of view, it’s good to be able to encapsulate web services in the different worlds and that there is a consensus on how to interact between the two.
Looking on Java end-to-end, we’ve only started. There have been problems with limited midp hardware and different profiles between small devices. These mobiles will be more powerful. The problem of coping with differences between small platforms will be forgotten in 5 years.
We’re not there yet… but the APIs will be the same and so on.

Q: Sun has had this ambiguous relationship where it both owns Java, but also lets it play with the programmers on the block. It’s not hard too find examples of changing times. Take for example the Apache Harmony project (an open source implementation of Java 5 SE) or the interest in dynamic languages and the effort to provide support for these in the JVM in Java 6 SE. Are Sun’s dual roles over Java changing?
There is a concept within Sun which refers to “church and “state” as far as Java is concerned. Some work within the state it’s no more difficult than that. Java is a big investment for Sun and we want to see it spread, to complete its potential. It’s important that standards are kept and that nobody can corrupt them. Journalists typically want to make a big thing out of this. Our customers think JCP (Java community process) works. Maybe it is true that Sun has been able to keep the two roles apart and at the same time have products.
One strategy that we have followed is that every time there has been a split view, a 50-50 vote, on any subject within a JCP committee, we haven’t used our deciding vote. We’ve actually let the other point of view “win”. I believe that off the record even IBM will agree to this. The common denominator for those who complain, is that they haven’t actually looked into the JCP. It’s rather quick work nowadays. In the beginning some smaller companies complained, but that has been resolved. It’s rare to hear someone say “Damn Sun”.

Java is more of an ecosystem than just a language. What’s the status of that system today? And where is it going?
A flora of tools and frameworks has grown up, but doesn’t necessarily run within the JCP. Java is sort of the centre and further out are things like Ajax. You can go far out from the middle of this system and that is a good thing. James Gosling himself thinks this is very good. He was asked what happens if something can be done smarter in a different language to Java.
“I’m all for it!” That goes for most Sun folks. There is no single person in the world that can answer all the questions within that field – that’s good.

At the far end of that ecosystem I actually place Microsoft and C#. Some of the recent improvements in Java were inspired by the “other camp”, just as ideas and open source flow in the other direction. To me this is “competing symbiosis”. Take away one and the other would stagnate, much like Internet Explorer vs. Netscape. What’s your view on this?
It’s good with competition, but I’m critical of the fact that Microsoft with a budget exceeding some countries’ GNP cannot produce anything better than a pure Java clone, C#, with some minor improvements. Pitiful! But then again, competition is good. The Java community has a keen ear for improvements.
I have to give Microsoft credit for building tools that are easy to use. But this is partly at the expense of power and has meant that some things have not been developed. For instance, I’m totally convinced that Excel is a threat to mankind. People make decisions based on small errors that are impossible to debug.
If we at Sun can maintain the power and at the same time increase ease of use.

What is the best/worst thing that has happened in Java?
The best thing that is happening right now it is that we succeeded with Javaforum. It’s an embryo for an independent organisation. Tremendous. On a more technical level there are lots of good things happening. One thing that I like, but I would not classify it as “tremendous”, is annotation in Java 5.
The worst thing that has happened in Java was Microsoft signing an agreement, only to break it again immediately after. That wasn’t nice – not funny at all. But what was funny was them being made to pay.

Looking into your crystal ball, where is Java heading? Is there a device that will _not_ hold a JVM?
Different times place different demands on tools. Java’s dominance will be broken sooner or later. For instance, there are more network-oriented languages at academic level. Old languages don’t die. Look at how much Cobol code there is.
Java was the fist programming language for the net and shows no signs of old age. I would guess that Java will have a lifecycle similar to Cobol. It will take hundreds of years before it disappears – ha ha.

It has been said that Java will be the Cobol of the future. I dare you to make a guess at when that will be!
20 years from now it will be legacy. Not within five.

I once managed to make James Gosling speechless. He was taking questions from the audience and I managed to sneak in _”What’s the meaning of Java, as in ‘what’s the meaning of life’?”_ What’s your take on that?
It was the first programming system built for the net, just as NFS was the first file system for the net. That was the meaning.

Dang, I thought that at least the last question would stomp Bert.

Originally published in JayView.

Jonas Bonér, your regular AOP guy

February 4, 2006

Jonas is still standing when I meet him at the Öredev conference to discuss an interview. Even though a cold is doing its best to bring him to his knees. He still has a ‘roundthe-globe’ trip to Tokyo to do and the following sprint to reach JavaOne in Japan another one of those Swedes going places.

Jonas Bonér is the founder of AspectWerkz, an aspect oriented framework without the need for special compiler. It later joined forces with AspectJ, which could be described as the Father Christmas of aspect oriented programming in the Java world. He used to work for BEA in Sweden, but has recently joined Terracotta, San Francisco.

Jonas you’re a long term AOP advocate. When and how did you start with aspects and, perhaps most importantly, why?
I started using AspectJ back in 2001 and I was immediately very fascinated by the concepts of AOP, its power, beauty and ability to solve really hard problems in such an elegant way. I have always valued clean design and simplicity and in AOP I found a way of letting design and concepts map directly to actual code.
I started by using ‘policy enforcement aspects’, e.g. AspectJ’s declare error/warning, logging and tracing aspects, as part of the build process. After a while I gradually started to use it for more advanced things, such as security, transaction demarcation, persistence, caching, “unit of work”, design patterns etc. for better modularity, reusability, maintainability and simplicity.

Why did AspectWerkz and AspectJ merge?
The short answer is that the users needed a standard. But let me elaborate a little bit. During recent years the AOP landscape for Java has flourished. When a technology is new and immature it is usually a good thing to have many frameworks/products competing, since they can inspire and push each other and the technology even further, but when the technology matures it starts to become a problem. For example, the users might end up not knowing which framework, syntax and semantics to start using, since they do not know which ones will even be around a year later. If you see this happen a lot, like it did with AOP, then there is a need for standardization.
During 2004, AspectWerkz and AspectJ slowly converged, but the two projects were still focused on different areas and had complimentary strength. We had some discussions and both teams saw the need for standardization and felt that both projects as well as the user community would benefit from unifying the projects, roadmaps and teams instead of competing. So we decided on the merger that was announced in the beginning of 2005 and was completed with the release of AspectJ 5 at the end of last year.

So where do you think we’re heading for in the future, as far as AOP is concerned?
I am confident that AOP will play an important role in the future of enterprise application development. Many vendors are already starting to embrace AOP as an enabling technology for separation of concerns, transparency, simplicity etc. For example Terracotta DSO, JBoss EJB 3, TopLink, Spring etc.
These are all examples of vendors using AOP to layer infrastructure services on the user’s domain models in a transparent fashion. That is great, but I hope that it won’t stop with that, my wish is that users will see the value of using AOP in their application specific code, in order to achieve more modular design, code that is more easily written, understood, reused and maintained. In some ways AOP had a healthy backlash over the last year, which has removed most of the hype around it. It has matured, has a standard in AspectJ 5 and is ready for real-world deployments.

Tool wise it seems that we have several things in place in Eclipse/AspectJ/ AJDT to get a good grip on aspects. Examples: being able to see what code an aspect affects, diff engines to compare changes to an aspect etc. Do we have all we need?
The AJDT team has done a tremendous job and I think that we have already come a long way, even though more work can of course be done and needs to be done in this area. But while this is true for static views of the system, like AJDT is showing, there is almost no support for monitoring and introspecting the runtime view of the system, something that is getting more important now when the usage of dynamic AOP is becoming more widespread, which has the ability of redefining the behavior of the system at runtime. This is an area in which I hope that the academic community will contribute more in the future.

Can you see anything that needs to be fixed or added before we see more widespread adoption of AOP?
I think that the most important thing now is education trainings, books, articles etc. We need to try to reach beyond the group of “expert” developers. Another thing is a standard library. We need an ADK (Aspect Development Kit) in the same sense that Java has its JDK. I am involved in an attempt to gather and develop a set of library aspects for AspectJ 5, and we’ll see where that leads. Finally I hope to see more best-practices, patterns as well anti-patterns emerge.

Terracotta, the new company you work for, has a distributed cache product. What’s so cool about it?
It is completely transparent. Meaning that zero code changes are required to cluster an arbitrary J2EE application. Our only requirement is that you have to write a correct multi-threaded application. We then take that multi-threaded single VM application and a tiny bit of XML configuration to declaratively specify what should be shared, lock semantics etc., and turn it into a multithreaded multi-JVM application. These are the features I find most interesting:
* True transparency.
* Noserialization.We only send the actual changes to an object graph (the delta) as well as keep track of who is referencing who, so we can send these changes only to the nodes that need it, e.g. lazily.
* Object identity is preserved. We maintain Java’s “pass-by-reference” semantics, so regular object references work, which means that you can use your domain model the way you have designed it, as well as traditional OO design patterns etc. without a need to think about distribution, caching or clustering. • Coordination and memory manage-ment. We maintain the Java Memory Model transparently throughout the cluster, including distributed coordination, for example wait(), notify(), synchronized() etc., distributed garbage collection etc. We work as a virtual heap, meaning that you can for example run an application with 200G heap on a machine with 4G of RAM in which we then page in and out the heap on a demand basis.

How does it work?
We use AOP technologies to adapt the application at class load time. In this phase we extend the application in order to ensure that the semantics of Java are correctly maintained across the cluster, this includes object references, thread coordination, garbage collection etc. For example, we extend the semantics of a regular synchronized block, or actually maintain them across the cluster, by taking a global lock for the object instance that you are synchronizing on right before entering the block and releasing it right after exiting the block. You can declaratively define the exact semantics for the block, e.g. read, write, concurrent etc.
Another example is the invocation of notifyAll(). Here we turn this call into a cluster wide notification, which will open up for all nodes to contend for the lock.
The architecture is hub and spoke based, meaning that we have a central server which manages the clients, we use TCP/IP so the server just needs to be somewhere on the network. The client in this case is simply your regular application together with the Terracotta JAR. People always ask whether the server is not a single-point of failure, but we have a SAN-based solution to support fail-over in an active-passive manner. You can have an arbitrary number of (passive) servers waiting in line and upon failure the selected one will pick up right where the master server that has crashed left off.

Finally, as an active open source contributor, do you have anything up your sleeve for the future?
I have some ideas, but nothing I can talk about now. Stay tuned…

Something tells me that we will hear more from Jonas Bonér.
His cold? Well the trip actually made it go away. He made JavaOne with half an hour to spare. I guess it’s that Viking blood.

Originally published in JayView.

Printers going Java

January 4, 2006

Embedding Java doesn’t always equate mobiles. Nor does it necessarily mean stuffing it into a rover on Mars. Introducing new possibilities is sometimes just a question of putting it on a small card hooked up to your printer.

Axis, a well-known creator of print servers, network cameras etc has done just this. They took one of their print servers and added Java for a big time printer manufacturer in Japan. How did they do it? And what did they learn? I teamed up with three of the people – Jens Johansson, Kristofer Johansson and Stefan Andersson – behind this Java enabled print server to get more of the gory details.

The Start

The project started around four years ago with a prototype after some seven months. The release onto the market was roughly one and half years ago. The first step, however, was to assess the feasibility of it. Could they perform to their customer’s demands? What type of handling of Java classes should they use just in time (JIT) or ahead of time (AOT) compiling? Even the choice of hardware wasn’t given from the start. Should they use third party chip dedicated to running Java or do it themselves? The former would have meant a bus and shuffling data in between the main chip and the Java chip.
Not interesting. Ten man-years later, here’s what they
ended up with.

A schematic view of the print server and its different parts. The functionality is extended by uploading Java classes as a normal jar file.

The Box

First off they had to decide which road to travel – base it on a hardor software? Roll their own or go third part?
Buying an ASIC block doing the Java and integrating it on their chip architecture would have been very expensive and wouldn’t have resulted in a cost effective system. Buying a third-party chip? That would have meant hooking it up on the system bus and making the whole system rather complex. In the end they settled on a software solution and doing it themselves.
Basically they took the Axis ETRAX 100LX chip, added more memory and used software to implement the Java support. The ETRAX 100LX systemon-chip is a 32 bit, 100 MHz RISC CPU. The design then uses the Java 2 Platform, Micro Edition (J2ME) with the CDC and Foundation profiles. This resembles the abilities of a PDA or a set top box on your TV.

The Java foundation classes reside in a separate memory and are loaded by the print server before any uploaded jar files from the file system on the board are run. The memory is divided into a 16 MB file system and 16 MB for all the executable code including the Java Foundation classes. Add to this a 64 MB RAM that houses the system, of which approximately 13 MB is used for Java. This is considerably more than the standard print server that typically has 4 MB of memory.

The operating system is Axis’ own proprietary RTOS for embedded systems. The unit consumes a sparing 0.3 W.Add to this a USB port and you get a pretty good idea of this competent little circuit board.
You change the function of the print server by uploading an ordinary Java jarfile. These compiled classes are then run on the print server and have access to the various ports and, of course, to its printer.
There is even a remote call possibility (RMI) to query the server for its status et al.

The result

Even though there were apprehensions at the start of the project about the performance, the result speaks for itself. CaffeineMark 3.0 is a Java benchmark. It is used to measure the speed of a Java Virtual Machine (JVM) running on a particular hardware platform. The test is calibrated to return a score of 100 running on a Pentium 133 MHz using Windows 95 and Sun’s JDK 1.1 interpreter. Even though this feels like an old PC, it does give you a good reference point for embedded solutions.
Axis’ print server landed on an impressive 60. This result is even more to write home about since the entire floating-point math is done in software.

The Lessons Learned

As with most projects, there were problems to overcome. In this case the floating-point library proved to contain bugs. The TCP stacks used at Axis were different compared to the Sun version. There were compiler issues going from Java to C. As coup de grace they needed the Java logo and that meant going through a grueling TCK test (Technology Compatibility Kit). This is a suite of tests, tools, etc that provide a way of testing an implementation for compliance with a Java.
”This has been among the most complex printer projects we’ve done ”, says Jens Johansson with a weary smile. However, the Java portion worked without any major problems as soon as the foundation classes were in order. ”For one and half year we haven’t had any Java related bugs”, explained the tech lead Kristofer Johansson.
They chose AOT compiling to implement the foundation classes, but this meant a large code base and it took time to implement. Hindsight is one of the best visual aids: might it have been better to use JIT instead?


So techno stuff is all just fine, but what can we do with this Java enabled print server? Ever forgot about an important printout that shouldn’t lie around? Well, why not a simply hook up a card reader? Once you arrive at the printer you swipe your card and, presto, your pulp stack pops out. Authentication and authorization could be done against an LDAP server.
Still too simple. Why not implement a distributed load-balancing scheme for printers – next available printer pulls a job from a print queue somewhere. Need more power? Just hook up another printer to the network. Could I combine the print server with a web camera and motion detection? Add something to sample the temperature and we get surveillance, fire detection and printing to go.

Puleeze Axis, can I get a couple of print servers to try out?

Originally published in JayView.

JavaForum – nu nära Dig!

December 4, 2005

En gång i tiden, före bubblan före dotcom dog sotdöden, fanns det ett Javaforum i Sverige. Nu är det dags att återfödas! Sun och Jayway är stolta föräldrar till Javaforum i södra Sverige, generation två!

Det fanns en tid då Java-utvecklare kunde samlas för att ta del av det senaste inom Java och dessutom mingla. Efter några års dvala är det nu dags att starta denna uppskattade form av kompetensutbyte igen. Fast nu kommer vi i en starkare version både ansikte-mot-ansikte och på nätet.
Java handlar om öppenhet och samarbete. Javaforum är ett sätt att förverkliga visionen.
”Det finns dessutom stora synergieffekter i att koordinera aktiviteterna” säger Bert Rubaszkin (Sun), en av de drivande krafterna bakom JavaForum. ”I och med Jayways engagemang i Javaforum börjar nu ett stabilt fundament för ett långsiktigt samarbeta att ta form. Nu förväntar vi oss att andra intresserade partners, som är beredda att engagera sig i detta hör av sig!”
Seminarierna kommer att variera från det praktiska till det teoretiska, från det lilla till det stora men alltid vara intressanta.
Du är hjärtligt välkommen till Javaforum.

Kom in och anmäl dig nu på, så är det redan klart!

Urkopplad Nästa forum är den 23 mars kl 13:00 i Jayways nya lokaler på Hans Michelsensgatan 9. Skriv till bjorn.granvik snabel-a och anmäl dig!

Vi ses!
Originally published in JayView.

Øredev 2005

November 20, 2005

Øredev is the biggest conference for developers in Sweden and covers several areas
such as Java, .Net, Methods & Tools and Embedded.

The conference in our own backyard got off to a very good start indeed and featured internationally renowned speakers such as Eric Evans, Rickard Öberg. Kai Tödter etc etc. The number of attendees was 320 and they came from as far away as Copenhagen and Gothenburg. “Own backyard” is quite literal since Jayway was one of the principal organizers. Nevertheless, why not try and analyze our own effort?


As Java fans we had not only the Java track to enjoy. The Methods & Tools and Embedded tracks contained several interesting sessions with a Java base. This made it sometimes difficult to choose between colliding talks, just as it should be.
Looking at the reviews from the sessions, we find “9 ways to hack a webapp” at the top. It was a great talk given by Martin Nystrom (yes, there are no dots above the “o”) – but somewhat embarrassing. The “I’ve made that mistake!” feeling was a bit too apparent. The top position of this talk came as no surprise. JavaOne attendees gave it an honorable second place in a review of all presentations given there.
The talks covered a lot of ground. They ranged from the challenges of implementing Java on a mobile phone to 64-bit technology and its effects on the JVM, from standards like EJB3 to up-and-coming technologies like aspect oriented programming. On the anecdotal side we note Bert Rubaszkin, Chief Technologist at Sun Sweden, who in his “10 years with Java” talked about several achievements. Some of the examples included Rickard Öberg sitting in the audience. Times flies by faster within IT. Or, better still, it’s just the family!
A few rough edges around the conference were apparent: the sun (yes, the one in the sky) together with a curtain on the loose, made life difficult for some lecturers on the main scene, nothing that can’t be fixed for round two of this annual conference.

Mingling at Øredev 2005


The success of this, the first, developer’s conference in Öresund is promising. The conference will return this year in November. Let’s hope it continues to build on its promising start.

Originally published in JayView.

DDSteps – data driven sanity

August 1, 2005

”Aaarrrgh”, is not only the sound of pirates. It is also the desperate cry from programmers who realize what a mess they’ve got themselves into. This is especially true in testing. The number of combination that you should run your code through easily outpaces you. But fear no more, here’s one pill with a great taste ordered by Dr Jay DDSteps.

DDSteps is on open source project the brainchild of Adam Skogman which aim to make sense of your unit tests when they consist of god knows how many permutations of input. Read on to find out exactly how.

A Typical Project

Let us tell the tale of a too-successful project. It all started out as a prototype web site, quickly hacked together over a week to show to the investors. Big hit! Some swift fixes later it went live slow, rocky, but making money. Refactoring the code was an inevitable next. As new people joined the project, test driven development was introduced to make sure that programmers did not break each other’s code. The unit tests covered more and more of the code. Even the happy hackers in the group had to admit that things were getting better and bugs were fewer.
The business side of the now booming company poured new feature requests into the development and demanded that they appear in production “real soon”. Therefore, a shorter release cycle was introduced, releasing every month. Also, a quality assurance test team was set up to test each release. So developers had to “pre-test” the release before the test team got it. Testing now took two weeks. Bugs were all around and had to be corrected. At this point the site went global, requiring small specific changes for each country.
This is when it all broke down. Testing now took over a month and bugs were often reported months after the release was made. Nobody wanted to re-release and do all those tests again, so the site went into production with known, serious bugs. It was at this point that the programmers went: “Aaarrrgh!”

The Remedy

Testing your solution for every type of input and making sure it will pass function tests is difficult at best. Usually, this is done “by hand” and requires real people. And, yes, it is boring and time consuming. So you have to automate it, just like you did with JUnit tests for unit testing.
By Function tests we mean, ”Does it work like the use case says it should”. We mean full, system, end-to-end tests, using a production-like environment. We mean using a browser, surfing the web application and then checking the real database for results.
Automating function tests is hard, mostly since they suffer from the same problem as the manual testing. Many of the tests are the same; just the data is slightly different in each test.

Data driven testing on the other hand separates the testing code from its input. This and the reusable test steps makes it easy to maintain and evolve the test cases, you get reuse instead of copy-paste. Need another field? Just add new column in your input and handle it in your test case and voila. There will be no explosion of test methods, no massive round of changes!
Ok, let’s start from the top and look what DDSteps is and how it makes your life easier.

Figure: Overview of the DDSteps’ process. 1) Input is retrieved from an Excel file and populates your test cases and test step POJOs. 2) Yellow is your code. TestSomething is your test method that (re)uses different test steps. 3) The fixture F typically sets the database in the correct state for the test. The Navigators, Executors and Validators in this example use JWebUnit to perform their respective step. 4) Output – reports, console etc.

Since most tests are the same, with just different data, first we separate data (input) and the test code. The data, found in an Excel file, is inserted into your test case, each row becoming a test case run. The second important part is the framework of reusable test steps. Test cases are broken down into steps:

  • Fixture – set up the needed data in your database.
  • Navigator – Navigate your web site.
  • Executor – Input data into the system, like filling out a form on a page and pressing a button.
  • Validator – Validate the output on a page or a row in the database etc.
  • Cleaner – Clean up any mess you have made.

You implement these types of test steps using for instance JWebUnit and Spring JDBC. You can easily imagine that many test cases will use the same executor, since they pass through the same web page, so reuse is everywhere.
Finally, DDSteps is just standard JUnit, so the test outcome is reported back via Ant, CruiseControl or your preferred JUnit compatible IDE. It integrates perfectly into your existing environment.

Divide and Conquer

Let us dive deeper. Again, how do we separate data from the code? You can do this several ways. We chose to use Excel. It’s not only easy to format and to enter data, it’s a de facto standard and you can use formulas etc. And if you want you can always use OpenOffice.
Let us take the following use case from PetClinic from Spring – ‘Add new pet’.

Figure: Use case ‘Add New Pet’. Green is data that is put into an Excel file. See next figure. The letters N, E and V are our test steps for this use case.

We have now divided our use case above into reusable test steps. Next is the Excel file. From our test case we can see that we need: Owner, pet name etc.

Figure: The Excel file that holds our input and fixture data.

Each test method is entered on separate worksheets. The fixture data we need for the database is entered as well. DDSteps find your test data in the spreadsheet using the method names in your test code, and uses JavaBeans get/set to inject data into your test case and your test steps.


Next is the test code. Let us look at just one part of it – navigating. The example covers points 1-6,7 of the use case, i.e. the first and second test step. In order for this part to work our HTML pages need to be written with JWebUnit in mind, i.e. an id is put on elements, so that we can find it and “click” etc. Let us assume this has been done. We also use Spring even though this is optional.
The test case would then look something like this.

// Our test case, see PetFTest.xls for input.
public class PetFTest extends PetclinicTestCase {

    protected NavigateToAddPet nav;
    protected ValidatePetForm valForm;

    // The test method, same as tab name in Excel file.
    public void testAddPet_Ok() throws Exception {

    // Used from data file to access navigator properties, e.g. “”.
    public NavigateToAddPet getNav() {
        return nav;

In the code snippet above we declare our PetFTest test case that inherits from PetclinicTestCase. This base class only holds common things like web browser, fixture etc. The test method testAddPet_Ok is simple and reduced to only use necessary test steps, nav (NavigateToAddPet) and valForm (ValidatePetForm).
Also note that we have getNav() which is used as the first part in the data file “nav. name”. The access to properties is JavaBean based, i.e. using get/set methods. We will only look at the first test step, the navigator NavigateToAddPet, since the other steps are similar in concept.

public class NavigateToAddPet extends JWebUnitTestStep {

    // Reuse another navigation step
    NavigateToOwner navigateToOwner;

    public NavigateToAddPet(WebBrowser webBrowser) {
        // The web browser is injected by Spring.
        navigateToOwner = new NavigateToOwner(webBrowser);

    public void runStep() throws Exception {

        // Delegate

        // Click ‘Add new pet’ button,
        // which is the submit button in the form ’formAddPet’

        // Snapshot of web page.
        writeTrail(”Add New Pet Form”);

    // Full name is populated from ”” in the data file.
    public void setName(String name) {

This test step is a composite of another navigator, NavigateToOwner, and going to the “Add new Pet” page. WriteTrail will write specified html page to the hard disk, a visual page debugger if you will.


Having come this far, we can now run our test code and get the result.

Figure: And finally the output, in this example, in Eclipse.

Oops, better fix our code! In the example above we see the errors per row, not just by test method.

What happened to the Project?

When our team employ DDSteps, they can automate their tests, and cut down the time for testing to minutes, not weeks. This means you can run all function tests every night or every time the source code changes! Suddenly, you can release to production with no bugs because as soon as you know there is a one, you fix it during ordinary development. Function testing is now a part of development, not an afterthought.


DDSteps both tries to break new ground and to reuse solutions perfected by others. We think it is a good mix to implement function tests using data driven testing. Is it for you? Who knows? Get it from to see for yourself.
Originally published in JayView.