Tumblelight

”Aaarrrgh”, is not only the sound of pirates. It is also the desperate cry from programmers who realize what a mess they’ve got themselves into. This is especially true in testing. The number of combination that you should run your code through easily outpaces you. But fear no more, here’s one pill with a great taste ordered by Dr Jay DDSteps.

DDSteps is on open source project the brainchild of Adam Skogman which aim to make sense of your unit tests when they consist of god knows how many permutations of input. Read on to find out exactly how.

A Typical Project

Let us tell the tale of a too-successful project. It all started out as a prototype web site, quickly hacked together over a week to show to the investors. Big hit! Some swift fixes later it went live slow, rocky, but making money. Refactoring the code was an inevitable next. As new people joined the project, test driven development was introduced to make sure that programmers did not break each other’s code. The unit tests covered more and more of the code. Even the happy hackers in the group had to admit that things were getting better and bugs were fewer.
The business side of the now booming company poured new feature requests into the development and demanded that they appear in production “real soon”. Therefore, a shorter release cycle was introduced, releasing every month. Also, a quality assurance test team was set up to test each release. So developers had to “pre-test” the release before the test team got it. Testing now took two weeks. Bugs were all around and had to be corrected. At this point the site went global, requiring small specific changes for each country.
This is when it all broke down. Testing now took over a month and bugs were often reported months after the release was made. Nobody wanted to re-release and do all those tests again, so the site went into production with known, serious bugs. It was at this point that the programmers went: “Aaarrrgh!”

The Remedy

Testing your solution for every type of input and making sure it will pass function tests is difficult at best. Usually, this is done “by hand” and requires real people. And, yes, it is boring and time consuming. So you have to automate it, just like you did with JUnit tests for unit testing.
By Function tests we mean, ”Does it work like the use case says it should”. We mean full, system, end-to-end tests, using a production-like environment. We mean using a browser, surfing the web application and then checking the real database for results.
Automating function tests is hard, mostly since they suffer from the same problem as the manual testing. Many of the tests are the same; just the data is slightly different in each test.

Data driven testing on the other hand separates the testing code from its input. This and the reusable test steps makes it easy to maintain and evolve the test cases, you get reuse instead of copy-paste. Need another field? Just add new column in your input and handle it in your test case and voila. There will be no explosion of test methods, no massive round of changes!
Ok, let’s start from the top and look what DDSteps is and how it makes your life easier.

Figure: Overview of the DDSteps’ process. 1) Input is retrieved from an Excel file and populates your test cases and test step POJOs. 2) Yellow is your code. TestSomething is your test method that (re)uses different test steps. 3) The fixture F typically sets the database in the correct state for the test. The Navigators, Executors and Validators in this example use JWebUnit to perform their respective step. 4) Output – reports, console etc.

Since most tests are the same, with just different data, first we separate data (input) and the test code. The data, found in an Excel file, is inserted into your test case, each row becoming a test case run. The second important part is the framework of reusable test steps. Test cases are broken down into steps:

• Fixture – set up the needed data in your database.
• Navigator – Navigate your web site.
• Executor – Input data into the system, like filling out a form on a page and pressing a button.
• Validator – Validate the output on a page or a row in the database etc.
• Cleaner – Clean up any mess you have made.

You implement these types of test steps using for instance JWebUnit and Spring JDBC. You can easily imagine that many test cases will use the same executor, since they pass through the same web page, so reuse is everywhere.
Finally, DDSteps is just standard JUnit, so the test outcome is reported back via Ant, CruiseControl or your preferred JUnit compatible IDE. It integrates perfectly into your existing environment.

Divide and Conquer

Let us dive deeper. Again, how do we separate data from the code? You can do this several ways. We chose to use Excel. It’s not only easy to format and to enter data, it’s a de facto standard and you can use formulas etc. And if you want you can always use OpenOffice.
Let us take the following use case from PetClinic from Spring – ‘Add new pet’.

Figure: Use case ‘Add New Pet’. Green is data that is put into an Excel file. See next figure. The letters N, E and V are our test steps for this use case.

We have now divided our use case above into reusable test steps. Next is the Excel file. From our test case we can see that we need: Owner, pet name etc.

Figure: The Excel file that holds our input and fixture data.

Each test method is entered on separate worksheets. The fixture data we need for the database is entered as well. DDSteps find your test data in the spreadsheet using the method names in your test code, and uses JavaBeans get/set to inject data into your test case and your test steps.

Code

Next is the test code. Let us look at just one part of it – navigating. The example covers points 1-6,7 of the use case, i.e. the first and second test step. In order for this part to work our HTML pages need to be written with JWebUnit in mind, i.e. an id is put on elements, so that we can find it and “click” etc. Let us assume this has been done. We also use Spring even though this is optional.
The test case would then look something like this.

PetclinicTestCase.java

// Our test case, see PetFTest.xls for input.
public class PetFTest extends PetclinicTestCase {

    protected NavigateToAddPet nav;
    protected ValidatePetForm valForm;
    ...

    // The test method, same as tab name in Excel file.
    public void testAddPet_Ok() throws Exception {
        nav.runStep();
        valForm.runStep();
        ...
    }

    // Used from data file to access navigator properties, e.g. “nav.name”.
    public NavigateToAddPet getNav() {
        return nav;
    }
    ...

In the code snippet above we declare our PetFTest test case that inherits from PetclinicTestCase. This base class only holds common things like web browser, fixture etc. The test method testAddPet_Ok is simple and reduced to only use necessary test steps, nav (NavigateToAddPet) and valForm (ValidatePetForm).
Also note that we have getNav() which is used as the first part in the data file “nav. name”. The access to properties is JavaBean based, i.e. using get/set methods. We will only look at the first test step, the navigator NavigateToAddPet, since the other steps are similar in concept.

public class NavigateToAddPet extends JWebUnitTestStep {

    // Reuse another navigation step
    NavigateToOwner navigateToOwner;

    public NavigateToAddPet(WebBrowser webBrowser) {
        // The web browser is injected by Spring.
        super(webBrowser);
        navigateToOwner = new NavigateToOwner(webBrowser);
    }

    public void runStep() throws Exception {

        // Delegate
        navigateToOwner.runStep();

        // Click ‘Add new pet’ button,
        // which is the submit button in the form ’formAddPet’
        setWorkingForm(”formAddPet”);
        submit();

        // Snapshot of web page.
        writeTrail(”Add New Pet Form”);
    }

    // Full name is populated from ”nav.name” in the data file.
    public void setName(String name) {
        navigateToOwner.setName(name);
    }
}

This test step is a composite of another navigator, NavigateToOwner, and going to the “Add new Pet” page. WriteTrail will write specified html page to the hard disk, a visual page debugger if you will.

Run

Having come this far, we can now run our test code and get the result.

Figure: And finally the output, in this example, in Eclipse.

Oops, better fix our code! In the example above we see the errors per row, not just by test method.

What happened to the Project?

When our team employ DDSteps, they can automate their tests, and cut down the time for testing to minutes, not weeks. This means you can run all function tests every night or every time the source code changes! Suddenly, you can release to production with no bugs because as soon as you know there is a one, you fix it during ordinary development. Function testing is now a part of development, not an afterthought.

Conclusion

DDSteps both tries to break new ground and to reuse solutions perfected by others. We think it is a good mix to implement function tests using data driven testing. Is it for you? Who knows? Get it from http://www.ddsteps.org to see for yourself.
Originally published in JayView.