Testing Web Apps with Selenium and Jnario

22 May 2013

As you might know Jnario is a new language for testing Java applications. Jnario takes the ideas of Cucumber and RSpec from the Ruby world to the Java world. At the same time, Jnario preserves the things we Java developers like so much: type safety and great tool support!

In this post I demonstrate how easy it is to write Cucumber style acceptance tests in Jnario. Jnario has some unique features, which makes writing given, when, then style scenarios a lot easier than in Cucumber or JBehave. I will show how to use Jnario by implementing a simple acceptance test for a web app using the browser automation tool Selenium. We will also cover more advanced topics such as how to encapsulate setup and teardown logic for scenarios and how to generate reports from your scenarios.

Jnario is a language specifically designed for testing based on Xtend. Xtend is a new language for the JVM featuring lambda expressions, type-inference, multi-line strings and more. The good thing about Xtend is that it uses the same type system as Java and compiles to readable Java code. This means when writing specs with Jnario, you can use all the goodness of Xtend resulting in less boilerplate code in your specs. Furthermore, Jnario is easy to integrate into existing Java projects, as specifications compile to plain Java JUnit tests.

Getting Started

First you need to install the Jnario tooling for Eclipse (if you haven't already), the easiest way to do this is via the update site or the Eclipse Marketplace. Eclipse is not strictly required for this tutorial, but its going to be a lot more convenient.

We will be using Maven to manage our dependencies and to compile our specs. The easiest way to set up a new Jnario project using maven to use the Jnario Maven Archetype. If your are using the Maven tooling for Eclipse just create a new maven project and select the jnario-archetype:

Otherwise enter the following on the command line to create a new maven project:

$ mvn archetype:generate                                  \
  -DarchetypeGroupId=org.jnario                           \
  -DarchetypeArtifactId=jnario-archetype                  \
  -DarchetypeVersion=0.4.2                                \
  -DgroupId=org.jnario                                    \
  -DartifactId=selenium

Afterwards we have to add the selenium dependencies to our pom file:

   ...
   <dependencies>
    ...
        <dependency>
            <groupId>org.seleniumhq.selenium</groupId>
            <artifactId>selenium-java</artifactId>
            <version>2.32.0</version>
        </dependency>
        <dependency>
            <groupId>com.opera</groupId>
            <artifactId>operadriver</artifactId>
        </dependency>
    </dependencies>
  <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>com.opera</groupId>
                <artifactId>operadriver</artifactId>
                <version>0.18</version>
                <exclusions>
                    <exclusion>
                        <groupId>org.seleniumhq.selenium</groupId>
                        <artifactId>selenium-remote-driver</artifactId>
                    </exclusion>
                </exclusions>
            </dependency>
        </dependencies>
    </dependencyManagement>

That is all we need to do for setting up Jnario. You can verify whether everything works by running:

$ mvn clean install

This will compile and run the example specs in the new project. If the build is successful, we can safely delete the example files "src/main/org/jnario/selenium/HelloWorld.xtend" and "src/test/org/jnario/selenium/HelloWorld.spec" and get started.

Our first Scenario

To demonstrate how to use Jnario with Selenium, we will test a web app most of us should be familiar with: the Google Search. Our goal is to test, when we search for a certain keyword that the search result contains matching websites.

Jnario features two different languages for writing tests, one for unit tests and one for high-level acceptance tests similar to Cucumber. As the feature we are going to test is pretty high-level and can easily be understood by a non-technical stakeholder, we will be using the acceptance spec language. To create a new feature definition use the wizard via New File->Jnario->Jnario Feature:

Jnario feature files use a format similar to Cucumber with some small differences, which we will look into later. The basic idea is the same, to use Given, When, Then steps to describe simple scenarios. In our example, we establish the precondition in the Given step that we open Google search in a browser. In the following When step we search for the "Jnario" keyword. Afterwards, we describe in the Then step the expected outcome, namely, to find the Jnario slogan "Executable Specifications for Java" in our search results:

package org.jnario.selenium

Feature: Searching for web pages via Google

  In order to safe time
  As a developer
  I want to find web sites faster

  Scenario: Searching for Jnario

   Given I opened "http://www.google.com" in a browser
   When I search for "Jnario"
   Then the result should contain "Jnario - Executable Specifications for Java"

Note that we use quotes to define step parameters, such as "http://www.google.com". We will use this later, when we make our specification executable.

Executable Specification FTW

So far we have defined a simple acceptance criterion for our feature. We can already execute it as a JUnit test by right-clicking our feature file and selecting Run as->JUnit Test. It might be surprising, but our scenario passes already. The reason is that our scenario will be ignored by JUnit as we haven't provided an implementation for our steps yet.

In Jnario you don't need to define a separate parser for your scenarios. Instead, you can directly add the necessary execution logic into your scenarios. You can think of a scenario as a normal Java class where the scenario is the class and each step is a separate method. So, we can setup the selenium infrastructure by declaring two fields. One field contains the web driver and one the web driver wait util. The web driver is responsible for controlling and accessing the browser. The syntax for declaring fields is similar to Xtend: 

  Scenario: Searching for Jnario
     val driver = new HtmlUnitDriver
     val wait = new WebDriverWait(driver, 30)

Now we need to make our Given step executable. We simply use the selenium API to retrieve a webpage via the web driver. Step parameters, such as the URL, can be accessed via the implicit variable args, which is a list of all parameter strings:

   Given I opened "http://www.google.com"
     driver.get(args.first)

The web search is performed using the driver's findElement API. Note that in Jnario static members are accessed by :: instead of . as in Java:

   When I search for "Jnario"
     val searchBox = driver.findElement(By::name("q"))
     searchBox.sendKeys(args.first)
     searchBox.submit

Finally, we check whether the search results contain the expected string. As it can take a while until the browser returns our search result, we need to define a waiting condition. We use the WebDriverWait, which we declared as a field earlier.

The nice thing in Jnario is that instead of implementing the wait condition in an anonymous subclass, we can use Xtend's lambda expressions. A lambda expression is declared within brackets and will be automatically converted into an instance of Function, which is required by WebDriverWait#until. Lambda expressions are one of the features making code written in Jnario/Xtend a lot more readable than plain Java code.

When the web page is returned on time, we retrieve the content of the webpage and assert that our expected string is contained. The assertion is written using Jnario's should expressions. Finally, we clean up and close the browser via the web driver:

   Then the result should contain "Jnario - Executable Specifications for Java"
     wait.until[ findElement(By::id("resultStats")) != null ]
     val content = driver.findElement(By::tagName("body")).getText() 
     content should contain args.first
     driver.close

That's all we need to do to make our scenario executable. We can now run it as a normal JUnit test and check whether the Google search really works as expected. Here is the complete definition of our feature:

package org.jnario.selenium

import org.openqa.selenium.By
import org.openqa.selenium.support.ui.WebDriverWait
import org.openqa.selenium.htmlunit.HtmlUnitDriver

Feature: Searching for web pages via Google

  ...

  Scenario: Searching for Jnario
     val driver = new HtmlUnitDriver
     val wait = new WebDriverWait(driver, 30)

   Given I opened "http://www.google.com"
     driver.get(args.first)

   When I search for "Jnario"
     val searchBox = driver.findElement(By::name("q"))
     searchBox.sendKeys(args.first)
     searchBox.submit

   Then the result should contain "Jnario - Executable Specifications for Java"
     wait.until[findElement(By::id("resultStats")) != null]
     val content = driver.findElement(By::tagName("body")).getText() 
     content should contain args.first
     driver.close

When you think now that your feature is no longer as readable as before, you might be right, but you can quickly change this by pressing Ctrl/Cmd + SHIFT + f in the editor to make the code magically disappear (and reappear).

Adding another Scenario

Let's test whether Google search not only works for Jnario, but also for the Xtend homepage. In Jnario you can reuse existing steps and their implementation in other scenarios. So we can easily add another scenario based on our existing steps. We just change the parameters to search for xtend homepage instead.

Feature: Searching for web pages via Google

  ...

  Scenario: Searching for Jnario

    Given I opened "http://www.google.com" in a browser
    When I search for "Jnario"
    Then the result should contain "Jnario - Executable Specifications for Java"
   
  Scenario: Searching for Xtend

    Given I opened "http://www.google.com"
    When I search for "Xtend Lang"
    Then the result should contain "Xtend - Modernized Java"

The Jnario editor offers code completion via Ctrl + SPACE showing all available steps. Note how the keyword color varies in steps. It is green if the step has an implementation, red if no step with implementation exists and grey if another step with implementation exists.

Using Spec Extensions

There is one problem in the implementation of our scenario. We close the browser in our Then step. This makes it impossible to write tests like:

  Scenario: Searching for Something

    Given I opened "http://www.google.com"
    When I search for "Something"
    Then the result should contain "Something"
    And the result should contain "Something else"

When we execute the Then step a second time we won't be able to retrieve the contents of the web page, as the driver is already closed. We need to move the closing of the driver into a separate tear down method. In Jnario this is supported via spec extensions. A spec extension encapsulates common helper method together with setup and tear down logic.

In our example, we create a new Xtend (or Java) class in which we extend the HtmlUnitDriver:

package org.jnario.selenium

import org.junit.After
import org.openqa.selenium.htmlunit.HtmlUnitDriver

class HtmlUnitDriverExtension extends HtmlUnitDriver {
    
    @After override void close(){
       super.close()
    }

}

We simply override the close method and add the JUnit @After annotation. This way we tell Jnario to execute this method after our scenario is performed. Now, we need to replace the HtmlUnitDriver in our feature with our HtmlUnitDriverExtension and add the additional extension keyword:

extension HtmlUnitDriverExtension driver = new HtmlUnitDriverExtension

Extensions are a Xtend feature which allows extending the behavior of objects without changing their class. In our example, it is now possible to call all driver methods directly without the field name. For example, we can now write:

val searchBox = findElement(By::name("q"))

instead of: 

val searchBox = driver.findElement(By::name("q"))

This makes extension fields a great way to encapsulate testing helpers without adding additional syntactic noise to your tests! Here is our updated scenario:

package org.jnario.selenium

import org.openqa.selenium.By
import org.openqa.selenium.support.ui.WebDriverWait

Feature: Searching for web pages via Google

  ...

  Scenario: Searching for Jnario
     extension HtmlUnitDriverExtension driver = new HtmlUnitDriverExtension
     val wait = new WebDriverWait(driver, 30)

   Given I opened "http://www.google.com"
     get(args.first)

   When I search for "Jnario"
     val searchBox = findElement(By::name("q"))
     searchBox.sendKeys(args.first)
     searchBox.submit

   Then the result should contain "Jnario - Executable Specifications for Java"
     wait.until[findElement(By::id("resultStats")) != null]
     val content = findElement(By::tagName("body")).getText() 
     content should contain args.first

Getting things dry

Another thing, which could be improved in our feature definition, is that we define the same precondition in both scenarios. This is a duplication, which makes it harder to maintain our features in the future. We can avoid this by using background scenarios, similar to Cucumber. A background scenario defines steps to be executed before each scenario. Using a background, we can rewrite our scenario by moving the given step from both scenarios into a single background scenario:

package org.jnario.selenium

import org.openqa.selenium.By
import org.openqa.selenium.support.ui.WebDriverWait

Feature: Searching for web pages via Google

  ...

  Background:
     extension HtmlUnitDriverExtension  driver = new HtmlUnitDriverExtension
     val wait = new WebDriverWait(driver, 30)
     
   Given I opened "http://www.google.com"
     get(args.first)

  Scenario: Searching for Jnario

   When I search for "Jnario"
     val searchBox = findElement(By::name("q"))
     searchBox.sendKeys(args.first)
     searchBox.submit

   Then the result should contain "Jnario - Executable Specifications for Java"
     wait.until[findElement(By::id("resultStats")) != null]
     val content = findElement(By::tagName("body")).getText() 
     content should contain args.first

  Scenario: Searching for Xtend
     
    When I search for "Xtend Lang"
    Then the result should contain "Xtend - Modernized Java"

Generating a report

As we set up our project using maven, we can easily create HTML reports from our spec. The generated reports contain a nicely formatted HTML version of our feature and will also tell us which scenario passed and which failed. These reports are a great way to communicate the current state of a project with the stakeholders.

First we need to execute our spec via maven: 

$ mvn clean test

Afterwards we can generate the HTML reports to target/jnario-doc via:

$ mvn org.jnario:report:generate

Summary

In this post we have seen how easy it is to write acceptance tests with Jnario. However, Jnario provides a lot more features than what we have covered in this post. If you want try Jnario or learn more about it, head over to the official page www.jnario.org. There is also a mailing list available for questions and discussions. You should follow me on twitter if you want to stay up-to-date with Jnario.

Follow me on Twitter.
I am a software developer at ESR Labs and the creator of Jnario.
Send me a message.