Home

Foreword

At the early stage of its development Aigrette was a prove of concept aimed to demonstrate that automating web UI tests with the Selenium webdriver API without having to write programming code was possible. Choice was first made to write test scenario with XML files.
The XML format was chosen for its declarative nature, the ease for parsing and also the fact that its structure is similar to the target page and elements structure that we want to test.
Moreover anybody involved in the software industry (developer, project manager, tester, functional analyst) should be able to write XML and it could even be automatically generated. This point is interesting when time comes to create some DSL or UI.

Later on I had to work on the development of a complex web application written with a JS framework that generates pages by directly writing to the DOM tree.
These pages were featuring lots of dynamic updates and custom designed widgets, all implemented in Javascript.
Automating the UI and end-user tests of this application was challenging and at this time no open source tool with the out-of-the-box capability for testing our application convinced us.
Indeed we needed to be able to automate all interactions with complex HTML elements structures and deal with delays induced by the fact that HTML is generated by a Javascript application. That means that even if an element is visible on the page it is not always ready for accepting events. Moreover as a constraint the application had to be usable on low configuration laptop with older browser and OS releases.
Therefore in parallel of the development of the customer project I kept on developing Aigrette, solving problems when they arose.
Thanks to the fact that its code was easily extensible I could quickly add elements to its XML element library for solving specific cases. Step by step I improved the tool and re-factored the code so that a minimum set of elements was required, keeping the possibility to add Java code for writing specific elements for particular cases if needed. I also improved the integration with Picvert for writing complete end-user test plan and integrate with Jenkins.
The possibility of designing reusable elements with XML was also developed. That reduced the effort for automating the test of some complex elements since no Java code was required for extending the test capabilities.
As of release 1.2 screen comparison and a true DSL were added.
Aigrette is now an open source project freely available that may be handy when no other testing tool is proposed with your development framework.

Purpose

• Automate GUI testing
• Reduce required technical skill for writing test script
• Reduce test script maintenance effort
• Provide extensible tool

Features

• HTML5 page automation testing
• Simple scripting language
• Screen image comparison
• Extensible through Java or XML
• Screencast recording

Introduction

Aigrette is written in Java/Groovy and is based on the Selenium Webdriver API. It aims to automate web application tests.

Installation

Install the binary package

Unzip the binary package. That's it!

Build from the sources

Prerequisites

Maven release > 3.0.5 must be installed and mvn command must be in the PATH.

Build the package

Create a directory, cd to it and checkout the sources.
At the prompt type mvn clean compile package
To skip the tests add the option -Dmaven.test.skip

When build ends the target subfolder under aigrette-core module must contain the executable jar file aigrette-{version}.jar.

Running Aigrette

Prerequisites

1. Install the JRE >= 1.7
2. Install the requires browser driver (more infos on the drivers on http://docs.seleniumhq.org/docs/03_webdriver.jsp#selenium-webdriver-s-drivers.

Running from the command line

Aigrette is a Java program that accepts the following options and parameters:

-p param_1=value1,..,param_n=value_n
    A comma delimited pair of key/value parameters passed to the scenario execution.
    Values will replace placeholders in the scenario description file.

-f scenario_file
    Pass the scenario description file

-v
    Run the scenario file description validation before running the test

-r path
    Record the test to a screencast and store mov file at specified path. File name is scenario file name appended with "mov" extension.

Example of test run with Firefox:
java -jar aigrette.jar -v -p DRIVER=Firefox,PARAMNAME=paramvalue -f scenario.xml

Exemple of same test run with Chrome:
java -Dwebdriver.chrome.driver=chromedriver.exe -jar aigrette.jar -v -p DRIVER=Firefox,PARAMNAME=paramvalue -f scenario.xml

Call aigrette API

First add aigrette-core and aigrette-api modules jar files as dependencies of the client program.
Test can now be executed by calling the method:
org.ome.aigrette.Aigrette ActionResult execute(String file, Map<String, Object> vars) with file the scenario file with path and vars the map object with the parameters (name/value pairs, equivalent to the parameters passed with option -p on the command line.

Writing a test scenario

In XML

Test scenario is described using a XML syntax.
The XML snippet below is a simple example where we are logging to an application via a logging form and then we check that we are directed to the next page containing any element with id 'elementId'.

<?xml version="1.0" encoding="UTF-8"?>

<scenario>
  <driver value="${DRIVER}" size="1200,800"/>
  <get>http://localhost:8080/app/login</get>
  <res>
    <wait name="username" timeout="4"/>
  </res>
  <req>
    <input name="username" value="user"/>
    <input name="password" value="passwd"/>
    <button id="submitButton"/>
  </req>
  <res>
    <wait class="id_productionButton" timeout="30"/>
  </res>
</scenario>

Parent element is always <scenario> element
Next the driver must be specified. In this example it is given as a parameter to the executable (-p DRIVER=Firefox for instance). Attribute size specifies the browser window size.
Following line contains the URL for accessing the first page of the application we want to test.
Then we can see <res> and <req> blocks. These blocks are delimiting the actions performed and the tests that must be performed on the web page.
The first <res> block is waiting that the page loads. Actually it is waiting the element with name=username be available until the timeout (timeout="4") has expired. Timeout unit is second.
Next block sends data to the input fields with names "username" and "password". A click event on the submit button is triggered on the next line.
Finally we check in the last <res> block that we get the next page where element with id="id_productionButton" is present. The timeout is set to 30s.

In Aigrette script (as from release 1.2)

Even if scenario may still be written directly in XML the release 1.2 features a convenient script language. The same test script may be written like this:

driver browser:'${DRIVER}', size:'1200,800'
// Could be: driver browser:Chrome, size:'1200,800'
get 'http://localhost:8080/app/login'
wait name:'username', timeout:4
input name:'username', value:'user'
input name:'password', value:'passwd'
click id:'submitButton'
wait className:'id_productionButton', timeout:30

List of XML elements

Understanding the XML syntax is useful for creating reusable custom widget. You find below a complete description of all elements.

Control elements

<scenario>
Description
Root element for all scenario file
Attributes
None

<driver>
Description
Element for specifying the browser driver to use and the screen size
Attributes
 value: driver name (Firefox, Chrome,...)
 size: screen size (Example: "800,600")

<get>
Description
Specify the starting URL for the test scenario
Attributes
None. The URL is specified as element text.

<res>
Description
This element contains tests that must be performed on the current page or the page that will be loaded.
There may be any number of <res> element within a <scenario> element and it contains any number of action elements.
Attributes
None

<req>
This element groups action or component elements that corresponds to the actions that will be performed on the page.
There may be any number of <req> element within a <scenario> element and it contains any number of action or component elements.
Attributes
None

Action elements

Action elements are elements "mapped" to UI components like input, button, div, etc.
They all have in common some attributes used to locate the component on the page. One of these attributes must be provided for each action element:
id: element id
name: element name
path: xpath expression
class: element class name
tagName: element tag name (img, a, input, etc)
css: css selector

They also have an additional attribute widget used to identify a custom action strategy to be executed instead of the built-in one. This is explained in details in the next section (Extending the capabilities).

<wait>
Description
- Inside a <res> element
First it may be used to perform an explicit wait even if I have never faced the need of a delay in a <res> block.
The purpose of this element inside such a block is to wait that an element is present on the page or until the timeout has expired.
- Inside a <req> element
Locator attributes are useless and do not have to be specified. The only purpose of this tag in such a block is to perform an explicit wait with the timeout attribute value in second.
This may be useful when testing some DOM generated UI and when some HTML elements are not attached to the DOM yet while already accessible.
Specific attributes
timeout: mandatory and holds a numeric value of seconds.

<uwait>
Description
This element is similar to the <wait> element but it is waiting that the located UI element gets invisible.
It is only usable in a <res> block.
Specific attributes
timeout: mandatory and holds a numeric value of seconds.

<button>
Description
This element aimed to send a click event on the element page.
Specific attributes:
It has no additional attribute.

<input>
Description
The input element aims to drive any input html element like text input, checkbox or radio button.
As a child of a <res> element it is used to check the value of the target element, if it is checked or not (via checked attribute) or if it is enabled/disabled.
Specific attributes
timeout: only required in <res> context
value: input text value
checked: true|false - May trigger a click event when child of a <req> element depending of the UI element selected state.
If the UI element is a checkbox then the checked value is set to the checkbox. For other input element a click event is triggered when checked=true.
As a child of a <res> element the checked value is compared against the selected state of the UI element. This is valid for checkbox or radio button.

<select>
Description
Drive a simple HTML select element (listbox) with it. Only allowed as child of a <req> element.
Specific attributes
value: item value to be selected.
visibleText: the text value, displayed on the screen, to select.

<block>
Description
All purpose element that may be used to access any HTML element.
Very useful for building custom <xwidget> elements for driving complex UI widgets.
Specific attributes
click: triggers a click event.
dbclick: triggers an double-click event.
text: when block element is child of a <res> element it is used to check some text value.
timeout: useful in the context of a <res> child.
Remark
May have child elements of type <input>, <button> or <block> and therefore is the most important element to compose <xwidget> elements.

<screen> (release >= 1.2)
Description
Take a screen snapshot and compare it with the snapshot file at the specified location. If none exists yet then the image is saved at the specified path and may be used for comparison at next test run.
Specific attributes
file:path and file name of the snapshot image.
name: name of the image
threshold (optional): percentage as a double value. It specifies the threshold as from the image difference must be reported. For instance if threshold value is '1' than a difference lower of equal to 1% between images won't be reported. Default value if not specified is '0'.

Component element

<xwidget>
Description
This element refers to an XML file that describes the processing of a HTML element. Further information on this is available in section "Extending the capabilities"
Attributes
file: absolute file path
vars: comma delimited key=value pairs

Scripting Aigrette

Test scenario may be written directly in XML but it is easier to write them in the scripting language of Aigrette.
A script must start with a driver followed by a get instruction, next any other instruction may be appended. Placeholders may also be used for attribute value. They are replaced by values provided at command line.
Example:
driver browser:'${browser}', size:'${screenSize}'

Script syntax

The script is a list of functions, one per line, with their parameters. A function call follows this general syntax:
keyword paramName_1:paramValue_1,...,paramName_n:paramValue_n
A commented line starts with '//'.

driver
Description
Initialize the driver to use and the screen size.
Syntax
driver browser:'browser name', size:'x,y' where browser name may be a string value or a predefined driver name. Predefined value are Chrome, Firefox or InternetExplorer
Parameters
-driver: driver name as a string, a placeholder or a constant. Constant value may be Chrome, Firefox or InternetExplorer.
-size: a formatted string to set the screen size 'x,y'.
Example
driver browser:Chrome, size:'1200,800'
To test against a Selenium Grid additional url parameter must be provided.
Example for server test
driver browser:Chrome, size:'1200,800', url:'http://localhost:4444/wd/hub'

get
Description
Get the starting page for the test
Syntax
get 'http://server/page.html'
Parameter
Function get takes a single anonymous string parameter for the application entry point URL.

delay
Description
Explicit delay in second
Syntax
delay n
Parameter
This function takes a single anonymous parameter for the delay value in second.
Example
delay 5

wait
Description
Wait until the element is visible on the page. This function aims to suspend the test until a certain element is available on the page. It is used to detect that a page is loaded or to ensure that an element is actually on the page before trying to interact with it. This second use case is very common with applications that dynamically add elements in response of user action for example.
Syntax
wait any_locator_parameter, timeout:n
Parameters
The first parameter is any of the locator parameter (see below)
timeout: value in second, if the timeout exceeds before the element is found than the test exits with a failure status.
Example
wait id:'elementId', timeout:3

uwait
Description
Similar to the wait instruction it waits until the located element has disappeared. It may be used for example to detect that a popup has disappeared before being able to interact with the parent page.
Syntax
uwait any_locator_parameter, timeout:n
Parameters
One of the parameter must be one of the locator parameter (see below)
timeout: value in second, if the timeout exceeds before the element is removed from the page than the test exits with a failure status.
Example
uwait id:'elementId', timeout:10

click
Description
Triggers the click event on the located element. It also accepts the widget parameter. Main purpose is to trigger a click event on a button but it may be used for any element on the page on which a click event must be triggered.
Syntax
click any_locator_parameter
Parameter
A parameter that may be any of the locator parameter (see below).
widget: optional parameter for the widget name
Examples
click id:'buttonId'
click className:'buttonClassId', widget:'SomeFramework'

input
Description
Used to provide a value to a text input element or to check/uncheck a box. It also accepts the widget parameter.
Syntax
input any_locator_parameter, value:'text value'|checked:[true|false]
Parameters
A parameter from the locator parameter list
value|checked: set the input text value with value parameter or the boolean value for a box with checked parameter.
widget: optional parameter for the widget name
Examples
input id:'textInputId', value:'some text', timeout:1
input id:'radioId', checked:true, timeout:1

select
Description
Its purpose is to interact with a list box. It also accepts the widget parameter.
Syntax
select any_locator_parameter, visibleText:'text item'|value:'value item'
Parameters
Any parameter from the locator parameter list.
visibleText: a text value for the select item text displayed that should be selected.
value: the value as string for the select item to be selected.
widget: optional parameter for the widget name
Example
select id:'listId', visibleText:'item text'

xwidget
Description
This instruction refers to an XML file that describes the processing of a HTML element. Further information on this is available in section "Extending the capabilities"
Syntax
xwidget file:'file path', vars:'key/value list'
Parameters
file: the file path and name of the XML file that contains the widget definition.
vars: parameters that are passed to the XML.
Example
xwidget file:'/home/project/customWidget.xml', vars:'value=some_value,text=someText'
Warning: always use forward slashes in any OS.

expect
Description
Aims to perform a validation on a page element. It may be used to check an input text element value, if a box is checked or not or if an element is read only or enabled.
If the timeout exceeds before the element value or the checked, readOnly or enabled attribute is not equal to the value specified as parameter than the test exists with a failure status.
Function expect does not aim to detect the presence of an element on the page. For that purpose function wait must be used.
Syntax
expect any_locator_parameter, text:'text value', value:'text value'|checked:[true|false]|readOnly:[true|false]|enabled[true|false], timeout:n
Parameters
A parameter from the locator parameter list.
text|value|checked|readOnly|enabled: depending on the type of input a text or a boolean value can be provided. The readOnly parameter takes a boolean value and is used to check that the tested field is writable or not. Enabled is also of boolean type and is used to checked that the tested field is enabled or not. Parameter text is used to match any element text (<any_element>text_to_match</any_element>).
timeout: numerical value, if the timeout exceeds and the tested value (value|checked|readOnly|enabled) does not match that the test fails.
Example
expect id:'textElementId', value:'expected text value'
expect id:'textElementId', enabled:true, timeout:3
expect id:'textElementId', readOnly:true
expect id:'boxElementId', checked:true, timeout:3

block
Description
Instruction used to interact with any element other than an input element. Also useful when the widget attribute is specified, to execute custom action strategy for specific elements See "Extending the capabilities" for further information. It is also the only instruction capable of sending a double click event to an element.
Syntax
block any_locator_parameter, widget:'wigdetId', click:[true|false], dbclick:[true|false], timeout:n
Parameters
A parameter from the locator parameter list.
click: send a lick event if set to true
dbclick: send a double click event if set to true
widget: optional parameter for the widget name

screen (release >= 1.2)
Description
Take a screen snapshot and compare it with the snapshot file at the specified location. If none exists yet then the image is saved at the specified path and may be used for comparison at next test run.
Syntax
screen file:'filename', name:'name', threshold:n
Parameters
file:path and file name of the snapshot image.
name: name of the image
threshold (optional): percentage as a double value. It specifies the threshold as from the image difference must be reported. For instance if threshold value is '1' than a difference lower of equal to 1% between images won't be reported. Default value if not specified is '0'.

Locator parameters
These attributes are used to find the element on the page.
id: element id
name: element name
path: xpath expression
className: element class name
tagName: element tag name (img, a, input, etc)
css: css selector

Extending the capabilities

The standard elements <input>, <button>, etc work well for testing HTML page using standard HTML components.
But many development frameworks are using complex components or even generate HTML through the DOM API. The resulting HTML code cannot always be directly driven with elements that are pre-defined in Aigrette.
To address this issue Aigrette offers two mechanism for extending its capability to test these kind of web pages.
First the tester can define a new reusable component using the Aigrette XML syntax.
For the cases where this method cannot be applied Aigrette also allow tester to create a new Java action strategy class for a HTML component in particular. The new class must be packaged in a jar file and will be dynamically loaded by Aigrette at runtime.

Defining a new element

Let's state that an UI application developed with ExtJS framework uses some input radio button.
The developer added a marker class id_myInputElement on his source code element and ExtJS generates the following HTML structure:

<table class="id_myInputElement" ... >
  <tbody>
    <tr<>
      <td<>
        complex html structure here
      </td>
      <td>
        <table>
          <tbody>
            <tr<>
              some td elements here
              <td>
                <input type="button" ... />
              </td<>
            </tr<>
          </tbody>
        </table>
      </td<>
    </>
  </tbody>
</table>

How can we easily generate a click event on this element?

First create a new XML file called customRadio.xml with the following content:

<block class="#{id}" >
  <input tagName="input" checked="#{check}"/<
</block>

This is a reusable component that we may reference and use for every test scenario.

Note that #{id} and #{check} is a placeholder that will be replaced at runtime.

Next add this entry in the scenario where we want to access such a input radio button:

<xwidget file="path_to_customRadio.xml" vars="id=id_myInputElement,check=true"/>

Creating a new action strategy

New strategies may be implemented for existing elements.
To add a new strategy apply following instruction steps:

• Create a new java project and add aigrette-api as dependency
• Create a new strategy class implementing the methods execute(...) and/or validate(...) of the interface IActionStrategy.
  Respect the following naming convention for the new class:
  <prefix><Input|Block|Button|Select|Wait|Uwait>ActionStrategy
  Examples: MyOwnInputActionStrategy or MyOwnBlockActionStrategy.
• Package the new class(es) in a jar file
• Copy the jar file into a extlib folder (create it if it does not exist yet) in the same folder that the executable aigrette jar file.
• Create a file aigrette.properties (same folder than aigrette jar file) and add the new action strategy package name (or names if you want to add multiple packages).
  Example:
  strategy=com.mycompany.strategies,com.mycompany.otherstrategies
• You can now use the new action strategy with existing element by providing the widget attribute to the scenario element.
  Example: if new new action strategy class is MyOwnInputActionStrategy then the widget attribute value for the <input> element will be MyOwn.
  <input id="myId" widget="MyOwn" value="someTextValue">

Screencast (release >= 1.1)

Run Aigrette with the additional parameter -r and the path where to store the recording file.
The recording file name is the scenario file name with extension .mov.
If using multiple screen be sure that the browser is running on the main screen.

Testing with selenium server (release >= 1.1)

Compatibility tests requires that you run the UI tests with different browsers (and/or OS). This is where the selenium server will help significantly. Indeed with the selenium server and the selenium grid you can distribute your tests across multiple machine or VM where different OS/browsers are installed.

For this purpose a new element has been added:
<remoteDriver>
Attributes:
value: driver name (Firefox, Chrome,...)
size: screen size (Example: "800,600")
url: URL of the selenium server

The example below describes the steps to run the test with a local selenium server.

1. Add a <remoteDriver> element to the scenario:
   <remoteDriver value="${DRIVER}" url="http://localhost:4444/wd/hub" size="1200,800" />
   Note the value for the value property (${DRIVER}) that will be
   replaced by the parameter value given as parameter to the command line.
   The url parameter value is the URL of the hub (central node), the driver value is used by the hub to dispatch the test execution to the specific node.

2. Run the selenium server:
   java -jar selenium-server-standalone-2.39.0.jar -role hub

3. Run the selenium grid and add nodes for Chrome and Firefox:
   java -Dwebdriver.chrome.driver=./chromedriver.exe -jar selenium-server-standalone-2.39.0.jar -role node -hub http://localhost:4444/grid/register -browser browserName=chrome -browser browserName=firefox
   Note that the Chrome driver path must be specified as VM parameter.
   The hub parameter value is the URL of the of hub.
   You can start multiple nodes for different browsers that all register to the hub. When you start Aigrette with a defined browser value the hub will dispatch the test execution to the corresponding node.

4. Start Aigrette to test your application with a specific browser:
   java -jar aigrette-1.1.jar -v -p DRIVER=Chrome -f scenario.xml

Additional information here: https://code.google.com/p/selenium/wiki/Grid2

Project Members:

• Olivier Meurice (admin)