Main Page
From fitlibrary
Contents |
Welcome to FitLibrary
FitLibrary is a test automation framework based on [Fit]. Tables are used to express storytests and are also used to report the results of a test. FitLibrary allows for specialised DSLs (domain specific languages) to be created for testing (and other purposes). FitLibrary started out as a library of new fixtures, such as DoFixture, but it has turned into a framework in it's own right. There are plans to entirely remove its dependency of Fit, to ease the evolution of the language and architecture.
Rick Mugridge
FitLibrary
An earlier version of FitLibrary is covered in [Fit for Developing Software, Rick Mugridge and Ward Cunningham, Prentice Hall, 2005]
One of the aims of FitLibrary is to express, through examples, BA-level specification of business rules, objects and processes in such a way that they can also be used for testing. FitLibrary has defined actions which allow for the details to be abstracted out, so that a storytest can express examples in a concise manner, as compared to defining a sequence of low-level steps to test a system.
The Java implementation of FitLibrary is included here. It defines the reference standard for FitLibrary, as invented by Rick Mugridge, with input from many others. There are implementations for Python, Smalltalk and C# as well.
For downloading, go to http://sourceforge.net/projects/fitlibrary/
The download here contains a user guide and comprehensive specifications for FitLibrary itself.
FitLibraryWeb
FitLibraryWeb extends FitLibrary with a set of fixtures for testing web applications, web services, XML, email, PDFs, as well as recording and mocking web services. It is provided as a separate download because of size. This is due to the large number of JARs that it depends on and that are included in the download.
FitLibraryWeb was kindly donated by Air New Zealand in October 2009. It was developed by Rick Mugridge and Darren Rowley while Rick was on contract at Air New Zealand from May 2008 - October 2009.
For downloading FitLibraryWeb, go to http://sourceforge.net/projects/fitlibrary/. This depends on the latest version of FitLibrary.
The download contains documentation and comprehensive specifications for FitLibraryWeb itself.
See below for further details.
October 2009 Release of FitLibrary
The release in October 2009 contains various additions.
See the FitNesse page ".FitLibrary.WhatIsNew", for a version of the following that has links to examples and specifications (once you've installed this in a FitNesse system).
It has been tested with FitNesse20090818, but should run on the previous version (20090709). It will not work on earlier versions, because of a change in the package of some important classes used in FitLibraryServer.
Here's a summary of the changes:
FitLibrary now depends on Java 6.
DoFixture
- Added infix special actions contains, eventually contains, does not contain, optionally.
- Added relationals, <, <=, >, >=, =.
- Added a stop watch" and sleep for
- When longer strings don't match with is, etc, a diff is shown.
- A new action uses pattern matching to extract pieces of text out of a string.
- Fixed auto-wrapping so that it doesn't auto-wrap Boolean, Number, Character
- Added some support for plain text for single-table actions. This is still experimental, somewhat incomplete, and will probably change. This is not supported within defined actions.
Defined Actions
- Sped up loading defined actions by reading them directly from the file system. This does mean that FitNesse features such as FitNesse variables and includes are not available. For backwards compatibility, a variation is available that uses the slow way.... Now defined action name lookup is case sensitive, while previously it was not.
- Changed the way that defined action calls are shown in the report.
- Partially implemented a multi-defined action. This may change. More details later...
SuiteFixture
- Fixed a problem with SuiteFixture so that it now passes a copy of its dynamic variables to the fixture in each storytest.
- Allow for set stop on error to be set at the SuiteFixture level.
SelectFixture
- Can now add to the SUT set a fixture that is returned from an action.
* Note that the behaviour of SelectFixture is soon going to be built into the fundamental flow mechanism.
New fixtures
- Added specialised Map fixtures.
- Randomly selecting the value of a dynamic variables.
Implementation Changes
- Some of the built-in mechanism of FitLibrary are now pluggable. Eg, you can alter the way that methods are looked up or the way that stack dumps for exceptions are shown in the report.
- I've also started to get ready to make more parts pluggable:
- In a later release, I am planning to integrate a ColumnFixture look-alike into FitLibrary and then remove all dependency on Fit code.
October 2009 Release of FitLibraryWeb
FitLibraryWeb contains various fixtures that support the specification and testing of web and other enterprise systems.
SpiderFixture
SpiderFixture supports the specification and testing of web systems based on HTML and ajax. It can be run through a variety of browsers.
SpiderFixture is based on [WebDriver, which has pluggable implementations for browsers (FireFox, IE, Google Chrome) as well as for "head-less" testing with HtmlUnit. It has coming support for iPhone and general Selenium.
SpiderFixture has a comprehensive set of actions for emulating a use on a web browser. It can check and manipulate forms, tables, inputs, selects, frames/iframes, windows (including popups). It can also execute JavaScript on the browser, and include screendumps in the report (when supported by the underlying browser).
It has built-in support for ajax (ie, JavaScript acting on the browser). It automatically polls for an element to appear, before checking or acting on it. This has a timeout period that can be set at any point in the storytest. This mechanism avoids the need for including sleeps in a storytest. The test delays only as long as necessary, so it does not do unnecessary waiting. Using DoFixture's becomes action (and variants), a test can also automatically poll for the value of the text or attribute of an element to becomes what's expected.
It's possible to run more than one copy of SpiderFixture at once, such as to emulate two users of a chat system. This is supported through FitLibrary's SelectFixture.
XML
XmlDoFixture is a thin layer on top of [xmlUnit]. It has actions for:
- Comparing two XML documents (diff)
- Transforming with xslt
- Checking for xpath values
Web Services
WebServicesClientFixture permits SOAP and other web services to be specified and testing.
This is based on the apache open-source system [HttpClient].
Recording Web Services
The Recorder will pass through and record web service calls on multiple ports. For each port, for each web service call:
- The call is passed on to the real web service
- The result from that call to the real web service is returned
- The request and response are saved to files
These recordings can then be altered (such as through the use of templates) to support mocking of web services.
Mocking Web Services
Why mock?
Consider a system ("our" system), which is being tested. It makes web service calls to other systems, to gather data or have an impact on those other systems.
It can be difficult to test our system when we don't have complete control of those other systems, or when those other systems are yet to be updated to handle our changes. Even if we can set up those other systems to respond appropriately, it can be a pain to coordinate. So often we're forced to test in the context of those other systems, making use of their current data. That brings its own problems.
It may be difficult to force those other systems to respond in odd ways (eg, with obscure errors), so it makes it very difficult to test our error-handling. For example, does our system react correctly, such as informing the user, if another system is unavailable for a few seconds.
There may also be problems with calling a web service that has serious impacts, such as carrying out a credit card payment.
A reasonable approach to these problems is to mock the web service to each of those other systems. This means that the test defines how another system should respond. Because the test is written to test certain behaviour, it's often OK to work out what specific ways the other systems should respond. There are techniques that help in defining the mock behaviour, as we discuss in the documentation.
MockWebServicesFixture allows for one or more web services to be mocked, on one or more ports. It has a way of defining complex options, sequences and repeats of service calls. A particular web service call from the system and test can be matched in various ways to determine the response.
Shell Fixture
ShellFixture allows a "shell" command to be executed, such as to ftp a file, and to monitor the responses. This can be useful when a storytest has to interact with the file system or some other application at some point in a larger test.
ShellFixture is a variant of a fixture available with !-FitNesse-!, but which integrates well with FitLibrary. As !-FitLibrary's-! SelectFixture allows for multiple fixtures to be running at once, this fixture provides a single shell command.
Create Dates
Including actual dates in storytests can be a pain, as they need to be changed to make sense.
The CreateDate fixture creates dates/times in the past and future, for different time zones. It takes a date format to specify the detailed form of the date/time required.
It is based on [Joda Time].
It is a good example of a "micro-DSL".
Databases
MySQL and Oracle are two simple fixtures that wrap fixtures from DbFit, so that they can be used as part of a larger storytest, with SelectFixture in FitLibrary.
The underlying fixtures are in [DbFit], written by Gojko Adzic.
Electronic Mail
The ElectronicMail fixture allow for testing that email has been sent. It has actions to:
- Connect to a email server
- Open a Folder
- Select a message that matches some criteria
- Check the message body for string matches
- Check that it has an attachment, and to download it
- Delete a message
This we created by Darren Rowley. It makes use of the [javamail] system.
PDFs
The PDFDocument fixture allows for the text within a PDF file to be checked.
It has commands to:
- Open a PDF file:
- Check a string appears somewhere in the text in the whole document:
- Confirm the number of pages:
- Select a specific page:
- Select the text below a given heading and up to the next heading (can also use contains and matches:
- Check that a string appears somewhere in the text of the current page
- Dump an image of the PDF and include it in the storytest report:
This we created by Darren Rowley. It makes use of the apache open-source [pdfbox] system.
