This paper explains one of the prime goals of Cairngorm; which is to allow developers to follow agile testing of Flex RIAs. Agile testing combines engineering with testing activities into one activity and can be expressed by following Test Driven Development where a developer would write tests first. In particular, this paper focuses on unit testing where units represent objects or single architectural layers in contrast to functional testing (customer tests) where a unit represents the complete application (many architectural layers). Agile functional testing is also vitally important to Flex project but the focus of a separate paper.
Most objects of a system depend on other objects (depended-on objects). Objects contain other objects and delegate work to them. When unit testing one object the decision is up to the developer to touch the depended-on object with the same unit test or substitute the depended-on object with an alternative object (test double).
The motivations to substitute (isolate) depended-on objects with test double are manyfold and have to be contrasted with the motivations against it.
Motivations for isolation (concrete depended-on objects are substituted with test doubles) are:
When concrete depended-on objects are executed by the object under test, then isolation is not performed and the motivations against isolation can be:
As shown above, motivations to isolate objects exists as motivations against it exists. When should a developer decide to isolate an object and when not?
As a general rule of thumb, this paper recommends isolating at each client side architectural layer. Presentation layer, domain layer, application layer and integration layer. Often developers can isolate objects beyond that and prevent the dangers of over-specified tests that hinder refactorings with focusing each test on a new, small piece of behaviour that is not repeatedly covered by another test. Also, interactions do not always need to be tested to the full API including parameters. A test might gain enough realism by just observing one method call that identifies an interaction with a depended-on object instead of observing all method calls of similar type including full parameter sets. Mock frameworks in Flex today (i.e. ASMock and mock-as3) allow this flexibility to prevent having to specify every detail.
Isolation can be achieved with substituting the depended-on object with an object that is controlled by the unit test suite; a test double. The test double takes the same interface as the production object but does have a different implementation, usually much simplified and targeted for a particular test case or suite.
Test doubles can come in many flavours. The most important test doubles are stubs and mocks. A stub directs inputs into the object under test. A mock object observes outputs of the object under test.
Two of the most popular approaches to get substituted behaviour into objects under test is dependency injection and dependency lookup. Dependency injection injects the test double via the API of the object under test. Dependency lookup retrieves the test double via another object. The other "locator" object should then provide some mechanism to configure it during unit tests with a test double.
Substitution by overriding can be achieved with subclassing the depended-on object with an object only used in tests that provide an alternative implementation for the behaviour not of interest in unit tests of the object under test.
Substitution by overwriting simply sets (overwrites) a depended-on object with a test double. However, this is only feasible if the depended-on object is exposed via an API of the object under test and typed as an interface or common base class.
A Test Hock is a control point (i.e. if-statement) within the object under test that decides to behave differently based on if it's being exercised during unit tests or during production. This can require a known object with global state to tell objects if they are in a unit tests or not. However, instead of hardcoding a conditional statement in production code, this could also be done using Flex's conditional compilation. In any case, this approach adds risk, size and complexity to production code that would not need to be there following the approaches above and should be used with caution.
While there are many solutions to how to organize unit tests, Cairngorm attempts to provide a convention to improve consistency between projects.
Prefer to keep test code in the same project as source code, separated by source paths.
Application code and test code are separated within the same project but use the same package. This is possible by putting them into different source paths. The application code remains in Flash Builder's standard "src" source path, while the test code is moved to the additional source path "test". This can also make searches for objects easier.
Prefer "Test" suffix instead of "Test" prefix. A suffix can ease type-ahead searches via Flash Builder as the test is immediately alongside the object under test. If a test class becomes too large, analyse if the design of the object under test can be improved. If not, multiple test classes can be created with the naming "Type_Feature_Test".
Self-documenting tests i.e. should or Given/When/Then naming.
http://dannorth.net/introducing-bdd
Test-infected developers never write their tests days after their code. Test-infected developers want to write tests, because that's the way they think about software development. They don't want to think otherwise. Test-infected developers never have excuses not to test. They are never too busy to test, their environments never take up too much time to create test data, and their customer never complain that testing is too expensive because it takes too much time.
If it is difficult to create an environment of test infected developers,
analyze the design of the application for testable code, prevent repetitive and over-specified tests, keep tests small, easy and close to the way objects are used in production.
don't attempt to test every single line of code in your Flex application. Realize that functional tests have a role. Focus your unit tests to test behaviour and not structure and wiring.
ensure libraries, frameworks, API projects, any other code that is shared by multiple developers achieve the highest coverage and quality of tests.
keep quality of test code as high as application code. Maintain tests with refactorings but realize that it's sometimes easier to throw away and rewrite tests as it's sometimes easier to throw away and rewrite code.
Meszaros, G. (June 2007). xUnit test patterns: refactoring test code. Addison-Wesley.
Beck, K. (2002). Test Driven Development: By Example. Boston, MA: Addison-Wesley Longman Publishing Co., Inc.
Williams, P. (2009). Unit Testing User Interfaces. Retrieved May 05, 2009 from http://weblogs.macromedia.com/paulw/archives/2008/03/unit_testing_us.html
http://blog.james-carr.org/2006/11/03/tdd-anti-patterns/
Feathers, M. (June 2008). The Flawed Theory Behind Unit Testing. Retrieved September 07, 2008 from http://michaelfeathers.typepad.com/michael_feathers_blog/2008/06/the-flawed-theo.html