IMPORTANT: A NEW RELEASE OF COARSE GRIND IS PENDING THAT WORKS SIGNIFICANTLY DIFFERENT FROM PREVIOUS VERSIONS. PLEASE USE THE BETA OF "COARSE GRIND QUATERNATE".
. [SETTING UP FOR USE WITH ECLIPSE]
. [DESCRIPTIONS FEATURE-SET]
- Coarse Grind's intended audience are SDETs and programmers needing automated testing at a higher level than unit testing; typically for a program or system that needs to be installed or set up before it can be tested.
- An example Eclipse project showing simple tests and the Descriptions feature-set is distributed with the JAR library.
What is Coarse Grind?
Coarse Grind is a test framework appropriate for "coarse grained" automated tests, which are more end-to-end and more-involved than unit tests. It is considered by the author to be the reference implementation of the various test frameworks he has created and used professionally.
What's wrong with JUnit?
Nothing. JUnit is for unit testing but Coarse Grind is for larger-grained testing, as might be required of anything needing installation or setup before it can be properly tested. These two types of testing have different needs and different priorities:
- Multiple Points-of-Failure: Unit testing is, by its nature, extremely granular. XUnit implementations typically use exceptions for EVERYTHING, including passing the test, and thus tests normally come to an end at the first sign of trouble. This is OK for unit testing where there is a single finely-grained criterion you are trying to prove. However, if the automated test is at a higher-level, closer to the end-user experience, it's usually more appropriate to continue the test when something fails and only "bail" if there's a good reason to do so. Coarse Grind test cases do evaluate to an ultimate pass, fail, or inconclusive, but this is arrived at via a collection of results. With any test you can have multiple criteria and see which passed and failed, as well as get to the bottom-line for the test case. Higher echelons of testing (Test Subjects have many cases; Test Suites have many subjects) can be evaluated in the same fashion.
- Retaining of Test Artifacts: Typically in coarse-grained automated testing there are files, network messages, Objects left in a certain state, thrown exceptions, and other test artifacts created along the way. XUnit does not provide a means to associate these artifacts with a specific test. Coarse Grind allows the programmer to save test artifacts, including exceptions, both as part of the programmatic test results the code can see and as files in a specific folder for each test case.
- Forensic Grade Logging: Coarse Grind provides a built-in logging system because large-grained automated tests typically must log every minute step they perform. The currently running Test Suite, Test Subject, and Test Case all have their own log making it easy to determine what happened when. Setup and cleanup portions of the test are explicitly marked in the logs, as are pass criteria checks, prerequisite checks, and each test's overall status.
- Test Datum Object Generation: Often times in automated testing you repeat many different kinds of test data (happy path, borders, explicit nulls, leave a field default, deliberate attempt to break something, etc.) into many different fields of an object. Coarse Grind provides an easy means to generate typical test data for most kinds of data fields, including strings, dates, and numeric data with or without upper and lower limits. It also provides the means to generate an object of the desired type, with the desired kind of test data populating the fields. This makes it possible to quickly generate an enormous number of individual test cases for the object (or sequence of objects) being scrutinized. (See [DESCRIPTIONS FEATURE-SET].)
Does Coarse Grind Implement XUnit?
No. That was an early design choice, and it was based on the many times the author had to adapt an XUnit system to meet the needs of coarse grained automated testing. Although it handles many of the same functions, Coarse Grind and XUnit get the job done in entirely different ways, with different priorities in-mind.
What's the difference?
Coarse Grind, like XUnit, does handle the basics of pass vs. fail, setup, cleanup, etc. However it takes an entirely different approach than XUnit does.
- No Annotations: It was originally planned to make Coarse Grind use the same annotations ("attributes" in .NET) that JUnit does, but it was taken into consideration what it's like to sort through a test fixture with 300 test cases in it. Coarse Grind does NOT use annotations of any kind as a deliberate design choice. It instead delimits tests with a purely object-oriented approach. Every test case must be its own object, and thus be in its own source code file. (One-object-per-file is a Java rule and would not apply if Coarse Grind was targeted at C#.) Coarse grained testing is not unit testing and thus a test case may have to perform many more steps as it proceeds. Instead of putting generic methods in the test fixture along with the tests themselves, the recommended equivalent in Coarse Grind is to use an abstract class that the test case classes extend.
- No "Test Runner" Programs: XUnit tests are typically compiled into a library, and a separate program is needed to run them. Every time the author has implemented a framework similar to Coarse Grind, the priority was running the tests from the command line. Coarse Grind's current implementation is designed to create an executable, though the resulting JAR could be treated as a library if needed. This makes running the tests from the command line (as a batch/script file would do) a trivial hurdle to overcome. There is no need to find a console-based test runner, because you are implementing one. (A graphical test runner may be implemented in a future version.)
- Required Logging and Artifact Folders: As previously mentioned this is not unit testing. There is a need to log every step the test program takes, and to retain artifacts generated by carrying out the test. Folders are created for each Test Suite, Test Subject, and every individual Test Case. Each of these folders contains a log file specific to its execution time, and serves as a place for associated test artifacts. When done right a test programmer can save all necessary artifacts to reproduce a test manually should something go wrong.
- Simple Boolean Assertions: The author has done test automation programming for about 15 years, and has so far never needed to assert anything more than "is this condition true". All assertions in Coarse Grind are simple true/false booleans. If there is demand for the variety of assertions typical to XUnit systems, it will be considered for implementation in a future version.