cl-unit-test Files

CL-UNIT-TEST
Copyright (c) 2008 Sami Makinen

Introduction
------------

cl-unit-test package is a simple unit testing utility. cl-unit-test is
released under MIT license, see LICENSE file for details.

cl-unit-test can be used in many ways in development process.  Each
assert and test can be evaluated individually or whole test file can
be loaded to run all tests in a file.  Since 0.3 cl-unit-test can be
easily integrated to asdf, too.

cl-unit-test concepts are asserts, tests and test files.  Asserts are the
simpliest forms of unit tests. Different kind of asserts exists for
different purposes. In fact only one would be feasible but to make
assert writing easier different asserts are provided for testing
true/false values and equality (in its all lisp forms).

A test is thought as set of asserts to be all checked for one test.

A test file may contain assert(s) or test(s) but usually file contains
set of named tests.


Installing
----------

Download and extract cl-unit-test-0.5.tgz package.
Make symbolic link from unit-test.asd to ASDF central registry, f.eg.

  $ cd ~/.sbcl/systems
  $ ln -s <unit test installation root>/unit-test.asd .

In Lisp eval

  (asdf:operate 'asdf:load-op 'unit-test)

Since 0.5 cl-unit-test has tests for itself. Due dependency circularities
a separate asd file is provided for tests. cl-unit-test needs to be loaded
before asdf unit-test extension can be used.

  $ cd ~/.sbcl/systems
  $ ln -s <unit test installation root>/unit-test-tests.asd .

In Lisp eval

  (asdf:operate 'asdf:load-op 'unit-test-tests)

Using
-----

Tests can be run in several ways. Each assert can be evaluated, each
def-test can be evaluated, test file containing assertions or
def-tests can be loaded, or file(s) can be loaded using test-runner.

The test output is controlled by unit-test:*show-details* parameter
and unit test formatter. Currently three different kind of 
formatters are provided and more can be implemented if needed.
A simple unit test formatter outputs test results to stream,
list formatter formats lists to stream and silent formatter
does not output anything.

The file example.lisp contains sample unit test definition.
Here is a sample run with Emacs+Slime+SBCL.

CL-USER> unit-test:*show-details*
NIL
CL-USER> (unit-test:assert-eql 'foo 'foo)
T
CL-USER> (unit-test:assert-eql 'foo 'bar)
TEST: 'BAR => BAR                                 Expected: FOO             FAIL
NIL
CL-USER> (setq unit-test:*show-details* t)
T
CL-USER> (unit-test:assert-eql 'foo 'foo)
TEST: 'FOO => FOO                                                             OK
T
CL-USER> (unit-test:def-test (foo) (unit-test:assert-eql 'foo 'foo))
TEST: 'FOO => FOO                                                             OK
FOO: 1/1 assertions passed, 0 failed
NIL
CL-USER> (unit-test:def-test (foo) (unit-test:assert-eql 'foo 'bar))
TEST: 'BAR => BAR                                 Expected: FOO             FAIL
FOO: 0/1 assertions passed, 1 failed
NIL
CL-USER> (setq unit-test:*show-details* nil)
NIL
CL-USER> (load "example")
EXAMPLE-TEST-CASE: 5/5 assertions passed, 0 failed
SAMPLE-FAILURE: 0/1 assertions passed, 1 failed
TEST FAILURE: Sample test failure.
SAMPLE-FAIL: 0/1 assertions passed, 1 failed
; in: LAMBDA NIL
;     (+ 1 A)
; 
; caught WARNING:
;   undefined variable: A

; 
; caught WARNING:
;   This variable is undefined:
;     A
; 
; compilation unit finished
;   caught 2 WARNING conditions
TEST: (+ 1 A) => The variable A is unbound.                                ERROR
SAMPLE-ERROR: 0/1 assertions passed, 1 failed
T
CL-USER> (unit-test:test-runner (directory "test/*.lisp"))
-- /home/smo/lisp/unit-test/trunk/test/unit-test-test.lisp -- 
UNIT-TEST-TRACKER: 37/37 assertions passed, 0 failed
ASSERT-T: 8/8 assertions passed, 0 failed
ASSERT-NIL: 8/8 assertions passed, 0 failed
ASSERT-EQ: 8/8 assertions passed, 0 failed
ASSERT-EQL: 8/8 assertions passed, 0 failed
ASSERT-EQUAL: 8/8 assertions passed, 0 failed
ASSERT-EQUALP: 8/8 assertions passed, 0 failed
FAIL: 4/4 assertions passed, 0 failed
/home/smo/lisp/unit-test/trunk/test/unit-test-test.lisp: 8/8 tests passed, 0 failed
TOTAL FILES: 1, 8/8 TESTS PASSED, 0 FAILED
NIL
CL-USER> 

Using with ASDF
---------------

Append a new defsystem after your existing defsystem and make new
system to depend on your system. New systems components is specified
by unit-test:unit-test-files component.  unit-test-files takes a file
pattern parameter and all matching files are loaded as tests.

Here is an simple example. Assuming the system is called 'sample'
and test files are located under 'test' directory with common
'-test.lisp' file postfix.

(in-package :common-lisp-user)

(require 'unit-test)

(defpackage sample-system
  (:use :asdf :common-lisp))

(in-package :sample-system)

(defsystem sample
  :name "A sample system definition"
  :description "This is a sample."
  :author "Sami Makinen <sami.o.makinen@gmail.com>"
  :version "0.1"
  :components ((:file "package")
               (:file "main" :depends-on ("package"))))

(defsystem sample-tests
  :name "Tests for sample"
  :description "This is a sample asdf system for integrated tests."
  :author "Sami Makinen <sami.o.makinen@gmail.com>"
  :version "0.1"
  :components ((unit-test:unit-test-files "test/*-test.lisp"))
  :depends-on (:sample))

Now evaluating (asdf:operate 'asdf:load-op 'sample-tests)
would first load sample system and then load tests associated
with it.

Mock functions
--------------

with-mock-function(s) macro is an experimental utility to override existing
defuns with a mock function within the macro scope.