Menu
▾
▴
SF.net SVN: matplotlib:[7700] trunk/matplotlib/test
|
From: <as...@us...> - 2009-09-07 20:27:52
|
Revision: 7700
http://matplotlib.svn.sourceforge.net/matplotlib/?rev=7700&view=rev
Author: astraw
Date: 2009-09-07 20:27:44 +0000 (Mon, 07 Sep 2009)
Log Message:
-----------
test conversion: remove infrastructure cruft
Modified Paths:
--------------
trunk/matplotlib/test/README.txt
trunk/matplotlib/test/_buildbot_mac_sage.sh
trunk/matplotlib/test/_buildbot_test.py
Removed Paths:
-------------
trunk/matplotlib/test/consolidate_diff_images.sh
trunk/matplotlib/test/movegood.py
trunk/matplotlib/test/run-mpl-test.py
Modified: trunk/matplotlib/test/README.txt
===================================================================
--- trunk/matplotlib/test/README.txt 2009-09-07 20:18:32 UTC (rev 7699)
+++ trunk/matplotlib/test/README.txt 2009-09-07 20:27:44 UTC (rev 7700)
@@ -1,299 +1,2 @@
-========================================================================
- matplotlib test structure
-========================================================================
-
-===== How To Use
-
-= Running
-
-Run the 'run-mpl-test.py' script to execute the test harness. This must
-be run with the version of python that you wish to test matplotlib with.
-This means that it must have nose installed (and PIL if image comparison
-is to be done). By default this will pick up whatever python is on your
-path, so make sure it is the correct one.
-
-- Command-Line Options
-In addition to the standard nose command-line options, there are several
-specific to the matplotlib test harness. They are as follows:
-
- -t TAG, --with-tag=TAG
- Will only run test cases that have the specified tag.
- Each test case should have a 'tag' attribute (if a
- case does not have one, then it is assumed to be an
- empty list). The 'tag' attribute is a list of
- strings, where each value is a representative propery
- of the test case. Example tags are 'qt' or 'units'.
- This can be specified multiple times.
- --without-tag=TAG This will run those test cases that do not have the
- specified tags.
- --clean This will remove all output files and saved results.
- If this is specified, no other processing will be
- performed.
- --all This will runn all test programs regardless of working
- directory.
- --keep Keep any generated output files in a directory called
- 'saved-results'. This directory will be created if it
- doesn't already exist. This directory is in the same
- location as the test case whose results are being
- saved.
- --keep-failed This acts just like '--keep' except will only keeps
- the results from tests that error or fail.
- --make-test=testName
- Creates a template test case file in the current
- directory with the name TestFoo. Where 'Foo' is the
- provided test name.
-
-
-- Running Specific Tests
-In order to can specify the exact test case you want to run use the
-standard nose mechanism. For example, if you have the following setup:
-
-TestFoo.py
- def test_func():
- ...
-
- class TestFoo:
- def test_bar( self ):
- ...
- def test_bug( self ):
- ...
-
-Then to test everything in TestFoo.py do the following:
-$> run-mpl-test.py TestFoo.py
-
-To run all tests in the test class TestFoo do this:
-$> run-mpl-test.py TestFoo.py:TestFoo
-
-To run the specific 'test_bar' methodd do the following:
-$> run-mpl-test.py TestFoo.py:TestFoo.test_bar
-
-
-= Detecting Test Cases
-
-When running the matplotlib test script it will search for all tests
-in the current working directory and below (unless '--all' is specified).
-This is provided that the current working directory is a sub-directory
-of the matplotlib test directory. In the event that it is not, then the
-matplotlib root test directory will be used and all appropriate test cases
-will be run.
-
-This will not search outside of the test structure and will not look in
-the mplTest module. This will only search for test cases in the root
-test directory and any of its sub-directories.
-
-= Saving Results
-
-When using the keep flag any generated files in the 'output' directory
-are copied to the 'saved-results/<classname>' directory, where <classname>
-is the name of the unit-test class. This means that for each test case
-within a given test class, all output files should have unique names.
-
-The 'saved-results' directory will always contain the results from the
-last test run. This is considered a volatile directory since running
-the test cases without the '--keep' flag will remove any existing
-'saved-results' directory. This is to ensure the integrity of the
-saved results, they will always match the last test run.
-
-= Filtering Tests
-
-In the case of filtering via tags, a unit-test cane have multiple tags.
-When running the test program if any tags are specified as 'skip' then
-this will take precedence over any tags that might say 'process'. For
-example, if a test case has both the 'gui' and 'qt' tag, but the command-
-line is specified with the following flags:
- '--with-tag=gui --without-tag=qt'
-then the example test case will not be run because it matches the skip
-tag.
-
-
-===== Directory Structure
-
-There are several directories in the matplotlib test structure. The first
-directory is the 'mplTest' directory. This is the matplotlib test module
-and contains the various python scripts that the test harness needs to
-run. The remaining directories are as follows and contain the various test
-cases for matplotlib.
-
-mplTest
- This directory does not contain any test cases, rather it is the location
- of the matplotlib specific utilities for performing unit tests.
-
-test_artists
- This directory contains tests that focus on the rendering aspects of
- the various artists. Essentially the artist derived functionality.
-
-test_backends
- This directory contains various tests that focus on making sure the
- various backend targets work.
-
-test_basemap
- This directory contains test cases that excercise the basemap add-on
- module.
-
-test_cxx
- This directoy contains tests that focus on testing the interface of
- the compiled code contained in matplotlib.
-
-test_mathtext
- This directory contains tests that focus on excercising the mathtext
- sub-system.
-
-test_numerix
- This directory contains tests that focus on validating the numerix
- component.
-
-test_plots
- This directory contains tests that validate the various plot funtions.
-
-test_pylab
- This directory has pylab specific test cases.
-
-test_transforms
- This directory has test cases that focus on testing the various
- transformation and projection functions.
-
-test_matplotlib
- This directory has all other test cases. This contins test that focus
- on making sure that Axis, Axes, Figure, etc are all acting properly. This
- has test cases that are general to the overall funtionality of matplotlib.
-
-
-===== Writing Test Cases
-
-= The Test Case
-
-As per the nose implementation, a test case is ultimately any function that
-has the phrase 'test' in its name. The matplotlib cases however are grouped
-into directories, by what is being tested, and from there are grouped into
-classes (one class per file), by similarity.
-
-It is desireable that all matplotlib tests follow the same structure to
-not only facilitate the writing of test cases, but to make things easier
-for maintaining them and keeping things uniform.
-
-There is a class 'MplTestCase' provided to be the base class for all matplotlib
-test classes. This class provides some extra functionality in the form of
-verification functions and test data management.
-
-= Comparison Functions
-
-There are several methods provided for testing whether or not a particular
-test case should fail or succeed. The following methods are provided by
-the base matplotlib test class:
-
-- MplTestCase.checkEq( expected, actual, msg = "" )
- Fail if the values are not equal, with the given message.
-
-- MplTestCase.checkNeq( expected, actual, msg = "" )
- Fail if the values are equal, with the given message.
-
-- MplTestCase.checkClose( expected, actual, relTol=None, absTol=None, msg="" )
- Fail if the floating point values are not close enough, with the given message.
- You can specify a relative tolerance, absolute tolerance, or both.
-
-- MplTestCase.checkImage( filename, tol = 1.0e-3, msg = "" )
- Check to see if the image is similair to the one stored in the baseline
- directory. filename can be a fully qualified name (via the 'outFile' method),
- or it can be the name of the file (to be passed into the 'outFile' method).
- The default tolerance is typically fine, but might need to be adjusted in some
- cases (see the 'compareImages' function for more details). Fails with
- the specified message.
-
-Note that several of the tests will perform image comparison for validation
-of a specific plot. Though not 100% accurate it at least flags potential
-failures and signals a human to come and take a closer look. If an image has
-changed and after a human deems the change is acceptable, then updating the
-baseline image with the appropriate image from the 'saved-results' directory
-(when using the '--keep' or '--keep-failed' command-line arguments) will make
-the test pass properly.
-
-Image comparison depends on the python imaging library (PIL) being installed.
-If PIL is not installed, then any test cases that rely on it will not
-pass. To not run these test cases, then pass the '--without-tag=PIL'
-option on the command-line.
-
-= Directories
-
-Input data files for a given test case should be place in a directory
-called 'inputs' with the test case that uses it. A convienence function
-is provided with each test class for accessing input files.
-
-For example if a test case has an input file of the name 'inputs.txt'
-you can get the path to the file by calling 'self.inFile("inputs.txt")'.
-This is to allow for a uniform convention that all test cases can follow.
-
-Output files are handled just like input files with the exception that
-they are written to the 'output' directory and the path name can be
-had by calling 'self.outFile'. It is more important to use this mechanism
-for getting the pathname for an output file because it allows for the
-management of cleaning up and saving generated output files (It also
-significantly reduces the probability of typo errors when specifying
-where to place the files).
-
-A Third and final directory used by the test cases is the 'baseline'
-directory. This is where data files used for verifying test results
-are stored. The path name can be had by using the 'self.baseFile'
-method.
-
-Accessing these directories can be made simple (and reduce the chance of a
-typo) via the following MplTestCase methods:
-
-- MplTestCase.inFile( filename )
- Returns the full pathname of filename in the input data directory.
-
-- MplTestCase.outFile( filename )
- Returns the full pathname of filename in the output data directory.
-
-- MplTestCase.baseFile( filename )
- Returns the full pathname of filename in the baseline data directory.
-
-= Units
-
-Located in the mplTest directory is a set of unit classes. These classes
-are provided for testing the various unitized data interfaces that matplotlib
-supports (ie unit conversion). These are used because they provide a very
-strict enforcement of unitized data which will test the entire spectrum of how
-unitized data might be used (it is not always meaningful to convert to
-a float without specific units given). This allows us to test for cases that
-might accidentally be performing operations that really do not make sense
-physically for unitized data.
-
-The provided classes are as follows:
-- UnitDbl
- UnitDbl is essentially a unitized floating point number. It has a
- minimal set of supported units (enough for testing purposes). All
- of the mathematical operation are provided to fully test any behaviour
- that might occur with unitized data. Remeber that unitized data has
- rules as to how it can be applied to one another (a value of distance
- cannot be added to a value of time). Thus we need to guard against any
- accidental "default" conversion that will strip away the meaning of the
- data and render it neutered.
-
-- Epoch
- Epoch is different than a UnitDbl of time. Time is something that can be
- measured where an Epoch is a specific moment in time. Epochs are typically
- referenced as an offset from some predetermined epoch. Conceptally an Epoch
- is like saying 'January 1, 2000 at 12:00 UTC'. It is a specific
- time, but more importantly it is a time with a frame. In the example
- the frame is 'UTC'. This class is provided to test the functionality of
- matplotlib's various routines and mechanisms for dealing with datetimes.
-
-- Duration
- A difference of two epochs is a Duration. The distinction between a
- Duration and a UnitDbl of time is made because an Epoch can have different
- frames (or units). In the case of our test Epoch class the two allowed
- frames are 'UTC' and 'ET' (Note that these are rough estimates provided for
- testing purposes and should not be used in production code where accuracy
- of time frames is desired). As such a Duration also has a frame of
- reference and therefore needs to be called out as different that a simple
- measurement of time since a delta-t in one frame may not be the same in another.
-
-
-
-Updating after diff
-====================
-
- python run-mpl-test.py --all --keep-failed
- ./consolidate_diff_images.sh
- # check your images, decide which are good
- python movegood.py
+Please see doc/devel/coding_guide.rst for information about the
+testing infrastructure.
Modified: trunk/matplotlib/test/_buildbot_mac_sage.sh
===================================================================
--- trunk/matplotlib/test/_buildbot_mac_sage.sh 2009-09-07 20:18:32 UTC (rev 7699)
+++ trunk/matplotlib/test/_buildbot_mac_sage.sh 2009-09-07 20:27:44 UTC (rev 7700)
@@ -13,4 +13,4 @@
cd test
rm -f failed-diff-*.png
-python run-mpl-test.py --verbose --all --keep-failed
+python -c "import matplotlib; matplotlib.test(verbosity=2)"
Modified: trunk/matplotlib/test/_buildbot_test.py
===================================================================
--- trunk/matplotlib/test/_buildbot_test.py 2009-09-07 20:18:32 UTC (rev 7699)
+++ trunk/matplotlib/test/_buildbot_test.py 2009-09-07 20:27:44 UTC (rev 7700)
@@ -18,5 +18,5 @@
for fname in previous_test_images:
os.unlink(fname)
-check_call('%s run-mpl-test.py --verbose --all'%TARGET_py,
+check_call('%s -c "import matplotlib; matplotlib.test(verbosity=2)"'%TARGET_py,
cwd='test')
Deleted: trunk/matplotlib/test/consolidate_diff_images.sh
===================================================================
--- trunk/matplotlib/test/consolidate_diff_images.sh 2009-09-07 20:18:32 UTC (rev 7699)
+++ trunk/matplotlib/test/consolidate_diff_images.sh 2009-09-07 20:27:44 UTC (rev 7700)
@@ -1,7 +0,0 @@
-#!/bin/bash
-set -e
-
-TARGET_DIR="diff-images"
-rm -rf $TARGET_DIR
-mkdir $TARGET_DIR
-find . -name 'failed-diff-*png' -exec mv {} $TARGET_DIR/ \;
Deleted: trunk/matplotlib/test/movegood.py
===================================================================
--- trunk/matplotlib/test/movegood.py 2009-09-07 20:18:32 UTC (rev 7699)
+++ trunk/matplotlib/test/movegood.py 2009-09-07 20:27:44 UTC (rev 7700)
@@ -1,53 +0,0 @@
-import os, sys, glob, shutil
-import matplotlib.cbook as cbook
-
-savedresults_dir = 'saved-results'
-baseline_dir = 'baseline'
-diff_dir = 'diff-images'
-basename = 'failed-diff-'
-nbase = len(basename)
-
-failed = set()
-for fname in glob.glob(os.path.join(diff_dir, '%s*.png'%basename)):
- ind = fname.find(basename)
- fname = fname[ind+nbase:]
- failed.add(fname)
-
-datad = dict()
-for fpath in cbook.get_recursive_filelist('.'):
- if not fpath.endswith('.png'): continue
- if fpath.find(diff_dir)>0: continue
- rel_dir, fname = os.path.split(fpath)
-
-
- saved = fpath.find(savedresults_dir)>0
- baseline = fpath.find(baseline_dir)>0
-
- if saved:
- datad.setdefault(fname, [None,None])[0] = fpath
- elif baseline:
- datad.setdefault(fname, [None,None])[1] = fpath
-
-nfailed = len(failed)
-for ithis, fname in enumerate(sorted(failed)):
- data = datad.get(fname)
- if data is not None:
- saved, baseline = data
- #print ithis, fname, saved, baseline
- if saved is None:
- print 'could not find saved data for', fname
- elif baseline is None:
- print 'could not find baseline data for', fname
- else:
- x = raw_input('Copy %d of %d\n saved="%s" to\n baseline="%s" (n|Y):'%(ithis, nfailed, saved, baseline))
- if x.lower()=='y' or x=='':
- shutil.copy(saved, baseline)
- print ' copied'
- elif x.lower()=='n':
- print ' skipping'
- else:
- print ' skipping unrecognized response="%s"'%x
- print
-
- else:
- print 'could not find data for', fname
Deleted: trunk/matplotlib/test/run-mpl-test.py
===================================================================
--- trunk/matplotlib/test/run-mpl-test.py 2009-09-07 20:18:32 UTC (rev 7699)
+++ trunk/matplotlib/test/run-mpl-test.py 2009-09-07 20:27:44 UTC (rev 7700)
@@ -1,100 +0,0 @@
-#!/bin/env python
-"""
-
- TODO-NOTES:
- Command-line executable that runs the tests.
- -- nice report on test pass/fail status
- -- hooks to add coverage checking and reporting
-
- Utilities
- -- image comparison tools (non-PIL dependant)
-
-"""
-import os
-import sys
-import os.path
-
-# Save stdout/stderr
-originalStdout = sys.stdout
-originalStderr = sys.stderr
-
-# get the current directory and the root test directory
-cwd = os.path.abspath( os.getcwd() )
-root = os.path.dirname( os.path.abspath( sys.argv[0] ) )
-sys.path = [ root ] + sys.path
-
-# command-line arguments
-args = [ arg for arg in sys.argv ]
-
-# determine the actual working directory to use
-if root in cwd:
- working = cwd
-else:
- working = root
-
-if '--all' in args:
- working = root
-
-# print "DBG: mpl.test.run - cwd = '%s'" % (cwd)
-# print "DBG: mpl.test.run - root = '%s'" % (root)
-# print "DBG: mpl.test.run - working = '%s'" % (working)
-
-# make the working directory current
-os.chdir( working )
-
-import nose
-import nose.plugins.builtin
-import matplotlib
-from matplotlib.testing.noseclasses import KnownFailure
-
-for arg in args:
- # We need to do this here, because we do not actually want nose to start.
- if arg.startswith( '--make-test=' ):
- testname = arg[ 12: ]
- # Remove any surrounding quotation marks
- if (testname[0] == '"' and testname[-1] == '"') or \
- (testname[0] == "'" and testname[-1] == "'"):
- testname = testname[1:-1]
-
- filename = os.path.join( cwd, 'Test' + testname + '.py' )
- templName = os.path.join( root, 'mplTest', "TestTEMPLATE.py" )
-
- fin = open( templName, "r" )
- fout = open( filename, "w" )
-
- lines = fin.readlines()
- for line in lines:
- newline = line.replace( 'UNITTEST', testname )
- fout.write( newline )
-
- fin.close()
- fout.close()
-
- print "Generated '%s'" % (filename)
-
- sys.exit( 0 )
-
-### Run nose
-args.append('.')
-args.extend( matplotlib.default_test_modules )
-
-plugins = [plugin() for plugin in nose.plugins.builtin.plugins]
-plugins.extend( [ KnownFailure() ])
-
-success = nose.run( argv = args,
- plugins = plugins,
- )
-
-### do other stuff here
-
-
-# $> nosetests [-w <working_directory>]
-# Run a specific test
-# $> nosetests tests/test_stuff.py:test_function
-# $> nosetests tests/test_stuff.py:TestClass.test_method
-
-# Restore the original stdout/stderr
-sys.stdout = originalStdout
-sys.stderr = originalStderr
-
-sys.exit(not success)
This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.
|
