SF.net SVN: matplotlib:[7719] trunk/matplotlib/test/TODO

[<as...@us...>]

Revision: 7719
          http://matplotlib.svn.sourceforge.net/matplotlib/?rev=7719&view=rev
Author:   astraw
Date:     2009-09-08 17:38:14 +0000 (Tue, 08 Sep 2009)

Log Message:
-----------
testing: add a TODO list

Added Paths:
-----------
    trunk/matplotlib/test/TODO

Added: trunk/matplotlib/test/TODO
===================================================================
--- trunk/matplotlib/test/TODO	                        (rev 0)
+++ trunk/matplotlib/test/TODO	2009-09-08 17:38:14 UTC (rev 7719)
@@ -0,0 +1,50 @@
+Here are a collection of notes about things we want to accomplish with
+the testing infrastructure.
+
+matplotlib.testing.image_comparison decorator
+=============================================
+
+Collect all the failures in the image_comparison() decorator and raise
+one failure that describes all the failed images. Right now the loop
+that iterates over the images raises an exception on the first
+failure, which clearly breaks out of the loop.
+
+Put image comparison results into more hierarchical view to facilitate
+knowing which test module creates which image. (Also, revive John's
+move_good.py script to deal with new image locations.)
+
+Misc testing stuff
+==================
+
+Recreate John's move_good.py script once the better hierarchy of test
+result images is established.
+
+Buildbot infrastructure
+=======================
+
+Put "actual" images online from buildbots, even without failure.
+
+Get a Windows XP 32-bit build slave.
+
+Build nightly binaries for win32 and Mac OS X.
+
+Build nightly source tarballs.
+
+Build nightly docs and put online somewhere.
+
+A note about the hack that is the failed image gathering mechanism
+==================================================================
+
+The trouble with the current actual/baseline/diff result gathering
+mechanism is that it uses the filesystem as a means for communication
+withing the nose test running process in addition to communication
+with the buildbot process through hard-coded assumptions about paths
+and filenames. If the only concern was within nose, we could
+presumably re-work some the old MplNoseTester plugin to handle the new
+case, but given the buildbot consideration it gets more difficult to
+get these frameworks talking through supported API calls. Thus,
+although the hardcoded path and filename stuff is a hack, it will
+require some serious nose and buildbot learning to figure out how to
+do it the "right" way. So I'm all for sticking with the hack right
+now, and making a bit nicer by doing things like having a better
+directory hierarchy layout for the actual result images.


This was sent by the SourceForge.net collaborative development platform, the world's largest Open Source development site.