[glean-dev] Changed on rik-0-1-branch

[Rik F. <fa...@va...>]

I've made changes on the rik-0-1-branch that I think simplify writing
tests.  I've merged the trunk in and I'd like to merge this work back to
the trunk tomorrow, assuming no one objects.



HOW TO CHECK OUT THE BRANCH

To check out the new branch, use -r rik-0-1-branch on the command line.

E.g.,

  cvs -d:pserver:ano...@cv...:/cvsroot/glean login
  cvs -d:pserver:ano...@cv...:/cvsroot/glean \
                     co -d glean.rik -r rik-0-1-branch glean




HOW TO MAKE A SIMPLE TEST

tbasic.h shows how to make a simple test.  The results are in a
different class from the test class, so you will derive your results
from BaseResult and add two methods for getting and putting the
information that is part of YOUR class -- BaseResult will handle the
header:

class BasicResult: public BaseResult {
public:
        bool pass;

        void putresults(ostream& s) const {
                s << pass << '\n';
        }
        
        bool getresults(istream& s) {
                s >> pass;
                return s.good();
        }
};


The test will be derived from the BaseTest template.  I've made a macro
to help with this -- the expanded version is below:

class BasicTest: public BaseTest<BasicResult> {
public:
        GLEAN_CLASS(BasicTest, BasicResult);
};


This expands into:

class BasicTest: public BaseTest<BasicResult> {
public:
        BasicTest(const char* aName, const char* aFilter,     
             const char* aDescription):                  
                BaseTest(aName, aFilter, aDescription) { 
                fWidth  = WIDTH;                         
                fHeight = HEIGHT;                        
                testOne = ONE;                           
        }                                                
        BasicTest(const char* aName, const char* aFilter,     
             const char* anExtensionList,                
             const char* aDescription):                  
                BaseTest(aName, aFilter, anExtensionList, aDescription) {
                fWidth  = WIDTH;                                         
                fHeight = HEIGHT;                                        
        }                                                                
        virtual ~BasicTest() {}                                               
                                                                         
        virtual void runOne(BasicResult& r, Window& w);                       
        virtual void compareOne(BasicResult& oldR, BasicResult& newR);
        virtual void logOne(BasicResult& r)
};


Then you make the rest of the test:

void BasicTest::runOne(BasicResult& r, Window&) { r.pass = true; }

void BasicTest::logOne(BasicResult& r) {
        logPassFail(r);
        logConcise(r);
}

void BasicTest::compareOne(BasicResult& oldR, BasicResult& newR) {
        comparePassFail(oldR, newR);
}

[NOTE: you do *NOT* provide your own run and compare functions -- these
you get from BaseTest.  If you only want to run for one configuration,
then you set testOne = true in your constructor.]




RATIONALE

My idea is that runOne won't print anything and that logOne will always
be called after runOne.  This seemed reasonable since it would be nice
to be able to have the comparison function print out _exactly_ the same
messages that the test itself prints out.  I know that a lot of tests
don't do this now, and I think part of the reason is that it's so hard
to get information into and out of a results file.  It is my sincere
hope that the new form of the Results class will make it trivially easy
to save, log, and print all of the information you'd like to.  If it's
not easy enough, I'd like to make it easier if anyone can figure out
how.

[Note that if you just want a pass/fail, you can derive from BasicTest.
However, it's really simple now to derive from BaseTest<YourResult> and
make your results as complex or as simple as you'd like.  You can start
by copying the tbasic.* files -- they're minimal and simple.]




FUTURE WORK

I think what we also need is a logStats() method that will print out all
the interesting information.  This can be used in compareOne when
verbosity is on:

        if (env->options.verbosity) {
                env->log << env->options.db1Name << ':';
                logStats(oldR);
                env->log << env->options.db2Name << ':';
                logStats(newR);
        }

and can mean that a single logOne function could would be sufficient for
most tests, e.g:

void BasicPerfTest::logOne(BasicPerfResult& r) {
        logPassFail(r);
        logConcise(r);
        logStats(r);
}

(In which case I'd like to make this part of BaseTest and *NOT* have the
code duplicated in each individual test.)


I'd also like to standardize logging more and reduce code duplication
there, too, but that seemed like more work than I have time for right
now.