[Senserelate-developers] test cases and testing

[Ted P. <dul...@gm...>]

I've been working a little on WordNet-SenseRelate-AllWords testing -
as you may have
noticed we've been getting some failures on one particular case, and
it has to do with
what version of WordNet is being used. The original test cases were
probably developed
with 2.0 of WordNet, and with WordNet 3.0 some things have changed, in
particular
the numbering of some senses due to a fairly big change in the cntlist
that was shipped
with WordNet 3.0 (it's much smaller than the previous cntlists, and
seems to have
caused more shifting of sense numbers than is usual with a new version
of WordNet).

So, in thinking about how to do deal with this, I've come up with a
plan. For existing test
cases like we have in AllWords - let's leave the 2.0 tests in place,
and fix them up as
needed with version specific checks for 3.0 and beyond as needed. I am
in the process
of doing that.

For new test cases, let's just use WordNet 3.0 as our test key, and if
a user isn't using
3.0 let's just skip the tests.

I thought about trying to go back and do backwards compatability
testing for older versions
of WordNet, but it's not clear where that ends - I mean do we go back
to 2.1, 1.7.1, 1.7 .. ?
So, as we add test cases, let's add them to new .t files, and let's
leave the old .t files
"as is" with checks going forward from whatever version they were
written using. In the
case of AllWords, I'm pretty sure it was 2.0.

So, WordNet-SenseRelate-AllWords.t and wsd.t should be considered
"2.0" tests, that we
will keep current by checking for changes in new versions of WordNet,
and putting in version
checks that change the answer key accordingly (based on the version of
WordNet the
user has).

As new test cases are added, create new .t files, and clearly indicate
that these are 3.0
tests. Then, if there are future releases of WordNet we can check them
out and fix going
forward.

So, in all the .t files we need to check the user WordNet version, and
if that is less than
2.0 we should always just skip the test cases. We should also display
the version and
path info in our test script, so that we know what a user has
installed when they report it.

I hope this sounds ok...let me know what you think - I'll be releasing
a 0.09 of AllWords
today that hopefully exhibits all of the above.

Thanks!
Ted

-- 
Ted Pedersen
http://www.d.umn.edu/~tpederse