Re: [Gramps-devel] unit testing -- review and recommendations

[James G. S. (jim) <jg...@sa...>]

Benny Malengier wrote:
>..
> 1/tests: yes.
> I just finished adding a test in 'if __main__' to Relationship.py, that is
> usable in all rel_xx.py modules. I fail to see if tests exist, and a patch
> comes in, why the test should not be rerun. If it fails it means we have a
> bug, so running it is good.
> However, I never used the present setup in test, because frankly, I wouldn't
> know if a test exists for what I'm hacking on. So a) does not look like a
> good option to me.
> 
> So I clearly vote for c), with a name like 'unittest' for the dir. 't' is
> too short, it's hard to click on such small directory names.
> Note that c) is important as one will need sample files to compare output,
> eg test xml file, test output file, .... which the test will diff to find
> changes. One cannot have these text files clutter the normal directories.
> About the name of the file, perhaps one must allow for separate test files
> for one module? That is, if I'm hacking on something that means only one of
> the 10 tests must be run, can I easily run only that one test instead of all
> 10? I would like that to be the case, some tests might take some time to
> run.

Good points. "t" is admittedly ultra short. A hangover from perl habits
I suppose. The thinking goes something like:
  The test subdirectory will be of use to programmers who will come
  to recognize it whatever it is named, and a short name is easier
  to type in editor file-load commands.
But ok, I can understand the complaint that t is both ultra cryptic, as
well as actually being hard to see!  Hmmm, my thought now is that
"unittest" and even "tests" deviate a bit from some small amount of
convention that does exist, so I offer "test" as a compromise.

I do foresee "test" being a potential source of confusion with python's
top-level test package,  but _believe_ it shouldn't really cause
problems. In this respect "unittest" might cause problems running tests
on our top-level modules. The conservative choice might be to choose a
unique name rather than a conventional one. That argument might suggest
something like "gramps_test" or "gtest" or "utest" or "gutest" ...

==> What do you (and others) think. I'm leaning toward "test".

> 2/Maintaining tests. If you change something, you need to test it anyway.
> Hacking an existing tests is far easier than coming up with a test from
> scratch. Important is that the directory of testing is in src. I do grep
> searches to find code fragments only in src, as searching them higher also
> finds the .svn dir, the manual,  and such. Changes in functions must be
> changed in the test too, so the grep search should find them in src.

Yes, good points. I would amplify that the persistent unit tests become
more valuable with time (if they are kept current), because they help
detect accidental blunders like deleting the wrong function, or any of a
million others.

By the way, it's been there all along, but I recently discovered a grep
option (--include) I like to use in combination with recursion.

 grep -rl --include='*.py' "import unittest" src

> 3/About the _ before names. Don has started to remove these. They are an
> artifact of the past, and new directories should not contain them anymore.

So if we wanted, we could use a "test_" prefix without the objections I
offered earlier.  Still, I'm thinking I actually prefer a suffix for the
display in sorted lists. While we're here, I might criticize my
suggestion of using "_Test" as violating a semi-convention of python of
using/tolerating upper case, mixed case and camel case in various places
but vigorously disliking things like "_Test"  (preferring "_test") .. so
I'm now preferring "_test". Man these trivial questions get tedious, but
now is the time to spend a few extra minutes on them, I suppose.

==> I think I'd like to use names like RelationShipView_test.py

> 4/About testing rules. I don't think we should force tests in any way. Also
> not needed for everything. However, a framework you can add to the moment
> you think a test is needed, is very welcome.
> It is a win-win situation. If you use it, Gramps get better, if you want to
> use it, much of the supporting code is there already, if you don't use it,
> it doesn't bother you.

Well put! My thinking exactly!

>.. 
> PS: What I think is needed in the framework: ability to load a .gramps file
> without interface so you can do test code on it, like add a person, and
> check the result.

Good suggestion. We may need various support modules like
test_utility.py that can be used by test code.

Regards and thanks for the feedback,
..jim