Re: [Flightgear-devel] RFC: Working on a testing strategy for FlightGear.

[James T. <ja...@fl...>]

> 
> My idea would be to include it into the 3rdparty/ directory.  The
> current version from
> https://www.freedesktop.org/wiki/Software/cppunit/ is pretty light:
> 
> Removing the examples directory, it is 3.2 Mb.  Comparing to the
> current 3rdparty packages:

Yeah agreed, if we can go that route it’s fine with me. Just be ware we then need a SYSTEM_CPPUNIT option to keep the distros happy :)

> I think this simplicity for test writers would make it easier for more
> potential developers to learn about the fgfs internals.  They could
> start contributing to fgfs by writing unit or system/functional tests
> to learn about the fgfs internal design.  If we had good test
> coverage, then you could say that all new developments need to come
> with one or more tests.  That is how I do things.

Well, personally I do not think the complexity for new developers (or old ones) is really so different between any of the test frameworks (our own macros or CppUnit or whatever), but, okay, it’s not such a big concern.
> 
> With the current monolithic design, the binary file is 215 Mb.  But
> with the current set of 8 unit and simgear tests, the whole test suite
> execution, including startup time, is 30-40 ms.  It does have more of
> an effect on gdb and valgrind though.

Yeah that is a slightly concern, since again debugging something like a terraSync issue is much easier in the test harness code than inside FG. So I am slightly concerned by that. This is why I went for the shared library approach - do you think that’s really not viable with CppUnit? or do you have some other reason to avoid it?

For me it has some ugly hacks to make the test-lib, but it’s proved relatively low hassle to maintain, and easy to make tests as ‘big’ or ‘small’ as I needed, and still run quickly. And I don’t think the test architecture relates to the test support library we use or don’t use - unless I am missing something?

> 
> In this workflow, do you run the individual tests or all tests?  If we
> end up with 100% test coverage, you would have to run individual tests
> otherwise it will take too long.  That is the idea with CppUnit, you
> would write code and then run:

I run all the tests in say catalog_test or test_repository or test_flight_plan - i.e these 4 or 8 or 20 tests should take maybe 15 or 20 seconds maximum, and then I know my hacking worked and I can continue. This is not the only workflow possible of course but it’s certainly the most productive for me.

Kind regards,
James