Re: [Plplot-devel] Release status: call for comprehensive testing

[Hazen B. <hba...@ma...>]

On 04/10/2015 09:06 PM, Alan W. Irwin wrote:
> On 2015-04-10 12:29-0400 Hazen Babcock wrote:
>
>>>
>>> That's a fairly sparse but still acceptable comprehensive test result
>>> for
>>> this release, but to start the next release cycle properly I strongly
>>> encourage everybody here to learn to run the
>>> scripts/comprehensive_test.sh bash script to completion on all
>>> platforms accessible to you (taking the approach that you should
>>> disable any PLplot component that has errors in order to get the
>>> script to complete).  Such tests are encouraged for essentially all
>>> platforms (e.g, MSVC, Cygwin, MinGW/MSYS, MinGW-w64/MSYS2, Mac OS X,
>>> all varieties of Linux distributions, and other Unices).
>>
>> I just tried to run the non-interactive comprehensive tests on lubuntu
>> without success. Perhaps it is because I'm not running the tests
>> properly? The QT devices that are causing the problems work fine if I
>> run them by hand. I looked on the sourceforge wiki but I could not
>> find and instructions on how to do this. Attached is what I tried and
>> the results of the test.
>
> Hi Hazen:
>
> Thanks for helping out with comprehensive testing.
>
> To answer your question, you should look at
> <https://sourceforge.net/p/plplot/wiki/Testing_PLplot/> in general and
> <https://sourceforge.net/p/plplot/wiki/Testing_PLplot/#Comprehensive%20testing>
>
> and
> https://sourceforge.net/p/plplot/wiki/Testing_PLplot/#Testing%20Reports>
> in specific for directions and results concerning comprehensive testing.
>
> The script results require no special setup other than what you do for
> your hand-crafted builds.  Greg had similar problems (but on the
> OpenSUSE platform) with segfaults for all Qt related ctest results.
>
> I suspect these severe memory management issues (which sometimes but
> not always will generate segfaults like you and Greg have seen) have
> nothing to do with PLplot but are instead due to bad Qt libraries or
> bad Qt library dependencies on some platforms. In contrast to your
> result and Greg's I have looked hard at qt results with valgrind on
> Debian stable, and they have no severe memory management issues with
> Qt4.8, and I think Andrew has done the same for his various Debian and
> Ubuntu platforms.  Andrew and I do have different severe memory
> management results for the Qt5 case; I get them with the epa_build of
> Qt5, and he does not for several different distros.  But Qt4 has
> always been completely reliable in this regard for us _for the
> platforms we have access to_.  But this is obviously not the case for
> you and Greg.
>
> Anyhow, for now we are just trying to keep track of exactly which
> platforms have good qt results with Qt4.8 and which do not.  However, we
> are
> also interested in exactly which components of PLplot work for all our
> different major configurations (shared libraries/dynamic devices,
> shared libraries/non-dynamic devices, and static libraries/non-dynamic
> devices. Thus, please try to get the script to finish
> with components removed that are giving trouble.  (I gave that same
> advice to Greg).
>
> So, for example, to
> drop everything Qt related you should specify
>
> --cmake_added_options "DEFAULT_NO_QT_DEVICES=ON -DENABLE_qt=OFF"
>
> when attempting to run scripts/comprehensive_testing.sh again.
>
> Once you get that script to complete, we will note in the
> corresponding report on the Wiki exactly which components had to be
> dropped and why.

It worked fine without the Qt components. Is there an output file that I 
should send?

Why do you think these Qt components fail in the tests, but work fine if 
they are run independently? What is the testing framework doing that is 
not being done from the command line?

-Hazen