From: Andrew R. <and...@us...> - 2013-12-19 22:29:14
|
On Thu, Dec 19, 2013 at 11:19:59AM -0800, Alan Irwin wrote: > On 2013-12-19 13:44-0000 Andrew Ross wrote: > > >On Thu, Dec 19, 2013 at 01:10:25PM +0000, Andrew Ross wrote: > >> > >>Tried to run build_plplot which fails straight away with > >> > >> 1%] Performing download step (verify and extract) for 'build_qt4_lite' > >>cd /tmp/build_plplot/build_dir-linux/epa_build/Source && /nfs/see-archive-15_a58/l > >>ecanr/software/plplot/build_script/install-linux_buildtools/bin/cmake -P /tmp/buil > >>d_plplot/build_dir-linux/epa_build/Stamp/build_qt4_lite/verify-build_qt4_lite.cmak > >>e > >>-- verifying file... > >> file='/home/software/build_qt4/qt-everywhere-opensource-src-4.8.5.tar.gz' > >>CMake Error at /tmp/build_plplot/build_dir-linux/epa_build/Stamp/build_qt4_lite/ve > >>rify-build_qt4_lite.cmake:5 (file): > >> file MD5 failed to read file > >> "/home/software/build_qt4/qt-everywhere-opensource-src-4.8.5.tar.gz": No > >> such file or directory > >> > >>Looks like this points to a file on Alan's hard disk rather than trying > >>to download. You probably want to comment out the temporary debugging line > >>before the release. I'll commit that change myself now. Having done that the > >>build is now chugging away. More later... > >> > > > >OK. So this fails for yelp-xsl because the source is compressed as a .xh > >file and my version of tar doesn't support this option. Looks like this > >will rule out the gnome stuff. I'm not sure how to fix / disable this so > >this is likely to be the extent of my epa testing for now, unless I find > >time to build a new version of tar. > > I believe I have now fixed that issue with revision 12888. > > And since you are currently failing on yelp-xsl, that implies that > build_plplot_lite (which does not depend on yelp) finished without > issues for you, and you went on to try build_plplot (which does > depend on yelp indirectly via the pango dependencies). > > Your results for epa_build on CentOS are extremely encouraging so far, > and I am glad to hear they have already been a practical help for you > on that platform. I'll give this a go and let you know how I get on. Andrew |
From: Andrew R. <and...@us...> - 2013-12-19 23:08:18
|
On Thu, Dec 19, 2013 at 10:29:00PM +0000, Andrew Ross wrote: > On Thu, Dec 19, 2013 at 11:19:59AM -0800, Alan Irwin wrote: > > I believe I have now fixed that issue with revision 12888. > > > > And since you are currently failing on yelp-xsl, that implies that > > build_plplot_lite (which does not depend on yelp) finished without > > issues for you, and you went on to try build_plplot (which does > > depend on yelp indirectly via the pango dependencies). > > > > Your results for epa_build on CentOS are extremely encouraging so far, > > and I am glad to hear they have already been a practical help for you > > on that platform. > > I'll give this a go and let you know how I get on. I can confirm using cmake -E tar rather than tar directly works fine. I've run into other problems though and run out of time for more testing. I'll focus on the qt_example problem. Andrew |
From: Andrew R. <and...@us...> - 2013-12-19 22:51:00
|
On Wed, Dec 18, 2013 at 09:35:52PM +0000, Andrew Ross wrote: > On Wed, Dec 18, 2013 at 09:06:53AM -0800, Alan Irwin wrote: > > To Andrew, Jerry, Arjen, and Phil: > > > > There are some important questions below for you. I need those > > questions answered for the release notes (README.release) for the > > forthcoming release. > > > > For example, when I am done with my testing, my test summary > > in README.release will read > > > > Comprehensive tests for a complete system build environment were run > > on 64-bit Debian Wheezy Linux on AMD-64 hardware. > > > > Comprehensive tests for a limited (qt, cairo, and wxwidgets PLplot > > components were dropped) epa_build environment were run on 64-bit > > Debian Wheezy Linux on AMD-64 hardware. > > > > Comprehensive tests for a complete epa_build environment were run > > on 64-bit Debian Wheezy Linux on AMD-64 hardware. > > > > Comprehensive tests for a limited (qt, cairo, and wxwidgets PLplot > > components were dropped) epa_build environment were run for 32-bit > > MinGW/MSYS/Wine on AMD-64 hardware. > > > > I need similar tests summaries from all of you that have run tests for > > this release. I give preliminary (and quite uncertain, see all the > > question marks) versions below. Please send back the corrected > > versions of these test summaries. In most cases the question marks > > should be quite easy to replace with your actual test data. However, > > the phrase "Limited tests??" is a bit of a special case. It should be > > replaced by one of "Comprehensive tests" (if you actually ran the > > scripts/comprehensive_test.sh script), a phrase summarizing the exact > > test you ran such as "ctest + the test_noninteractive target in the > > build tree" for intermediate testing, or "Limited tests" for those > > cases where you actually only did a few tests by hand. > > > > @Andrew: > > > > Limited tests?? for a complete system build environment were run > > on 64-bit?? Debian unstable on AMD-64 hardware?? > > I ran make test_noninteractive and test_interactive for a complete system > build environment on a 64-bit Debian unstable system on AMD-64 hardware. > > > Limited tests?? for a complete system build environment were run > > on 64-bit Ubuntu version?? on AMD-64 hardware? > > I ran make test_noninteractive and test_interactive for a complete system > build environment on a 64-bit Ubuntu Saucy (13.10) system on AMD-64 > hardware. > > I did not get chance to run the tests on 32-bit Debian unstable with > AMD-64 hardware (though I can using the same pbuilder setup used to test > Debian 64 bit) I've now also run the scripts/comprehensive_test.sh test on my Ubuntu system. I encountered some problems in that on several occasions, without warning, by kde session died. It was not reproducible, but in each case it was while running the test_interactive tests, mostly in the install tree. I've never seen this when running the test individually in serial so I don't know whether it is a parallel build problem or some hardware problem. Eventually I managed to get through almost all the tests though. The only issue I encountered was a segfault with the C++ qt_example for the nondyndriver case. Valgrind shows an invalid call to free / delete and a memory access violation. The shared case is fine with valgrind so I assume it is something about the static drivers. For the fully static case I do not get a segfault, but valgrind shows a load or invalid reads in wx libraries ?!? Alan, do your tests reproduce this? I'll try a debug build to see if I can get to the bottom of it. Andrew |
From: Alan W. I. <ir...@be...> - 2013-12-19 22:53:03
|
On 2013-12-19 22:26-0000 Andrew Ross wrote: > Cmake chose to build a static library, not me, so I assume something was > missing for building shared versions. I assume you are discussing just the PLplot part of the epa_build. Just like for ordinary PLplot builds the default should be shared libraries, see option(BUILD_SHARED_LIBS "Build shared libraries" ON) in cmake/modules/plplot.cmake. What does your cache file say about that option? I don't know of any logic in our build system that would turn it to OFF unless the user explicitly uses -DBUILD_SHARED_LIBS=OFF as a cmake option for an ordinary build or that variable was set for an epa_build of PLplot (which it isn't as far as I can tell). So I hope you can figure out what is going on here. >> [...]when I follow those directions here, I get the following RPATH results: >> >> -- TCL_RPATH = /home/wine/newstart/build_script/install-linux_buildtools/lib >> -- TCL_TK_RPATH = /home/wine/newstart/build_script/install-linux_buildtools/lib >> -- TCL_TK_ITCL_ITK_RPATH = /home/wine/newstart/build_script/install-linux_buildtools/lib;/home/wine/newstart/build_script/install-linux_buildtools/lib/itcl3.4;/home/wine/newstart/build_script/install-linux_buildtools/lib/itk3.3 >> >> If you don't get lines similar to the above, then those variables have >> been emptied because they point to the system locations as explained >> above. The above variables are used appropriately in the PLplot build >> system so at least for my case is not necessary for me to set >> LD_LIBRARY_PATH. > > I get just this, so cmake is setting the RPATH variables correctly. ldd > on libplplotd.so in the build directory looks fine, but ldd in the > install directory can't find itcl or itk. The tcl and tk libraries are > found fine in their epa install location. This is most odd... I just discovered the other day that readelf -d <library name> |grep -i rpath could help confirm what rpath values were actually being set for a library. Of course, "make VERBOSE=1" does that as well but buried so deep in details it is tough to figure out. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Andrew R. <and...@us...> - 2013-12-19 23:41:14
|
On Thu, Dec 19, 2013 at 02:52:55PM -0800, Alan Irwin wrote: > On 2013-12-19 22:26-0000 Andrew Ross wrote: > > > Cmake chose to build a static library, not me, so I assume something was > > missing for building shared versions. > > I assume you are discussing just the PLplot part of the epa_build. Just > like for ordinary PLplot builds the default should be shared > libraries, see > > option(BUILD_SHARED_LIBS "Build shared libraries" ON) > > in cmake/modules/plplot.cmake. > > What does your cache file say about that option? I don't know of any > logic in our build system that would turn it to OFF unless the user > explicitly uses -DBUILD_SHARED_LIBS=OFF as a cmake option for an > ordinary build or that variable was set for an epa_build of PLplot > (which it isn't as far as I can tell). So I hope you can figure out > what is going on here. I've probably not been clear. The problem arose with libharu not plplot. By default this builds both shared and static versions. The problem arose with the static build. For plplot I have libltdl missing so I end up with ENABLE_DYNDRIVERS:BOOL=OFF BUILD_SHARED_LIBS:BOOL=ON but that is an aside. > >> [...]when I follow those directions here, I get the following RPATH results: > >> > >> -- TCL_RPATH = /home/wine/newstart/build_script/install-linux_buildtools/lib > >> -- TCL_TK_RPATH = /home/wine/newstart/build_script/install-linux_buildtools/lib > >> -- TCL_TK_ITCL_ITK_RPATH = /home/wine/newstart/build_script/install-linux_buildtools/lib;/home/wine/newstart/build_script/install-linux_buildtools/lib/itcl3.4;/home/wine/newstart/build_script/install-linux_buildtools/lib/itk3.3 > >> > >> If you don't get lines similar to the above, then those variables have > >> been emptied because they point to the system locations as explained > >> above. The above variables are used appropriately in the PLplot build > >> system so at least for my case is not necessary for me to set > >> LD_LIBRARY_PATH. > > > > I get just this, so cmake is setting the RPATH variables correctly. ldd > > on libplplotd.so in the build directory looks fine, but ldd in the > > install directory can't find itcl or itk. The tcl and tk libraries are > > found fine in their epa install location. This is most odd... > > I just discovered the other day that > > readelf -d <library name> |grep -i rpath > > could help confirm what rpath values were actually being set for a library. > Of course, "make VERBOSE=1" does that as well but buried so deep in > details it is tough to figure out. In the install tree readelf -d libitcl3.4.so shows rpath is not set. On the other hand readelf -d libtcl8.6.so correctly sets rpath. Andrew |
From: Alan W. I. <ir...@be...> - 2013-12-20 01:09:18
|
On 2013-12-19 23:41-0000 Andrew Ross wrote: > In the install tree > > readelf -d libitcl3.4.so > shows rpath is not set. On the other hand > readelf -d libtcl8.6.so > correctly sets rpath. I confirm the above for the epa_build results here. And post-release I suppose I should modify the autotools epa_build of itcl3.4 (and other Tcl etc., components) to get rid of all potential rpath issues for those components. But I cannot figure out how the above situation actually creates problems for you. For example, I thought it might be possible because of the above that Itcl would not load properly when executing package require Itcl from either tclsh or wish. But our build system uses that sort of command a lot when figuring out the consistency of Tcl/Tk/Itcl/Itk/Iwidgets versions, and it appears to be working despite the autotools epa_build not setting rpath properly for certain components (such as Itcl) of Tcl/Tk/Itcl/Itk/Iwidgets. Furthermore, for the PLplot case we set rpath properly for pltcl and plserver, e.g., wine@raven> readelf -d ../install-linux/bin/pltcl |grep -i rpath 0x000000000000000f (RPATH) Library rpath: [/home/wine/newstart/build_script/install-linux/lib:/home/wine/newstart/build_script/install-linux_buildtools/lib:/home/wine/newstart/build_script/install-linux_buildtools/lib/itcl3.4:/home/wine/newstart/build_script/install-linux_buildtools/lib/itk3.3] wine@raven> readelf -d ../install-linux/bin/plserver |grep -i rpath 0x000000000000000f (RPATH) Library rpath: [/home/wine/newstart/build_script/install-linux/lib:/home/wine/newstart/build_script/install-linux_buildtools/lib:/home/wine/newstart/build_script/install-linux_buildtools/lib/itcl3.4:/home/wine/newstart/build_script/install-linux_buildtools/lib/itk3.3] So at least for those PLplot executables, all four of the Tcl/Tk/Itcl/Itk library locations should be found by the run-time loader. Is there some PLplot executable or library where I forgot to set rpath properly? Or would the potential rpath issues for the "package require Itcl" command not occur when just finding the version of Itcl, but deeper use (say like in the pltcl_standard_examples or wish_standard_examples scripts) finally cause issues on some systems? So to figure out why the rpath issues for the epa_build of Tcl/Tk/Itcl/Itk/Iwidgets are affecting PLplot, I need more details about exactly where the PLplot build or test problem is occurring for your platform. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-12-20 00:11:10
|
On 2013-12-19 22:50-0000 Andrew Ross wrote: > I've now also run the scripts/comprehensive_test.sh test on my Ubuntu > system. I encountered some problems in that on several occasions, > without warning, by kde session died. It was not reproducible, but in > each case it was while running the test_interactive tests, mostly in > the install tree. I've never seen this when running the test > individually in serial so I don't know whether it is a parallel build > problem or some hardware problem. I have never seen anything like that from my KDE desktop despite running scripts/comprehensive_test.sh many times recently with a high degree of parallel interactive results. However, right in the middle of one of my _non_ interactive tests my X server (from my X-terminal where I am displaying results) died, and that killed everything. But that only happened once in the last week and never before or after. So since that is not repeatable, I am going to attribute it to a networking (since X-terminals work over the network), hardware, or X server glitch. But your situation with repeated KDE issues sounds quite different, and another possibility to consider there is that your version of KDE may not be as stable as mine. If I were in your situation, I would shift to some much simpler desktop (fvwm is the one I am familiar with, but you may have your own favorite) to see if the problems continue. > > Eventually I managed to get through almost all the tests though. The > only issue I encountered was a segfault with the C++ qt_example for the > nondyndriver case. Valgrind shows an invalid call to free / delete and > a memory access violation. The shared case is fine with valgrind so I > assume it is something about the static drivers. > > For the fully static case I do not get a segfault, but valgrind shows a > load or invalid reads in wx libraries ?!? > > Alan, do your tests reproduce this? I'll try a debug build to see if I > can get to the bottom of it. qt_example has had a severe memory management issue in the recent past (say over the last year or so) which I worked around by not permitting qt_example to be built for the static drivers case (e.g., for the PLplot-5.9.10 release). But my recent build-system changes for the Qt4-related parts of the PLplot build solved that issue (as confirmed by valgrind) so I enabled qt_example again for the static drivers case. But in light of your seeing more symptoms, I will check that again with valgrind runs for my system version of Qt4 and also the epa_build version of Qt4 and get back to you with those results. Please keep in touch on what your further investigation finds as well. It's possible the qt_example memory management trouble only occurs for certain versions of Qt4. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-12-20 02:26:57
|
On 2013-12-19 16:11-0800 Alan W. Irwin wrote: > On 2013-12-19 22:50-0000 Andrew Ross wrote: >> Eventually I managed to get through almost all the tests though. The >> only issue I encountered was a segfault with the C++ qt_example for the >> nondyndriver case. Valgrind shows an invalid call to free / delete and >> a memory access violation. The shared case is fine with valgrind so I >> assume it is something about the static drivers. > >> >> For the fully static case I do not get a segfault, but valgrind shows a >> load or invalid reads in wx libraries ?!? >> >> Alan, do your tests reproduce this? I'll try a debug build to see if I >> can get to the bottom of it. > > qt_example has had a severe memory management issue in the recent past > (say over the last year or so) > which I worked around by not permitting qt_example to be built for the > static drivers case (e.g., for the PLplot-5.9.10 release). But my > recent build-system changes for the Qt4-related parts of the PLplot > build solved that issue (as confirmed by valgrind) so I enabled > qt_example again for the static drivers case. > > But in light of your seeing more symptoms, I will check that again > with valgrind runs for my system version of Qt4 and also the epa_build > version of Qt4 and get back to you with those results. Please keep in > touch on what your further investigation finds as well. It's possible > the qt_example memory management trouble only occurs for certain > versions of Qt4. Here is the build-tree result for the system version of Qt4 (i.e., -- Found Qt4: /usr/bin/qmake (found suitable version "4.8.2", minimum required is "4.8.2") -- NP_QT_COMPILE_DEFINITIONS = QT_SVG_LIB;QT_GUI_LIB;QT_CORE_LIB -- QT_COMPILE_DEFINITIONS = -DQT_SVG_LIB;-DQT_GUI_LIB;-DQT_CORE_LIB -- NP_QT_INCLUDE_DIRECTORIES = /usr/include/qt4;/usr/include/qt4/QtSvg;/usr/include/qt4/QtGui;/usr/include/qt4/QtCore -- QT_INCLUDE_DIRECTORIES = -isystem /usr/include/qt4;-isystem /usr/include/qt4/QtSvg;-isystem /usr/include/qt4/QtGui;-isystem /usr/include/qt4/QtCore ) for the -DENABLE_DYNDRIVERS=OFF case and after running "make test_qt_example" to build all the dependencies of qt_example: ==31199== Memcheck, a memory error detector ==31199== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==31199== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==31199== Command: examples/c++/qt_example ==31199== ==31199== ==31199== HEAP SUMMARY: ==31199== in use at exit: 275,941 bytes in 2,457 blocks ==31199== total heap usage: 44,202 allocs, 41,745 frees, 15,616,431 bytes allocated ==31199== ==31199== LEAK SUMMARY: ==31199== definitely lost: 3,888 bytes in 37 blocks ==31199== indirectly lost: 11,082 bytes in 223 blocks ==31199== possibly lost: 2,448 bytes in 7 blocks ==31199== still reachable: 258,523 bytes in 2,190 blocks ==31199== suppressed: 0 bytes in 0 blocks ==31199== Rerun with --leak-check=full to see details of leaked memory ==31199== ==31199== For counts of detected and suppressed errors, rerun with: -v ==31199== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 12 from 8) Here is the equivalent -DENABLE_DYNDRIVERS=OFF build-tree results for the epa_build version of Qt4 (which I obtained by running "make build_qt4_lite" under epa_build and simply changing the PATH prior to my ordinary PLplot build so that qmake version was found (which sucks in the rest of that version of Qt4). -- Found Qt4: /home/wine/newstart/build_script/install-linux/bin/qmake (found suitable version "4.8.5", minimum required is "4.8.2") -- NP_QT_COMPILE_DEFINITIONS = QT_SVG_LIB;QT_GUI_LIB;QT_CORE_LIB -- QT_COMPILE_DEFINITIONS = -DQT_SVG_LIB;-DQT_GUI_LIB;-DQT_CORE_LIB -- NP_QT_INCLUDE_DIRECTORIES = /home/wine/newstart/build_script/install-linux/include;/home/wine/newstart/build_script/install-linux/include/QtSvg;/home/wine/newstart/build_script/install-linux/include/QtGui;/home/wine/newstart/build_script/install-linux/include/QtCore -- QT_INCLUDE_DIRECTORIES = -isystem /home/wine/newstart/build_script/install-linux/include;-isystem /home/wine/newstart/build_script/install-linux/include/QtSvg;-isystem /home/wine/newstart/build_script/install-linux/include/QtGui;-isystem /home/wine/newstart/build_script/install-linux/include/QtCore ==5108== Memcheck, a memory error detector ==5108== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. ==5108== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info ==5108== Command: examples/c++/qt_example ==5108== ==5108== ==5108== HEAP SUMMARY: ==5108== in use at exit: 244,694 bytes in 2,230 blocks ==5108== total heap usage: 42,734 allocs, 40,504 frees, 15,350,113 bytes allocated ==5108== ==5108== LEAK SUMMARY: ==5108== definitely lost: 1,000 bytes in 23 blocks ==5108== indirectly lost: 5,518 bytes in 65 blocks ==5108== possibly lost: 2,448 bytes in 7 blocks ==5108== still reachable: 235,728 bytes in 2,135 blocks ==5108== suppressed: 0 bytes in 0 blocks ==5108== Rerun with --leak-check=full to see details of leaked memory ==5108== ==5108== For counts of detected and suppressed errors, rerun with: -v ==5108== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 6) These valgrind results and the ones above show memory management is fine (except for some leak issues) for both these cases on my platform so I do not confirm the bad valgrind results for the -DENABLE_DYNDRIVERS=OFF qt_example case you observe on your platform. To investigate further whether it is the Qt4 version accessible on your platform that is the cause of the qt_example memory management issue on your platform, I suggest you might want to try the build_qt4_lite target of epa_build yourself to see if you can match my good results for that case. (That "lite" Qt4 build only takes ~15 minutes on my computer and satisfies all of the ordinary PLplot build needs according to my tests.) Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Andrew R. <and...@us...> - 2013-12-20 21:19:24
|
On Thu, Dec 19, 2013 at 06:26:48PM -0800, Alan Irwin wrote: > On 2013-12-19 16:11-0800 Alan W. Irwin wrote: > > > On 2013-12-19 22:50-0000 Andrew Ross wrote: > >> Eventually I managed to get through almost all the tests though. The > >> only issue I encountered was a segfault with the C++ qt_example for the > >> nondyndriver case. Valgrind shows an invalid call to free / delete and > >> a memory access violation. The shared case is fine with valgrind so I > >> assume it is something about the static drivers. > > > >> > >> For the fully static case I do not get a segfault, but valgrind shows a > >> load or invalid reads in wx libraries ?!? > >> > >> Alan, do your tests reproduce this? I'll try a debug build to see if I > >> can get to the bottom of it. > > > > qt_example has had a severe memory management issue in the recent past > > (say over the last year or so) > > which I worked around by not permitting qt_example to be built for the > > static drivers case (e.g., for the PLplot-5.9.10 release). But my > > recent build-system changes for the Qt4-related parts of the PLplot > > build solved that issue (as confirmed by valgrind) so I enabled > > qt_example again for the static drivers case. > > > > But in light of your seeing more symptoms, I will check that again > > with valgrind runs for my system version of Qt4 and also the epa_build > > version of Qt4 and get back to you with those results. Please keep in > > touch on what your further investigation finds as well. It's possible > > the qt_example memory management trouble only occurs for certain > > versions of Qt4. > > Here is the build-tree result for the system version of Qt4 (i.e., > > -- Found Qt4: /usr/bin/qmake (found suitable version "4.8.2", minimum > required is "4.8.2") > -- NP_QT_COMPILE_DEFINITIONS = QT_SVG_LIB;QT_GUI_LIB;QT_CORE_LIB > -- QT_COMPILE_DEFINITIONS = -DQT_SVG_LIB;-DQT_GUI_LIB;-DQT_CORE_LIB > -- NP_QT_INCLUDE_DIRECTORIES = /usr/include/qt4;/usr/include/qt4/QtSvg;/usr/include/qt4/QtGui;/usr/include/qt4/QtCore > -- QT_INCLUDE_DIRECTORIES = -isystem /usr/include/qt4;-isystem /usr/include/qt4/QtSvg;-isystem /usr/include/qt4/QtGui;-isystem /usr/include/qt4/QtCore > ) for the -DENABLE_DYNDRIVERS=OFF case and after > running "make test_qt_example" to build all the dependencies of > qt_example: > > ==31199== Memcheck, a memory error detector > ==31199== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. > ==31199== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info > ==31199== Command: examples/c++/qt_example > ==31199== > ==31199== > ==31199== HEAP SUMMARY: > ==31199== in use at exit: 275,941 bytes in 2,457 blocks > ==31199== total heap usage: 44,202 allocs, 41,745 frees, 15,616,431 bytes allocated > ==31199== > ==31199== LEAK SUMMARY: > ==31199== definitely lost: 3,888 bytes in 37 blocks > ==31199== indirectly lost: 11,082 bytes in 223 blocks > ==31199== possibly lost: 2,448 bytes in 7 blocks > ==31199== still reachable: 258,523 bytes in 2,190 blocks > ==31199== suppressed: 0 bytes in 0 blocks > ==31199== Rerun with --leak-check=full to see details of leaked memory > ==31199== > ==31199== For counts of detected and suppressed errors, rerun with: -v > ==31199== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 12 from 8) > > Here is the equivalent -DENABLE_DYNDRIVERS=OFF build-tree results for the epa_build version of Qt4 (which I > obtained by running "make build_qt4_lite" under epa_build > and simply changing the PATH prior to my ordinary PLplot build > so that qmake version was found (which sucks in the rest of that > version of Qt4). > > -- Found Qt4: /home/wine/newstart/build_script/install-linux/bin/qmake (found suitable version "4.8.5", minimum required is "4.8.2") > -- NP_QT_COMPILE_DEFINITIONS = QT_SVG_LIB;QT_GUI_LIB;QT_CORE_LIB > -- QT_COMPILE_DEFINITIONS = -DQT_SVG_LIB;-DQT_GUI_LIB;-DQT_CORE_LIB > -- NP_QT_INCLUDE_DIRECTORIES = /home/wine/newstart/build_script/install-linux/include;/home/wine/newstart/build_script/install-linux/include/QtSvg;/home/wine/newstart/build_script/install-linux/include/QtGui;/home/wine/newstart/build_script/install-linux/include/QtCore > -- QT_INCLUDE_DIRECTORIES = -isystem /home/wine/newstart/build_script/install-linux/include;-isystem /home/wine/newstart/build_script/install-linux/include/QtSvg;-isystem /home/wine/newstart/build_script/install-linux/include/QtGui;-isystem /home/wine/newstart/build_script/install-linux/include/QtCore > > ==5108== Memcheck, a memory error detector > ==5108== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al. > ==5108== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright > info > ==5108== Command: examples/c++/qt_example > ==5108== > ==5108== > ==5108== HEAP SUMMARY: > ==5108== in use at exit: 244,694 bytes in 2,230 blocks > ==5108== total heap usage: 42,734 allocs, 40,504 frees, 15,350,113 > bytes allocated > ==5108== > ==5108== LEAK SUMMARY: > ==5108== definitely lost: 1,000 bytes in 23 blocks > ==5108== indirectly lost: 5,518 bytes in 65 blocks > ==5108== possibly lost: 2,448 bytes in 7 blocks > ==5108== still reachable: 235,728 bytes in 2,135 blocks > ==5108== suppressed: 0 bytes in 0 blocks > ==5108== Rerun with --leak-check=full to see details of leaked memory > ==5108== > ==5108== For counts of detected and suppressed errors, rerun with: -v > ==5108== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 6) > > These valgrind results and the ones above show memory management is > fine (except for some leak issues) for both these cases on my platform > so I do not confirm the bad valgrind results for the > -DENABLE_DYNDRIVERS=OFF qt_example case you observe on your platform. > > To investigate further whether it is the Qt4 version accessible on > your platform that is the cause of the qt_example memory management > issue on your platform, I suggest you might want to try the > build_qt4_lite target of epa_build yourself to see if you can match my > good results for that case. (That "lite" Qt4 build only takes ~15 > minutes on my computer and satisfies all of the ordinary PLplot build > needs according to my tests.) Hi Alan, Using the epa qt_lite approach as you suggest I still see the problem. Output of cmake is identical to you: -- Found Qt4: /home/andrew/software/build_script/install-linux/bin/qmake (found suitable version "4.8.5", minimum required is "4.8.2") -- NP_QT_COMPILE_DEFINITIONS = QT_SVG_LIB;QT_GUI_LIB;QT_CORE_LIB -- QT_COMPILE_DEFINITIONS = -DQT_SVG_LIB;-DQT_GUI_LIB;-DQT_CORE_LIB -- NP_QT_INCLUDE_DIRECTORIES = /home/andrew/software/build_script/install-linux/include;/home/andrew/software/build_script/install-linux/include/QtSvg;/home/andrew/software/build_script/install-linux/include/QtGui;/home/andrew/software/build_script/install-linux/include/QtCore -- QT_INCLUDE_DIRECTORIES = -isystem /home/andrew/software/build_script/install-linux/include;-isystem /home/andrew/software/build_script/install-linux/include/QtSvg;-isystem /home/andrew/software/build_script/install-linux/include/QtGui;-isystem /home/andrew/software/build_script/install-linux/include/QtCore -- pc_qt_COMPILE_FLAGS = -I/home/andrew/software/build_script/install-linux/include -I/home/andrew/software/build_script/install-linux/include/QtSvg -I/home/andrew/software/build_script/install-linux/include/QtGui -I/home/andrew/software/build_script/install-linux/include/QtCore ==18128== Memcheck, a memory error detector ==18128== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. ==18128== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==18128== Command: ./qt_example ==18128== ==18128== Invalid free() / delete / delete[] / realloc() ==18128== at 0x4C2BADC: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==18128== by 0x6DD0479: __cxa_finalize (cxa_finalize.c:55) ==18128== by 0x62CFD12: ??? (in /home/andrew/software/plplot/build_nondyn2/src/libplplotd.so.12.0.0) ==18128== by 0x400FF26: _dl_fini (dl-fini.c:253) ==18128== by 0x6DD0070: __run_exit_handlers (exit.c:77) ==18128== by 0x6DD00F4: exit (exit.c:99) ==18128== by 0x6DB5DEB: (below main) (libc-start.c:294) ==18128== Address 0xafb44b0 is 0 bytes inside a block of size 40 free'd ==18128== at 0x4C2BADC: operator delete(void*) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==18128== by 0x6DD0479: __cxa_finalize (cxa_finalize.c:55) ==18128== by 0x60A8F52: ??? (in /home/andrew/software/plplot/build_nondyn2/bindings/qt_gui/libplplotqtd.so.1.0.0) ==18128== by 0x400FF26: _dl_fini (dl-fini.c:253) ==18128== by 0x6DD0070: __run_exit_handlers (exit.c:77) ==18128== by 0x6DD00F4: exit (exit.c:99) ==18128== by 0x6DB5DEB: (below main) (libc-start.c:294) ==18128== ==18128== Invalid read of size 1 ==18128== at 0x5CCF71D: QObject::~QObject() (in /home/andrew/software/build_script/install-linux/lib/libQtCore.so.4.8.5) ==18128== by 0x60AE780: MasterHandler::~MasterHandler() (qt.h:96) ==18128== by 0x6DD0479: __cxa_finalize (cxa_finalize.c:55) ==18128== by 0x62CFD12: ??? (in /home/andrew/software/plplot/build_nondyn2/src/libplplotd.so.12.0.0) ==18128== by 0x400FF26: _dl_fini (dl-fini.c:253) ==18128== by 0x6DD0070: __run_exit_handlers (exit.c:77) ==18128== by 0x6DD00F4: exit (exit.c:99) ==18128== by 0x6DB5DEB: (below main) (libc-start.c:294) ==18128== Address 0x20 is not stack'd, malloc'd or (recently) free'd ==18128== ==18128== ==18128== Process terminating with default action of signal 11 (SIGSEGV) ==18128== Access not within mapped region at address 0x20 ==18128== at 0x5CCF71D: QObject::~QObject() (in /home/andrew/software/build_script/install-linux/lib/libQtCore.so.4.8.5) ==18128== by 0x60AE780: MasterHandler::~MasterHandler() (qt.h:96) ==18128== by 0x6DD0479: __cxa_finalize (cxa_finalize.c:55) ==18128== by 0x62CFD12: ??? (in /home/andrew/software/plplot/build_nondyn2/src/libplplotd.so.12.0.0) ==18128== by 0x400FF26: _dl_fini (dl-fini.c:253) ==18128== by 0x6DD0070: __run_exit_handlers (exit.c:77) ==18128== by 0x6DD00F4: exit (exit.c:99) ==18128== by 0x6DB5DEB: (below main) (libc-start.c:294) ==18128== If you believe this happened as a result of a stack ==18128== overflow in your program's main thread (unlikely but ==18128== possible), you can try to increase the size of the ==18128== main thread stack using the --main-stacksize= flag. ==18128== The main thread stack size used in this run was 8388608. ==18128== ==18128== HEAP SUMMARY: ==18128== in use at exit: 329,908 bytes in 4,265 blocks ==18128== total heap usage: 45,923 allocs, 41,659 frees, 7,854,870 bytes allocated ==18128== ==18128== LEAK SUMMARY: ==18128== definitely lost: 7,704 bytes in 35 blocks ==18128== indirectly lost: 4,592 bytes in 160 blocks ==18128== possibly lost: 4,676 bytes in 83 blocks ==18128== still reachable: 312,936 bytes in 3,987 blocks ==18128== suppressed: 0 bytes in 0 blocks ==18128== Rerun with --leak-check=full to see details of leaked memory ==18128== ==18128== For counts of detected and suppressed errors, rerun with: -v ==18128== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 2 from 2) Segmentation fault (core dumped) As you can see, even compiling with debug flags (-g) the output is not particularly enlightening. Looks like it is not the Qt version per se that is the problem. Perhaps g++ version? Anyway, I'm still not sure how to take this further. Since it is not reproducible by you and most Linux users with use dynamically loaded drivers I don't think this is release critical, but I would like to understand it. Andrew |
From: Alan W. I. <ir...@be...> - 2013-12-21 01:51:52
|
I am sorry to say my input mail chain has been silently disrupted by my University today. What perfect timing for them to screw up! However, I do have access to what people say (as long as you say it directly on plplot_devel) via http://sourceforge.net/p/plplot/mailman/plplot-devel/ As far as I can tell (what a lame interface!) only Andrew has said anything about the release since early this morning. Andrew Ross said: > As you can see, even compiling with debug flags (-g) the output is not particularly enlightening. Looks like it is not the Qt version per se that is the problem. Perhaps g++ version? Anyway, I'm still not sure how to take this further. Since it is not reproducible by you and most Linux users with use dynamically loaded drivers I don't think this is release critical, but I would like to understand it. This is a long shot, but I have to ask: did you remember to build all the dependencies first using "make test_qt_example"? I get very similar valgrind error reports to you if I forget to do that. Assuming the answer is yes, (i.e., you do have a real alarm rather than a false one), then I agree that g++ version comes to mind as a possible cause of this. Thank you for giving me your opinion this is not release critical. Following that hint I will move on without turning off qt_example for the non-dynamic drivers case, but I would appreciate you saying something about your finding memory management issues on one platform which are not present in my platform in README.release. Meanwhile, I have made some good progress today with the last pre-release task concerning MinGW/MSYS/Wine tests. A limited build of buildtools (just swig and pkg-config) went well as did build_plplot_lite with no run-time testing. I have currently just started with run-time testing using comprehensive_test.sh. It's going very slowly because that is the nature of the Wine beast; sometimes it takes 30x (!) longer to complete a computer task than the equivalent task run on Linux (principally because of horrendous startup latency for the thousands of little tiny tasks (such as each CMake progress report) that occurs for a given build and test). So far all ctest tests have passed and comprehensive_test.sh has now started the test_noninteractive target in the build tree. This is just for the shared libraries/dynamic devices case so there is still lots (!) more that comprehesive_test.sh will do. That is finish the shared library/dynamic driver case with test_noninteractive and test interactive in the build tree, installed examples tree, and (traditional) installed examples tree. Then it will do it all over again for the shared library/static devices case and also the static library/static devices case. However, there are no issues so far so it seems promising. IMPORTANT. I am declaring a commit freeze (for everything except things like README.release) at this point until the release process is done since I think there is a fairly good chance this final test will be a success overnight which would imply I could start the release process tomorrow (Saturday) morning (Pacific time). That involves lots of tasks that are listed in README.Release_Manager_Cookbook so even if I get an early start tomorrow in may be Sunday before I finish. Note again, e-mail communication with me is pretty problematic at the moment, but that could change at any time (whenever the University gets its act together about e-mail). For now, I am monitoring the plplot-devel archive via the web so say something there if you find anything that is incredibly release critical. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-12-20 02:55:51
|
I applaud everyone's recent efforts here to make this the best-tested release of PLplot ever. One side effect, however, is I have been responding (with my own short tests to confirm results or not) to e-mail all day from you guys. And there is still a qt_example issue and a Mac OS X build issue that are not completely resolved. That is fine, and I am game for some more such interactions tomorrow. After all, such collaboration is a good way to shake out these remaining PLplot bugs, and we should keep doing that until we can find nothing more we want to tackle before release. But this extremely productive debugging activity does mean I have been unable to to even start the MinGW/MSYS/Wine tests that I still need to do before release. Therefore, the actual date of the release is still very likely to be before the start of next week but obviously that time is still not finalized very well. By the way, this whole situation reminds me of Andrew's advice which supported the "earlier" December 14th release date. His reasoning was if our last-minute testing turned up any bugs (which turned out to be a large understatement) it would give us an extra week to deal with those while still being able to make the release before Christmas. Thanks for that extremely good advice, Andrew! Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-12-21 08:37:01
|
My tests went somewhat further than they ever had before on MinGW/MSYS/Wine. The test_noninteractive and test_interactive targets worked just as they had before in the build tree, but also this time ctest worked there as well. But I ran into errors when I attempted to extend the tests beyond that. For example, I could build the installed examples, but the Ada variants of those immediately errored out with a return code of 3 (presumably due to some Ada path transformation issues for MinGW/MSYS.) I have decided this result is good enough for this release cycle; I have proved at least we don't have a MinGW/MSYS regression for this release, and actually some improvement for that platform. To get tests of the MinGW/MSYS platform working for more than just the build tree will require additional post-release work. So roughly 8 hours from now (after I get some sleep) I plan to start the release process. N.B. Commit freeze is still in effect. N.B. My incoming e-mail chain is still broken so I will continue to monitor http://sourceforge.net/p/plplot/mailman/plplot-devel/ in case one of you has some urgent release news for me. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2013-12-22 06:41:27
|
Today I have gone through a large majority of the release process so I feel confident the actual release will be tomorrow (Sunday). Note the commit freeze is still in effect. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Arjen M. <Arj...@de...> - 2013-12-20 07:44:54
|
Hi Alan, will do. Regards, Arjen > -----Original Message----- > From: Alan W. Irwin [mailto:ir...@be...] > > Hi Arjen: > > Thanks for this information. > > I have attempted to incorporate the information above and everything else you have > said about your three different test platforms in revision 12889 of README.release. > Please carefully review that material and edit it if necessary. > DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail. |
From: Alan W. I. <ir...@be...> - 2013-12-21 17:17:35
|
Hi Phil: My incoming e-mail is still down, but I saw your note at http://sourceforge.net/p/plplot/mailman/plplot-devel/ giving more background information about your tests. I will update the appropriate paragraphs in README.release according to that information. Thanks! For everybody monitoring the plplot-devel list, I plan to start the actual release process for 5.9.11 in about an hour. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |