From: Alan W. I. <ir...@be...> - 2013-10-15 05:50:31
|
Hi Andrew: Happy (Canadian) Thanksgiving! Here are some recent test results I have obtained on my 32-bit MinGW/MSYS/Wine platform using the build_projects approach. c++ Missing examples : Differing postscript output : Missing stdout : Differing stdout : f95 Missing examples : Differing postscript output : Missing stdout : Differing stdout : python Missing examples : Differing postscript output : 00 03 04 05 06 08 09 11 12 14 14a 15 16 17 18 19 20 21 22 23 25 26 27 29 33 Missing stdout : Differing stdout : 23 ada Missing examples : Differing postscript output : Missing stdout : Differing stdout : adathick Missing examples : Differing postscript output : Missing stdout : Differing stdout : lua Missing examples : Differing postscript output : Missing stdout : Differing stdout : I am pretty proud of these results because this is the most extensive language coverage that I have obtained so far with Wine. C (the comparison language), and C++, Fortran 95, and Ada are all provided by the default MinGW-4.7.2 install that you get with the automated MinGW/MSYS installer. But MinGW also provides Lua which can be installed using the command (after the automatic install) mingw-get --recursive install mingw32-lua-dev mingw32-lua-bin It's possible that Lua result is the first PLplot Lua result obtained on the MSYS platform since it isn't obvious until you look pretty hard that Lua is available from MinGW. Python (the remaining language represented above) and the associated numpy must be downloaded from SourceForge and installed. (More about those Python results below.) The missing languages are Tcl, OCaml, Java, Octave, and D. I have plans to build Tcl with the build_projects approach. I presume the other languages can be downloaded in binary form for the Windows case, but I have not done so because even legitimate sources of Windows binaries can be compromised which implies the larger the set of different Windows binary vendors you use, the more chance of a security threat being introduced on your platform. Which is why I am trying to stick to just CMake (Kitware), MinGW/MSYS (SourceForge), and Python (SourceForge) for downloads with the idea of building the rest of what PLplot needs from source using the build_projects approach. (I guess ultimately I could drop both CMake and Python from that list and build them instead as well as OCaml, Java, Octave, and D, but that is a long way down the road.) Now I want to focus on the crummy Python results above. If I recall correctly, Arjen got similar bad Python results for a test he did on a 32-bit Microsoft Windows platform, and we blew these results off at the time with what I am now sure is a spurious argument that floating-point Python values are 32-bits on a 32-bit platform. Do you have access to a 32-bit Linux machine where you can do a similar test to confirm it is not some internal Python issue that occurs just for the 32-bit Windows case? To show this is a spurious argument, for 32-bit Python on a 32-bit Wine platform, I get the following results: >>> x = ones(10) >>> dx = 1.e-13*x >>> dxprime = (x + dx) - x >>> dxprime array([ 9.99200722e-14, 9.99200722e-14, 9.99200722e-14, 9.99200722e-14, 9.99200722e-14, 9.99200722e-14, 9.99200722e-14, 9.99200722e-14, 9.99200722e-14, 9.99200722e-14]) and similar for the x=1. case which demonstrates there are ~16 decimal digits (or 64-bits minus the exponent bits) of numerical precision in numpy floating point arrays by default, and similarly with ordinary Python floating-point variables even though it is a 32-bit platform. The Python documentation claims that virtually all Python platforms use 64 bit floating point (IEEE 754 floating-point to be exact) which those numerical tests help confirm. And the C examples should be similar on 32-bit platforms with all floating point numerical constants being represented as doubles (64-bit floating point). Therefore, from this argument I conclude that in general on 32-bit platforms the calculation of all floating-point numbers in the python and C examples should be proceeding with 64-bit floating-point precision. In the python case, the resulting 64-bit floating-point arguments of the python functions are transformed into the equivalent arguments of the C routines of the PLplot core library which implies the arguments used for all core library routines are identical to roughly 64 bits of precision between the C examples call and the corresponding Python examples call. So from such an argument I would expect 32-bit Python results rounded down to 4 figures as in the PostScript cases summarized above should be the same as the results of the rest of the languages on this 32-bit platform, but that expected result is obviously not confirmed by the above test results. I have looked in some detail at all such differences using (for number in $(seq 0 33); do echo $number ndiff x$(printf '%.2d\n' $number)[cp].psc done) |less Some examples (17, 19, 20, 22, 25, and 29) have heavy differences but the rest have small or no differences other than date. I explored the x00 case even further because it is a very simple case, and there is only one rounded PostScript difference. wine@raven> ndiff x00[cp].psc 6c6 < %%CreationDate: Sun Oct 13 23:25:17 2013 --- field 5 > %%CreationDate: Sun Oct 13 23:54:39 2013 310c310 < 1350 220 M --- field 1 relative error 7.41e-04 > 1349 220 M (N.B. ndiff can automatically be built by build_projects and is quite useful for such PostScript file difference analysis.) The "M" means a move is being made, and looking at the surrounding context to that one non-date difference between these two postscript files, that move occurs just before the start of writing the Y axis annotation "y=100 x#u2#d", i.e., the difference is a slight change in the position of that annotation. The relevant Python code from the example is xmin = 0. xmax = 1. ymin = 0. ymax = 100. # Prepare data to be plotted. x = arange(NSIZE) / float( NSIZE - 1 ) y = ymax*x**2 # Create a labelled box to hold the plot. plenv( xmin, xmax, ymin, ymax, 0, 0 ) pllab( "x", "y=100 x#u2#d", "Simple PLplot demo of a 2D line plot" ) Note, the x and y arrays have no influence on the position of the Y annotation (but I left the definition of those in the above listing for completeness.) In fact, the only floating-point data that should control the initial position of that Y axis label should be xmin, xmax, ymin, ymax, and how those numbers propagate to the C core library version of plenv. But my 32-bit Python experiments clearly indicate xmin, xmax, ymin, and ymax are identical precision to the corresponding C (and other language) results, and that was confirmed in detail when I ran gdb to check the arguments of the c_plenv C routine from our core library. They turned out to be identical between the Python and C cases as can be shown by the following gdb results (gdb) x/xg &xmin 0x60f728: 0x0000000000000000 (gdb) x/xg &xmax 0x60f720: 0x3ff0000000000000 (gdb) x/xg &ymin 0x60f718: 0x0000000000000000 (gdb) x/xg &ymax 0x60f710: 0x4059000000000000 for both cases. Furthermore, http://babbage.cs.qc.cuny.edu/IEEE-754.old/64bit.html translates those hex values as the IEEE 754 exact representations of the decimal floating numbers 0., 1., 0., and 100 (as expected, see the Python code above). If the c_plenv arguments are C core routine receives are identical to 64-bits as in the above C and Python example 00 cases, then it follows that the PLplot results should be absolutely identical. But obviously that is not the case so so I am completely floored by these results. One of my assumptions must be wrong so I would appreciate you taking a look as well with gdb assuming you have access to a 32-bit Linux machine. In sum, there is an issue with our Python results on 32-bit Windows that needs to be confirmed also on 32-bit Linux. All theoretical analysis and practical gdb results I can come up with says this issue cannot occur. So I must be missing something and need help to figure this out. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Arjen M. <Arj...@de...> - 2013-10-15 06:46:15
|
Hi Alan, > -----Original Message----- > From: Alan W. Irwin [mailto:ir...@be...] ... > > In sum, there is an issue with our Python results on 32-bit Windows that needs to be > confirmed also on 32-bit Linux. All theoretical analysis and practical gdb results I can > come up with says this issue cannot occur. So I must be missing something and > need help to figure this out. > Could this be an issue with "extended precision"? It is a bit of a horror that I deemed banished from the earth, but it may be lingering on. The problem with extended precision is that it makes reasoning about programs with floating-point numbers almost impossible. Extended precision uses 80 bits to store intermediate results that may at unpredictable moments in the computations get stored as 32- or 64-bits numbers. While the intermediate results may be more accurate, they cause havoc in the final results. Regards, Arjen DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail. |
From: Alan W. I. <ir...@be...> - 2013-10-15 16:08:06
|
On 2013-10-15 06:46-0000 Arjen Markus wrote: > Hi Alan, > >> -----Original Message----- >> From: Alan W. Irwin [mailto:ir...@be...] > > ... >> >> In sum, there is an issue with our Python results on 32-bit Windows that needs to be >> confirmed also on 32-bit Linux. All theoretical analysis and practical gdb results I can >> come up with says this issue cannot occur. So I must be missing something and >> need help to figure this out. >> > > Could this be an issue with "extended precision"? It is a bit of a horror that I deemed banished from > the earth, but it may be lingering on. The problem with extended precision is that it makes reasoning > about programs with floating-point numbers almost impossible. Extended precision uses 80 bits to > store intermediate results that may at unpredictable moments in the computations get stored as 32- or > 64-bits numbers. While the intermediate results may be more accurate, they cause havoc in the > final results. Good question. Actually extended precision (using 80-bits to store intermediate floating-point results) is just a part of the deal whenever you are running software on Intel hardware. It's actually a good thing from the numerical precision point of view because the end result is that the final 64-bit values have much less significance loss than they do on non-Intel equipment. So I disagree with the negative connations of your "cause havoc in final results" comment. However, I do agree with you that because Intel equipment tends to have higher numerical precision results than expected, you do have to be careful about your floating-point precision arguments, especially when comparing results between library versions that have been compiled in different ways so the order of computations and the decisions about when to use extended precision are done differently. That said, I don't think we can appeal to extended precision to explain these results since we are comparing results calculated for _exactly the same_ core C library. The floating-point arguments of the PLplot C routines are definitely 64-bits and not 80 bits (regardless of whether the operating system and/or hardware is 32-bit or 64-bit). For the 32-bit Wine and x00 case, I have proven with gdb that the 64-bit floating-point arguments to the c_plenv C routine of our core C library are identical when invoked from either C or Python. So with absolutely identical input to the _same_ core C library in both cases (so those identical arguments are processed absolutely identically with all computations done in the same order and with the same internal decisions about whether to use extended precision or not), how can that library possibly produce different results in the two cases? The only way I can think of to answer that question, is there is some uninitialized variable in our core C library which effectively adds some unpredictability to our results depending on how the memory for that uninitialized variable was used previously. And Python has a large memory footprint so perhaps it is effectively initializing that uninitialized variable in a way that is very different from every other language. (And this would "explain" why most of the Python standard example results are affected.) However, I should point out this variable in our core C library must be properly initialized in the Linux 64-bit case since x00c produces absolutely clean valgrind results in that case with the -dev psc device. I don't have valgrind available to me on Wine (and you don't on Microsoft windows, either). But if Andrew has access to 32-bit Linux, and if he confirms these bad Python results occur for that case, then I suggest the first experiment he should do is a valgrind run in the C case to check for uninitialized variables on whichever standard examples have the simplest differences. (That was x00 in my case, but it might not be in his case.) And, Arjen, if you again confirm these same bad 32-bit python results on your 32-bit Windows platform, you might want to use every proprietary Windows code analysis tool at your disposal (purify?) on that system to look for any programming issues with our core C library that might lead to variables that are uninitialized. Finally, I should emphasize the above hypothesis is fairly improbable, but it is the best I can come up with to explain these improbably different Python results on a 32-bit operating system that is designed carefully to perform floating-point computations with the same level of precision (64-bit with some 80-bit internally) as 64-bit operating systems. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Arjen M. <Arj...@de...> - 2013-10-16 08:00:28
|
Hi Alan, > -----Original Message----- > From: Alan W. Irwin [mailto:ir...@be...] > > Good question. Actually extended precision (using 80-bits to store intermediate > floating-point results) is just a part of the deal whenever you are running software on > Intel hardware. It's actually a good thing from the numerical precision point of view > because the end result is that the final 64-bit values have much less significance loss > than they do on non-Intel equipment. So I disagree with the negative connations of > your "cause havoc in final results" comment. > However, I do agree with you that because Intel equipment tends to have higher > numerical precision results than expected, you do have to be careful about your > floating-point precision arguments, especially when comparing results between > library versions that have been compiled in different ways so the order of > computations and the decisions about when to use extended precision are done > differently. > I will not claim to understand all details regarding extended precision, but as far as I do understand them, extended precision is controlled by the hardware instruction set, not so much by the human-readable source code. Furthermore, debuggers can be easily fooled: they deal with the numbers representable in ordinary precision, whereas the extra bits used in extended precision remain hidden. Because there are only a few registers capable of holding them, a program has to convert results back and forth and in a manner that does not allow analysis. Hence, the result may be more precise, but in an unpredictable way. A slight change in the order of operations and things can go another way. >From more or less recent newsgroup postings I gather that the newer processors do not include extended precision instructions - they are also bad for vectorisation. Some more information: http://www.vinc17.org/research/extended.en.html (One of my favourite arguments against extended precision is a simple program to determine the value of "epsilon", that is, the smallest number eps such that 1.0+eps can be distinguished from 1.0. With extended precision in charge, you get too small a value!) That said, I will have a look at the Python/Windows results, as I do not remember ever seeing such a long list of differing examples before. Regards, Arjen DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail. |
From: Alan W. I. <ir...@be...> - 2013-10-17 00:02:06
|
On 2013-10-16 08:00-0000 Arjen Markus wrote: > [...] I will have a look at the Python/Windows results, as I do not remember ever seeing > such a long list of differing examples before. Thanks in advance for that. It will be a big help to see exactly which 32-bit platforms have this problem. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Arjen M. <Arj...@de...> - 2013-10-17 19:52:07
|
Hi Alan, I am trying to get the Python bindings and examples ready under Windows so that I can compare the output with the C examples. However, I am getting a set of weird error messages. (I am not quite sure I have all the build tools right - I had some problems the past week with the stuff installed on my laptop) Anyway here are the messages: Microsoft (R) Program Maintenance Utility Version 10.00.30319.01 Copyright (C) Microsoft Corporation. All rights reserved. [ 0%] Built target csirocsa [ 1%] Built target deltaT-gen [ 1%] Built target tai-utc-gen [ 1%] Built target tai-utc.h_built [ 1%] Built target deltaT.h_built [ 1%] Built target qsastime [ 1%] Built target plhershey-unicode-gen [ 2%] Built target plhershey-unicode.h_built [ 12%] Built target plplotd [ 13%] Built target plplotcxxd [ 13%] Building C object bindings/python/CMakeFiles/_plplotcmodule.dir/plplotcmodulePYTHON_wrap.c.obj plplotcmodulePYTHON_wrap.c D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(3578) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(3581) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(3961) : error C2065: 'PyArray_FLOAT' : undeclared identifier D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(3962) : error C2065: 'PyArray_FLOAT' : undeclared identifier D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(5714) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(7188) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(7992) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(8123) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(8407) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(8425) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(8485) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(8777) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(8864) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(9112) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(9204) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(9315) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(9450) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(9561) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(11787) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(11992) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(13535) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(13552) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(14218) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(14348) : error C2036: 'void *' : unknown size D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(14545) : error C2036: 'void *' : unknown size NMAKE : fatal error U1077: 'C:\PROGRA~2\MICROS~3.0\VC\bin\cl.exe' : return code '0x2' Stop. NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\amd64\nmake.exe"' : return code '0x2' Stop. NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\BIN\amd64\nmake.exe"' : return code '0x2' Stop. The PyArray_FLOAT symbol is part of Numpy pre-1.7 and it appears in plplotcmodule.i but I do not know how to repair this. I also do not know what the "void *" messages are all about. Any suggestions, anyone? Regards, Arjen > -----Original Message----- > From: Alan W. Irwin [mailto:ir...@be...] > Sent: Thursday, October 17, 2013 2:02 AM > To: Arjen Markus > Cc: Andrew Ross; PLplot development list > Subject: RE: [Plplot-devel] Recent diff results for 32-bit Wine > > On 2013-10-16 08:00-0000 Arjen Markus wrote: > > > [...] I will have a look at the Python/Windows results, as I do not > > remember ever seeing such a long list of differing examples before. > > Thanks in advance for that. It will be a big help to see exactly which 32-bit platforms > have this problem. > > Alan > > __________________________ > Alan W. Irwin > > Astronomical research affiliation with Department of Physics and Astronomy, > University of Victoria (astrowww.phys.uvic.ca). > > Programming affiliations with the FreeEOS equation-of-state implementation for > stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); > PLplot scientific plotting software package (plplot.sf.net); the libLASi project > (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure > Project (lbproject.sf.net). > __________________________ > > Linux-powered Science > __________________________ DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail. |
From: Alan W. I. <ir...@be...> - 2013-10-17 22:31:31
|
On 2013-10-17 19:51-0000 Arjen Markus wrote: > Hi Alan, > > I am trying to get the Python bindings and examples ready under Windows so that I > can compare the output with the C examples. However, I am getting a set of weird error messages. > (I am not quite sure I have all the build tools right - I had some problems the past week with the stuff > installed on my laptop) > > Anyway here are the messages: > > > Microsoft (R) Program Maintenance Utility Version 10.00.30319.01 > Copyright (C) Microsoft Corporation. All rights reserved. > > [ 0%] Built target csirocsa > [ 1%] Built target deltaT-gen > [ 1%] Built target tai-utc-gen > [ 1%] Built target tai-utc.h_built > [ 1%] Built target deltaT.h_built > [ 1%] Built target qsastime > [ 1%] Built target plhershey-unicode-gen > [ 2%] Built target plhershey-unicode.h_built > [ 12%] Built target plplotd > [ 13%] Built target plplotcxxd > [ 13%] Building C object bindings/python/CMakeFiles/_plplotcmodule.dir/plplotcmodulePYTHON_wrap.c.obj > plplotcmodulePYTHON_wrap.c > D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON_wrap.c(3578) : error C2036: 'void *' : unknown size [...] > The PyArray_FLOAT symbol is part of Numpy pre-1.7 and it appears in plplotcmodule.i but I do not know how to repair this. > I also do not know what the "void *" messages are all about. > Hi Arjen: I believe you ran into Python trouble before when you were testing te_gen for me with MinGW/MSYS. I cannot recall whether that was resolved or not. Anyhow, I may be repeating what was discussed then, but please bear with me as I go through absolutely everything you should need. >From the above results, I suspect you are pointing to an incorrect Python header version so macros are not being defined properly ==> all kinds of compiler errors. Have you installed numpy to the correct location? I just now created a successful Python-2.7.3 + numeric installation on Wine. Here is what I did from my notes. Where appropriate drop the wine command prefix to do the equivalent Microsoft windows command. # Install python-2.7.3 wine msiexec /i python-2.7.3.msi In the GUI I chose the installation directory to be z:\home\wine\newstart\python\python-2.7.3 Everything else was default. # Install numpy that is consistent with python-2.7.3 wine numpy-1.5.1-win32-superpack-python2.7.exe The GUI found the above python-2.7.3 installation so I could choose everything to be default for the installation. Here is a quick check that all is well with that numpy installation: Get into the python-2.7.3 command-line by simply executing the python.exe command. Then from that python prompt, execute import numpy print numpy.get_include() The results here under Wine are Z:\home\wine\newstart\python\python-2.7.3\lib\site-packages\numpy\core\include (This is the method used by our CMake build system to discover the location of the numpy include files, see below for my results for that.) Here is how I set up Python before I do anything else on the bash.exe command line with MinGW/MSYS. You can just put these commands in a file which you run from bash.exe by source <filename> (which is what I do) or you can enter these values by hand from the bash,exe command line. # Directory where python.exe can be found. Tailor this to the right # location on your system (see above how that install prefix is determined when you install python). PYTHON_PATH=/z/home/wine/newstart/python/python-2.7.3 # Once the above line is tailored correctly, the rest follows automatically # Put Python on the PATH PATH=$PYTHON_PATH:$PATH # Help CMake find Python headers and library. # This form of the commands assumes that # both CMAKE_INCLUDE_PATH and CMAKE_LIBRARY_PATH already exist # as environment variables. CMAKE_INCLUDE_PATH=$PYTHON_PATH/include:$CMAKE_INCLUDE_PATH CMAKE_LIBRARY_PATH=$PYTHON_PATH/libs:$CMAKE_LIBRARY_PATH That's all that should be normally required to find and use Python from CMake. Assuming you have set up PATH, CMAKE_INCLUDE_PATH, and CMAKE_LIBRARY_PATH correctly for your Python installation, then the CMake output for the PLplot configuration should tell you everything you need to know about your Python setup. Here are the relevant cmake output lines from my recent successful builds with Python on MinGW/MSYS/Wine. -- Found PythonInterp: z:/home/wine/newstart/python/python-2.7.3/python.exe (found version "2.7.3") -- Found PythonLibs: z:/home/wine/newstart/python/python-2.7.3/libs/libpython27.a (found version "2.7.3") -- PYTHON_VERSION = 2.7.3 -- Building Python binding with plsmem() support PYTHON_EXECUTABLE: z:/home/wine/newstart/python/python-2.7.3/python.exe PYTHON_INCLUDE_PATH: z:/home/wine/newstart/python/python-2.7.3/include PYTHON_LIBRARIES: z:/home/wine/newstart/python/python-2.7.3/libs/libpython27.a NUMERIC_INCLUDE_PATH: Z:/home/wine/newstart/python/python-2.7.3/Lib/site-packages/numpy/core/include/numpy Do you get expected results for all of these locations once (a) python and numpy have been installed correctly as tested above with the "print numpy.get_include()" command, and (b) you have let CMake know about the appropriate Python locations using the environment variables PATH, CMAKE_INCLUDE_PATH, and CMAKE_LIBRARY_PATH? Let me know if there is anything I need to explain further in the above notes, and I hope that by following these notes you will achieve a permanent end to the Python troubles that you have been having. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Arjen M. <Arj...@de...> - 2013-10-18 04:25:32
|
Hi Alan, on my machine Python 2.7.5 is installed with Numpy 1.7 in a location c:\Python27\Lib\site-packages\numpy\core\include as reported by numpy.get_include(). So I think it is all correct. This installation came with the territory so to say - it was all packed together. >From the include files themselves I see that there is a new API and I guess that that is where the trouble is coming from. I will try and see if I can find a solution. Possibly I need to reinstall Python, using 2.7.3 and 1.5.1 instead. Regards, Arjen > -----Original Message----- > From: Alan W. Irwin [mailto:ir...@be...] > Sent: Friday, October 18, 2013 12:31 AM > To: Arjen Markus > Cc: Andrew Ross; PLplot development list > Subject: RE: [Plplot-devel] Recent diff results for 32-bit Wine > > On 2013-10-17 19:51-0000 Arjen Markus wrote: > > > Hi Alan, > > > > I am trying to get the Python bindings and examples ready under > > Windows so that I can compare the output with the C examples. However, I am > getting a set of weird error messages. > > (I am not quite sure I have all the build tools right - I had some > > problems the past week with the stuff installed on my laptop) > > > > Anyway here are the messages: > > > > > > Microsoft (R) Program Maintenance Utility Version 10.00.30319.01 > > Copyright (C) Microsoft Corporation. All rights reserved. > > > > [ 0%] Built target csirocsa > > [ 1%] Built target deltaT-gen > > [ 1%] Built target tai-utc-gen > > [ 1%] Built target tai-utc.h_built > > [ 1%] Built target deltaT.h_built > > [ 1%] Built target qsastime > > [ 1%] Built target plhershey-unicode-gen [ 2%] Built target > > plhershey-unicode.h_built [ 12%] Built target plplotd [ 13%] Built > > target plplotcxxd [ 13%] Building C object > > bindings/python/CMakeFiles/_plplotcmodule.dir/plplotcmodulePYTHON_wrap > > .c.obj > > plplotcmodulePYTHON_wrap.c > > D:\plplot-svn\build-windows-python\bindings\python\plplotcmodulePYTHON > > _wrap.c(3578) : error C2036: 'void *' : unknown size > [...] > > The PyArray_FLOAT symbol is part of Numpy pre-1.7 and it appears in > plplotcmodule.i but I do not know how to repair this. > > I also do not know what the "void *" messages are all about. > > > > Hi Arjen: > > I believe you ran into Python trouble before when you were testing te_gen for me > with MinGW/MSYS. I cannot recall whether that was resolved or not. Anyhow, I may > be repeating what was discussed then, but please bear with me as I go through > absolutely everything you should need. > > From the above results, I suspect you are pointing to an incorrect Python header > version so macros are not being defined properly ==> all kinds of compiler errors. > Have you installed numpy to the correct location? > > I just now created a successful Python-2.7.3 + numeric installation on Wine. Here is > what I did from my notes. Where appropriate drop the wine command prefix to do > the equivalent Microsoft windows command. > > # Install python-2.7.3 > wine msiexec /i python-2.7.3.msi > > In the GUI I chose the installation directory to be > > z:\home\wine\newstart\python\python-2.7.3 > > Everything else was default. > > # Install numpy that is consistent with python-2.7.3 > > wine numpy-1.5.1-win32-superpack-python2.7.exe > > The GUI found the above python-2.7.3 installation so I could choose everything to be > default for the installation. > > Here is a quick check that all is well with that numpy installation: > > Get into the python-2.7.3 command-line by simply executing the python.exe > command. Then from that python prompt, execute > > import numpy > print numpy.get_include() > > The results here under Wine are > > Z:\home\wine\newstart\python\python-2.7.3\lib\site-packages\numpy\core\include > > (This is the method used by our CMake build system to discover the location of the > numpy include files, see below for my results for that.) > > Here is how I set up Python before I do anything else on the bash.exe command line > with MinGW/MSYS. > You can just put these commands in a file which you run from bash.exe by > > source <filename> > > (which is what I do) or you can enter these values by hand from the bash,exe > command line. > > # Directory where python.exe can be found. Tailor this to the right # location on your > system (see above how that install prefix is determined when you install python). > PYTHON_PATH=/z/home/wine/newstart/python/python-2.7.3 > > # Once the above line is tailored correctly, the rest follows automatically > > # Put Python on the PATH > PATH=$PYTHON_PATH:$PATH > > # Help CMake find Python headers and library. > > # This form of the commands assumes that # both CMAKE_INCLUDE_PATH and > CMAKE_LIBRARY_PATH already exist # as environment variables. > > CMAKE_INCLUDE_PATH=$PYTHON_PATH/include:$CMAKE_INCLUDE_PATH > CMAKE_LIBRARY_PATH=$PYTHON_PATH/libs:$CMAKE_LIBRARY_PATH > > That's all that should be normally required to find and use Python from CMake. > > Assuming you have set up PATH, CMAKE_INCLUDE_PATH, and > CMAKE_LIBRARY_PATH correctly for your Python installation, then the CMake > output for the PLplot configuration should tell you everything you need to know about > your Python setup. > > Here are the relevant cmake output lines from my recent successful builds with > Python on MinGW/MSYS/Wine. > > -- Found PythonInterp: > z:/home/wine/newstart/python/python-2.7.3/python.exe (found version > "2.7.3") > -- Found PythonLibs: > z:/home/wine/newstart/python/python-2.7.3/libs/libpython27.a (found version "2.7.3") > -- PYTHON_VERSION = 2.7.3 > -- Building Python binding with plsmem() support > > PYTHON_EXECUTABLE: > z:/home/wine/newstart/python/python-2.7.3/python.exe > PYTHON_INCLUDE_PATH: > z:/home/wine/newstart/python/python-2.7.3/include > PYTHON_LIBRARIES: > z:/home/wine/newstart/python/python-2.7.3/libs/libpython27.a > NUMERIC_INCLUDE_PATH: > Z:/home/wine/newstart/python/python-2.7.3/Lib/site- > packages/numpy/core/include/numpy > > Do you get expected results for all of these locations once (a) python and numpy > have been installed correctly as tested above with the "print numpy.get_include()" > command, and (b) you have let CMake know about the appropriate Python locations > using the environment variables PATH, CMAKE_INCLUDE_PATH, and > CMAKE_LIBRARY_PATH? > > Let me know if there is anything I need to explain further in the above notes, and I > hope that by following these notes you will achieve a permanent end to the Python > troubles that you have been having. > > Alan > __________________________ > Alan W. Irwin > > Astronomical research affiliation with Department of Physics and Astronomy, > University of Victoria (astrowww.phys.uvic.ca). > > Programming affiliations with the FreeEOS equation-of-state implementation for > stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); > PLplot scientific plotting software package (plplot.sf.net); the libLASi project > (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure > Project (lbproject.sf.net). > __________________________ > > Linux-powered Science > __________________________ DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail. |
From: Alan W. I. <ir...@be...> - 2013-10-18 07:17:56
|
On 2013-10-18 04:25-0000 Arjen Markus wrote: > Hi Alan, > on my machine Python 2.7.5 is installed with Numpy 1.7 in a location c:\Python27\Lib\site-packages\numpy\core\include as reported by numpy.get_include(). So I think it is all correct. This installation came with the territory so to say - it was all packed together. > From the include files themselves I see that there is a new API and I guess that that is where the trouble is coming from. I will try and see if I can find a solution. Possibly I need to reinstall Python, using 2.7.3 and 1.5.1 instead. Hi Arjen: My guess is the integrated python+numpy package version you have installed on your computer combines so much from different versions of python and numpy that they are interfering with each other. So I would advise you only install what you need for PLplot and no more which is the latest release in the python-2.7.x series, and the subpackage of the latest release of numpy that is compatible with that. My research indicates those minimal but latest pieces are python-2.7.5.msi which you should download from http://www.python.org/ftp/python/2.7.5/ and numpy-1.7.1-win32-superpack-python2.7.exe which you should download from http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/. (There are later numpy releases, but they are betas and release candidates but nothing final yet.) I am virtually positive those should work if you follow my instructions for the install GUI's below. However, I haven't tried them myself. What I did the other day because I was being lazy was combine python-2.7.3.msi that I downloaded some time ago from http://www.python.org/ftp/python/2.7.3 with numpy-1.5.1-win32-superpack-python2.7.exe I downloaded some time ago from http://sourceforge.net/projects/numpy/files/NumPy/1.5.1/ So that successful build and install was done quite recently, but it was based on old downloads. You can use those old versions as well if you want to exactly replicate the versions I used, but the above latest downloads should work as well, and in any case I am about to shift to those myself since I am now aware of them (due to the research for this e-mail) and later patch versions are almost always better in the Python world. Anyhow, when you try this again, please include all the relevant cmake output results for PLplot that I showed below for my own MinGW/MSYS platform. It is those results that should tell the tale of whether your Python installation and your setup of PATH, CMAKE_INCLUDE_PATH, and CMAKE_LIBRARY_PATH is suitable for a PLplot build. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Arjen M. <Arj...@de...> - 2013-10-18 07:27:12
|
Hi Alan, > -----Original Message----- > From: Alan W. Irwin [mailto:ir...@be...] > ... > Hi Arjen: > > My guess is the integrated python+numpy package version you have installed on > your computer combines so much from different versions of python and numpy that > they are interfering with each other. So I would advise you only install what you need > for PLplot and no more which is the latest release in the python-2.7.x series, and the > subpackage of the latest release of numpy that is compatible with that. > See my follow-up mail. It may be a MSVC/C++ thing - extra checks (I think I understand the error messages now and it is right about them) and an undefined macro that activates a different part of the source code. Regards, Arjen DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail. |
From: Alan W. I. <ir...@be...> - 2013-10-18 08:10:59
|
On 2013-10-18 07:27-0000 Arjen Markus wrote: > Hi Alan, > >> -----Original Message----- >> From: Alan W. Irwin [mailto:ir...@be...] >> > ... > >> Hi Arjen: >> >> My guess is the integrated python+numpy package version you have installed on >> your computer combines so much from different versions of python and numpy that >> they are interfering with each other. So I would advise you only install what you need >> for PLplot and no more which is the latest release in the python-2.7.x series, and the >> subpackage of the latest release of numpy that is compatible with that. >> > > See my follow-up mail. It may be a MSVC/C++ thing - extra checks (I think I understand the > error messages now and it is right about them) and an undefined macro that activates a > different part of the source code. I also had a follow up which crossed with yours. I think we pretty much agree now about what is going on, and only differ in that I would suggest doing the MinGW/MSYS testing first. But doing that after you have completed the MSVC case would also work if you prefer that order. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); the Time Ephemerides project (timeephem.sf.net); PLplot scientific plotting software package (plplot.sf.net); the libLASi project (unifont.org/lasi); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Arjen M. <Arj...@de...> - 2013-10-18 08:14:59
|
Hi Alan, > > I also had a follow up which crossed with yours. I think we pretty much agree now > about what is going on, and only differ in that I would suggest doing the > MinGW/MSYS testing first. But doing that after you have completed the MSVC case > would also work if you prefer that order. > I do not have a great preference for one or the other - I just had a directory "build-windows-python" on my machine and started working from there :). I do think your suggestion is worthwhile - start with MinGW/MSYS as we have had success there before and then look into the MSVC/C++ case again. After all, the immediate problem was identifying where the differences you detected in WINE are coming from. Regards, Arjen DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail. |