Re: [Plplot-devel] comprehensive testing of MSVC/ifort/nmake

[Arjen M. <Arj...@de...>]

Hi Alan,



> -----Original Message-----
> From: Alan W. Irwin [mailto:ir...@be...]
> Sent: Monday, March 14, 2016 10:23 PM
> To: Arjen Markus
> Cc: PLplot development list
> Subject: comprehensive testing of MSVC/ifort/nmake
>
> On 2016-03-14 07:35-0000 Arjen Markus wrote:
>
> > Alas, I tried that [msvc/ifort comprehensive test] this weekend: somehow CMake
> gets confused. I have stared at the error messages, but I can make heads nor tails
> of the cause.
> >
> > Here is what I did:
> >
> > 1.     I added the directory containing MinGW-w64's sh.exe to the path, so that
> the shell script would be run properly
>
> From your comprehensive_test_env.out file, your PATH (as understood by
> bash.exe) was as follows:
>
> PATH=/d/cmake3.4.3/bin:/usr/bin:/c/Program Files (x86)/Intel/Composer XE
> 2015/bin/intel64:/c/Program Files (x86)/Intel/Composer XE
> 2015/redist/intel64/compiler:/c/Program Files (x86)/Microsoft Visual Studio
> 12.0/Common7/IDE/CommonExtensions/Microsoft/TestWindow:/c/Program Files
> (x86)/MSBuild/12.0/bin/amd64:/c/Program Files
> (x86)/MSBuild/12.0/bin/amd64:/c/Program Files (x86)/Microsoft Visual Studio
> 12.0/VC/BIN/amd64:/c/WINDOWS/Microsoft.NET/Framework64/v4.0.30319:/c/Pro
> gram Files (x86)/Microsoft Visual Studio 12.0/VC/VCPackages:/c/Program Files
> (x86)/Microsoft Visual Studio 12.0/Common7/IDE:/c/Program Files (x86)/Microsoft
> Visual Studio 12.0/Common7/Tools:/c/Program Files (x86)/HTML Help
> Workshop:/c/Program Files (x86)/HTML Help Workshop:/c/Program Files
> (x86)/Microsoft Visual Studio 12.0/Team Tools/Performance Tools/x64:/c/Program
> Files (x86)/Microsoft Visual Studio 12.0/Team Tools/Performance Tools:/c/Program
> Files (x86)/Windows Kits/8.1/bin/x64:/c/Program Files (x86)/Windows
> Kits/8.1/bin/x86:/c/Program Files (x86)/Microsoft SDKs/Windows/v8.1A/bin/NETFX
> 4.5.1 Tools/x64:/c/Program Files (x86)/Intel/Composer XE
> 2015/redist/intel64/mkl:/c/Program Files (x86)/Intel/Composer XE
> 2015/redist/intel64/compiler:/c/ProgramData/Oracle/Java/javapath:/d/CAF/bin:/c/GT
> K/bin:/c/Program Files (x86)/MATLAB/MATLAB Compiler
> Runtime/v82/runtime/win32:/c/Program Files
> (x86)/Skype/Phone:/c/windows/system32:/c/windows:/c/Python27:/c/Python27/DLL
> s:/c/Python27/Scripts:/c/Program Files (x86)/pythonxy/console:/c/Program Files
> (x86)/pythonxy/gettext/bin:/c/MinGW32-xy/bin:/c/Program Files
> (x86)/pythonxy/SciTE-3.3.2-3:/c/Program Files
> (x86)/pythonxy/swig:/c/tcl/bin:/c/windows/system32:/c/Program Files (x86)/MiKTeX
> 2.9/miktex/bin:/c/Program Files (x86)/Intel/Composer XE 2015/redist/intel64/mpirt
>
> That's an extraordinarily complicated PATH.  Is all that really necessary?
>

Quite possibly not, but in Windows these things tend to accumulate without an easy way to reduce it. (Yes, I know, I can go into the "Advanced System Properties" dialogue and the "Environment variables" bit, but the path is then presented as a long long string in a tiny entry control. I do not know which of these directories are really necessary, so that would require some experimenting.)

> From
>
> SH_EXECUTABLE:FILEPATH=C:/msys64/usr/bin/bash.exe
>
> in shared/noninteractive/build_tree/CMakeCache.txt it appears that somehow you
> have indeed found the MinGW-w64/MSYS2 version of bash.exe yet
> C:/msys64/usr/bin is not on your PATH.  For comprehensive testing you will likely
> need other things in C:/msys64/usr/bin so is there some issue with putting that
> PATH component last in your PATH?  (Not first since you want MinGW-
> w64/MSYS2 versions to be the last resort and not the first resort.)
>

Should well be possible - it is just my habit to put such things first.

> Also, I noticed from shared/noninteractive/build_tree/CMakeCache.txt
> you are using D:/cmake3.4.3/bin/cmake.exe.  Can you confirm that is indeed the
> ordinary Windows version of cmake that you should be using for MSVC/ifort/nmake
> that you can download from kitware rather than some MinGW-w64/MSYS2 version?


Yes, that is the version I use for building under MinGW-w64/MSYS2 as well (actually copied the path from the build script I use for that configuration).

>
> >
> > 2.     I invoked the script like this:
> >
> > ../plplot-git/scripts/comprehensive_test.sh --generator_string "NMake Makefiles" -
> -do_clean_as_you_go yes --do_test_interactive no --do_test_traditional_install_tree
> yes --cmake_added_options "-DTCL_INCLUDE_PATH=/usr/include" --
> cmake_added_options "-DENABLE_ada=OFF -DENABLE_octave=OFF" --do_ctest
> no --build_command "nmake" --cmake_added_options "-
> DCMAKE_Fortran_COMPILER=ifort"
> >
> > to make sure the Intel Fortran compiler is selected ("NMake Makefiles" was not
> enough). It took some experimenting ;).
>
> That invocation likely has two showstopper issues.
>
> 1.  The option --cmake_added_options was specified three times with different
> values. The result was (see comprehensive_test.sh.out)
>
> --cmake_added_options "-DCMAKE_Fortran_COMPILER=ifort"
>
> i.e., only the last specified value was honored.
>
> Instead you should use that option only once, e.g.,
>
> --cmake_added_options "-DTCL_INCLUDE_PATH=/usr/include -
> DENABLE_ada=OFF -DENABLE_octave=OFF"
>
> Note, in this example I have dropped -DCMAKE_Fortran_COMPILER=ifort because
> it is problematic, see 2.
>

Oh dear, I completely overlooked that! One of those cases where one needs a fresh pair of eyes.

> 2. That added option -DCMAKE_Fortran_COMPILER=ifort is problematic since it
> does not get propagated properly to the soft landing workaround for Fortran.  So in
> that soft-landing workaround mode a non-intel compiler was checked (and was
> working).  Because of that success it then tried the Intel compiler for real which had
> a crash landing (cmake errored out).  To avoid this inconsistency issue between
> soft-landing mode and real mode I suggest you use the well-debugged environment
> variable method of specifying the compiler, i.e.,
>
> export FC=ifort
>
> which will test that compiler in soft-landing mode and only try it for real if the soft-
> landing mode has a success.  And if the soft-landing mode fails, then Fortran will be
> dropped from the real mode and at least the non-fortran part of the comprehensive
> test will continue.
>

Okay, that sounds like a better plan.

> [....]
> > I am particularly puzzled by the message from the linker
> that it knows no option "s" and what is more the Intel Fortran compiler is invoked
> with an option "-o name-of-executable", which is not the proper option to set the
> name of the executable (it should be -exe:name-of-executable).
>
> These showstopping ifort errors are indeed a puzzle since you have been having
> great success recently with configuring and testing PLplot by hand for the
> msvc/ifort/nmake platform.  So if that includes using -
> DCMAKE_Fortran_COMPILER=ifort rather than export FC=ifort, that is already
> proof that cmake should work fine with -DCMAKE_Fortran_COMPILER=ifort. So
> the fundamental question is how is the comprehensive test script invoking cmake
> differently than you do by hand?
>
> So I would look carefully at comprehensive_test_env.out to see if your PATH
> variable is way too complex compared to when you invoke cmake by hand
> (although MinGW-w64/MSYS2 should be on your PATH as a last resort, see
> above), and also look carefully at comprehensive_test.sh.out for the cmake options
> actually used (such as the --cmake_added_options issue I noted above) to see if
> they are different than what cmake options you use by hand.
>
> Also, you should try the usual trick to debug bash scripts, i.e., run it like this:
>
> export FC=ifort
> bash.exe -x ../plplot-git/scripts/comprehensive_test.sh --generator_string "NMake
> Makefiles" --do_clean_as_you_go yes --do_test_interactive no --
> do_test_traditional_install_tree yes --do_ctest no --build_command "nmake" --
> cmake_added_options "-DTCL_INCLUDE_PATH=/usr/include -
> DENABLE_ada=OFF -DENABLE_octave=OFF "
>
> which should print out a lot of intermediate values used in the script such as the
> complete invocation of the cmake command.  For example, when I try that bash
> trick here, I get the following result (buried deep in a lot of others)
>
> + cmake
> -
> DCMAKE_INSTALL_PREFIX=/home/software/plplot/HEAD/plplot.git/../comprehens
> ive_test_disposeable/shared/noninteractive/install_tree
> -DBUILD_TEST=ON -DDEFAULT_NO_DEVICES=ON -DPLD_ps=ON -
> DDEFAULT_NO_BINDINGS=ON -DENABLE_f95=ON -DENABLE_tcl=ON -
> DBUILD_SHARED_LIBS=ON -G 'Unix Makefiles'
> /home/software/plplot/HEAD/plplot.git
>
> So this allows me to see exactly how cmake was invoked and absolutely confirms
> the script is getting it right in my case.  So if you do the equivalent there you should
> be able to quickly discover the error in how cmake is invoked by the script in your
> case.
>
> Note, also if ifort continues to be problematic with script results but not by hand
> even though you have proved by the above bash trick that the cmake invocations
> are identical for the two cases, then try the "by hand" result with exactly the same
> PATH as indicated in comprehensive_test_env.out just to check that putting
> MinGW-w64/MSYS2 bash.exe and sh.exe last on your PATH is not somehow
> interfering with CMake's tests of ifort.  If that possible interference turns out to be
> an issue for the "by hand" case, then you try putting Cygwin's bash.exe and sh.exe
> last on your PATH instead of the MinGW-w64/MSYS2 to see if there is also
> interference in that case.
>
> Let me know how it goes, and we can take it from there.
>
Okay, will do.

Regards,

Arjen



DISCLAIMER: This message is intended exclusively for the addressee(s) and may contain confidential and privileged information. If you are not the intended recipient please notify the sender immediately and destroy this message. Unauthorized use, disclosure or copying of this message is strictly prohibited. The foundation 'Stichting Deltares', which has its seat at Delft, The Netherlands, Commercial Registration Number 41146461, is not liable in any way whatsoever for consequences and/or damages resulting from the improper, incomplete and untimely dispatch, receipt and/or content of this e-mail.