Recent changes to WindowsBuild

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Thu, 02 May 2024 21:58:34 -0000

--- v24
+++ v25
@@ -79,8 +79,6 @@
     * Replace everything below 2. (Boost) by
         `add_library(Boost::headers INTERFACE IMPORTED GLOBAL)
         target_include_directories(Boost::headers INTERFACE <path_to_unzipped_boost>)`
-* Modify cpp/CMakeLists.txt, python/CMakeLists.txt
-    `set_target_properties(wavepacket (or wavepacket_python) PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS ON)`
 * Run cmake as in Vagrantfile; additional option `-D WP_BUILD_DOC=OFF -G Ninja -D CMAKE_MAKE_PROGRAM=<path to="" ninja="" exe="">`
 * Run `ninja`, `ninja install`
 * Try out the tests / demos
@@ -91,7 +89,6 @@

 ###Assemble Python package###

-* Install wavepacket `ninja install`
 * open "x64 native tools console", go to ${install_dir}/lib/wavepacket_python/wavepacket
 * Check dependencies `dumpbin /DEPENDENTS wavepacket_python.pyd`
     * python311.dll, vcruntime140.dll should be provided by Python
@@ -101,7 +98,7 @@
 * Copy all licenses to a licenses/ subfolder, rename to "$dependency.$extension"
     * most comes from the vcpkg install folder; note that you should copy everything from "vcpkg-gfortran"
     * tensor lib from the tensor sources
-    * EULA for the MSVC library can be taken from https://visualstudio.microsoft.com/en/license-terms/vs2022-runtime
+    * EULA for the MSVC library can be taken from https://visualstudio.microsoft.com/en/license-terms/vs2022-cruntime
 * add the Readme_PythonPackage.md from the python/ subdirectory
 * copy the Demos folder from the checkout: Prune, copy only demo.py and required files, add an output/ directory to all Python demos, so that they can be run in-place
 * compress
</path></path_to_unzipped_boost>

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Thu, 02 May 2024 21:53:41 -0000

--- v23
+++ v24
@@ -53,6 +53,7 @@
                endif`
     * include/tensor/flags.h
         * replace `extern Flags FLAGS;` by same pattern
+        * and the TENSOR_DEBUG_BLOCK_SVD as well
     * test/alloc_informer.h
         * replace allocations and deallocations by same pattern (use TESTMAIN_EXPORT instead of TENSOR_EXPORT)
 * Run CMake commands similar to Vagrantfile
@@ -68,6 +69,8 @@
     * remove the line `find_dependency(lapack)`

 ###Building Wavepacket###
+
+You might have to run the build commands in a "native tools x64 prompt".

 * Install the Python venv; not as in Vagrantfile (Linux-specific)
     * `py -3.11 -m venv my_venv`

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Thu, 02 May 2024 21:52:03 -0000

--- v22
+++ v23
@@ -39,7 +39,7 @@
         * LAPACK::LAPACK -> lapack
         * FFTW3 -> FFTW3::fftw3
         * add `set_target_properties(tensor PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS ON)`
-        * add `target_compile_definitions(tensor PRIVAE TENSOR_EXPORT)`
+        * add `target_compile_definitions(tensor PRIVATE TENSOR_EXPORT)`
     * test/CMakeLists.txt
         * add `set_target_properties(testmain PROPERTY WINDOWS_EXPORT_ALL_SYMBOLS ON)`
         * add `target_compile_definitions(testmain PRIVATE TESTMAIN_EXPORT)`
@@ -56,7 +56,7 @@
     * test/alloc_informer.h
         * replace allocations and deallocations by same pattern (use TESTMAIN_EXPORT instead of TENSOR_EXPORT)
 * Run CMake commands similar to Vagrantfile
-    * additional flags: `-G Ninja -D CMAKE_BUILD_TOOL=<path_to_ninja.exe>`
+    * additional flags: `-G Ninja -D CMAKE_MAKE_PROGRAM=<path_to_ninja.exe>`
 * Go to build directory and run `ninja -k 0`
     * ignore the build errors; the build tries to run the tests, which fails due to missing dependencies
 * Check that all tests run through
@@ -78,7 +78,7 @@
         target_include_directories(Boost::headers INTERFACE <path_to_unzipped_boost>)`
 * Modify cpp/CMakeLists.txt, python/CMakeLists.txt
     `set_target_properties(wavepacket (or wavepacket_python) PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS ON)`
-* Run cmake as in Vagrantfile; additional option "-D WP_BUILD_DOC=OFF -G Ninja"
+* Run cmake as in Vagrantfile; additional option `-D WP_BUILD_DOC=OFF -G Ninja -D CMAKE_MAKE_PROGRAM=<path to="" ninja="" exe="">`
 * Run `ninja`, `ninja install`
 * Try out the tests / demos
     * Copy all dlls to each test subdir: wavepacket.dll, tensor.dll, all vcpkg dlls
</path></path_to_unzipped_boost></path_to_ninja.exe></path_to_ninja.exe>

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Mon, 22 Apr 2024 21:32:37 -0000

--- v21
+++ v22
@@ -99,7 +99,8 @@
     * most comes from the vcpkg install folder; note that you should copy everything from "vcpkg-gfortran"
     * tensor lib from the tensor sources
     * EULA for the MSVC library can be taken from https://visualstudio.microsoft.com/en/license-terms/vs2022-runtime
-* add the readme(s)
+* add the Readme_PythonPackage.md from the python/ subdirectory
+* copy the Demos folder from the checkout: Prune, copy only demo.py and required files, add an output/ directory to all Python demos, so that they can be run in-place
 * compress

 * Now we can run the Python tests of Wavepacket

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Fri, 19 Apr 2024 20:25:33 -0000

--- v20
+++ v21
@@ -95,3 +95,14 @@
     * ignore api-ms-win-crt-* (Windows system libs)
     * probably missing -> install recursively: tensor.dll, msvcp140.dll
         * msvcp*.dll found in a directory like C:/Program Files (x86)/Microsoft Visual Studio/2022/(BuildTools|Professional|Community)/Redist/MSVC/.../x64/Microsoft.VC143.CRT
+* Copy all licenses to a licenses/ subfolder, rename to "$dependency.$extension"
+    * most comes from the vcpkg install folder; note that you should copy everything from "vcpkg-gfortran"
+    * tensor lib from the tensor sources
+    * EULA for the MSVC library can be taken from https://visualstudio.microsoft.com/en/license-terms/vs2022-runtime
+* add the readme(s)
+* compress
+
+* Now we can run the Python tests of Wavepacket
+    * copy the Python package (wavepacket/ directory) to all demo directories in the source folder, and to the pyhon/test folder in the build directory
+    * install FFMpeg and add ffmpeg.exe to the PATH
+    * run `ctest -R '.*Python.*'`

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Tue, 09 Apr 2024 20:00:52 -0000

--- v19
+++ v20
@@ -76,6 +76,8 @@
     * Replace everything below 2. (Boost) by
         `add_library(Boost::headers INTERFACE IMPORTED GLOBAL)
         target_include_directories(Boost::headers INTERFACE <path_to_unzipped_boost>)`
+* Modify cpp/CMakeLists.txt, python/CMakeLists.txt
+    `set_target_properties(wavepacket (or wavepacket_python) PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS ON)`
 * Run cmake as in Vagrantfile; additional option "-D WP_BUILD_DOC=OFF -G Ninja"
 * Run `ninja`, `ninja install`
 * Try out the tests / demos
</path_to_unzipped_boost>

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Tue, 09 Apr 2024 19:17:55 -0000

--- v18
+++ v19
@@ -3,7 +3,7 @@
 If you build things under Windows, you face some problems

 1. Ignoring many edge cases, a library / program under Windows must have all the .dll files that it depends on in the same directory. At the same time, the standard Windows toolchains provide zero support for tracking and copying dependencies (vcpkg has a CMake hack, but that does not help too much here). That means, for every stupid test you need to copy all the dependencies to the test folder before you can run it.
-2. Windows DLLs behave slightly differently with respect to exporting symbols than Linux. If, as we do here, we use libraries developed primarily under Linux, we need to directly hack the code a little to get them to work. And the CMake scripts have little support for the stupid workflows needed because of point 1.
+2. Windows DLLs behave slightly differently with respect to exporting symbols than Linux. If, as we do here, we use libraries developed primarily under Linux, we need to directly hack the code a little to get them to work.
 3. Quite a few other differences. For example, dlls usually use different naming schemes, files have different line endings etc. All kinds of bugs that we can trigger here. Altogether, the system is also simply less friendly to developing applications, you typically have no package manager to just install boost libraries that work and so on and so forth.

 This page is mostly a Howto for me with a tiny bit of explanations; usually you would not build Wavepacket yourself, but download a Windows package instead. In particular, file locations may be slightly incorrect and so on.
@@ -66,3 +66,30 @@
 * Install everything `ninja install`
 * Modify installed tensorConfig.cmake
     * remove the line `find_dependency(lapack)`
+
+###Building Wavepacket###
+
+* Install the Python venv; not as in Vagrantfile (Linux-specific)
+    * `py -3.11 -m venv my_venv`
+    * `my_venv/Scripts/pip install pybind11 matplotlib numpy`
+* Modify cpp/CMakeLists.txt
+    * Replace everything below 2. (Boost) by
+        `add_library(Boost::headers INTERFACE IMPORTED GLOBAL)
+        target_include_directories(Boost::headers INTERFACE <path_to_unzipped_boost>)`
+* Run cmake as in Vagrantfile; additional option "-D WP_BUILD_DOC=OFF -G Ninja"
+* Run `ninja`, `ninja install`
+* Try out the tests / demos
+    * Copy all dlls to each test subdir: wavepacket.dll, tensor.dll, all vcpkg dlls
+        `for demo in $(find Demos -name 'demo.exe'); do d=$(dirname $demo); cp cpp/wavepacket.dll $d; done
+    * Also copy all dlls to the cpp subdir
+    * In root directory, run tests, ignoring Python tests `ctest -R '.*_Test'`
+
+###Assemble Python package###
+
+* Install wavepacket `ninja install`
+* open "x64 native tools console", go to ${install_dir}/lib/wavepacket_python/wavepacket
+* Check dependencies `dumpbin /DEPENDENTS wavepacket_python.pyd`
+    * python311.dll, vcruntime140.dll should be provided by Python
+    * ignore api-ms-win-crt-* (Windows system libs)
+    * probably missing -> install recursively: tensor.dll, msvcp140.dll
+        * msvcp*.dll found in a directory like C:/Program Files (x86)/Microsoft Visual Studio/2022/(BuildTools|Professional|Community)/Redist/MSVC/.../x64/Microsoft.VC143.CRT
</path_to_unzipped_boost>

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Sun, 07 Apr 2024 20:38:02 -0000

--- v17
+++ v18
@@ -16,6 +16,8 @@
     * You might want to fine-tune, for example, use the oldest possible Windows SDK for maximum backwards compatibility
 * Download the ninja build tool
 * Download and unzip boost
+* Install the target Python
+    * check the switch "install py.exe" or whatever it is called (should be on by default)
 * clone the repos for vcpkg, the tensor library and Wavepacket; see the Vagrantfile for the repo locations and commit IDs.

 ###Installing vcpkg dependencies###

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Sun, 07 Apr 2024 20:36:18 -0000

--- v16
+++ v17
@@ -1,4 +1,4 @@
-=====Preface to understand the complex workflow=====
+###Preface to understand the complex workflow###

 If you build things under Windows, you face some problems

@@ -8,7 +8,7 @@

 This page is mostly a Howto for me with a tiny bit of explanations; usually you would not build Wavepacket yourself, but download a Windows package instead. In particular, file locations may be slightly incorrect and so on.

-=====Prepare the system=====
+###Prepare the system###

 * Install VisualStudio
     * side remark: we only need the BuildTools, but, curiously, its license prohibits OSS development. Visual Studio Community, which is a superset of the build tools, allows OSS development.
@@ -18,7 +18,7 @@
 * Download and unzip boost
 * clone the repos for vcpkg, the tensor library and Wavepacket; see the Vagrantfile for the repo locations and commit IDs.

-=====Installing vcpkg dependencies=====
+###Installing vcpkg dependencies###

 * Go to the vcpkg directory
 * Run "bootstrap-vcpkg.bat"
@@ -26,3 +26,41 @@
 * Fix the lapack package
     * not required once https://github.com/microsoft/vcpkg/issues/38009 is resolved
     * in share/lapack-3.11.0/lapack-targets-(debug|release).cmake, change the filenames "liblapack" -> "lapack" (2 occurences in each file)
+
+### Building the tensor library###
+
+You might have to run the build commands in a "native tools x64 prompt".
+
+* Change the build scripts
+    * top-level CMakeLists.txt
+        * LAPACK -> lapack
+        * LAPACK::LAPACK -> lapack
+        * FFTW3 -> FFTW3::fftw3
+        * add `set_target_properties(tensor PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS ON)`
+        * add `target_compile_definitions(tensor PRIVAE TENSOR_EXPORT)`
+    * test/CMakeLists.txt
+        * add `set_target_properties(testmain PROPERTY WINDOWS_EXPORT_ALL_SYMBOLS ON)`
+        * add `target_compile_definitions(testmain PRIVATE TESTMAIN_EXPORT)`
+* Modify source code
+    * include/tensor/ranges.h
+        * replace `extern const Range _;` by
+            `#ifdef TENSOR_EXPORT
+               __declspec(dllexport) extern const Range _;
+               #else
+               __declspec(dllimport) extern const Range _;
+               endif`
+    * include/tensor/flags.h
+        * replace `extern Flags FLAGS;` by same pattern
+    * test/alloc_informer.h
+        * replace allocations and deallocations by same pattern (use TESTMAIN_EXPORT instead of TENSOR_EXPORT)
+* Run CMake commands similar to Vagrantfile
+    * additional flags: `-G Ninja -D CMAKE_BUILD_TOOL=<path_to_ninja.exe>`
+* Go to build directory and run `ninja -k 0`
+    * ignore the build errors; the build tries to run the tests, which fails due to missing dependencies
+* Check that all tests run through
+    * copy all dlls from vcpkg/installed/x64-windows/bin to the "tests/" subdirectory
+    * copy tensor.dll from the top-level build folder to the "tests/" subdirectory
+    * run `ctest` in the tests/ subdirectory
+* Install everything `ninja install`
+* Modify installed tensorConfig.cmake
+    * remove the line `find_dependency(lapack)`
</path_to_ninja.exe>

WindowsBuild modified by Ulf Lorenz

Ulf Lorenz — Sun, 07 Apr 2024 20:22:28 -0000

--- v15
+++ v16
@@ -4,3 +4,25 @@

 1. Ignoring many edge cases, a library / program under Windows must have all the .dll files that it depends on in the same directory. At the same time, the standard Windows toolchains provide zero support for tracking and copying dependencies (vcpkg has a CMake hack, but that does not help too much here). That means, for every stupid test you need to copy all the dependencies to the test folder before you can run it.
 2. Windows DLLs behave slightly differently with respect to exporting symbols than Linux. If, as we do here, we use libraries developed primarily under Linux, we need to directly hack the code a little to get them to work. And the CMake scripts have little support for the stupid workflows needed because of point 1.
+3. Quite a few other differences. For example, dlls usually use different naming schemes, files have different line endings etc. All kinds of bugs that we can trigger here. Altogether, the system is also simply less friendly to developing applications, you typically have no package manager to just install boost libraries that work and so on and so forth.
+
+This page is mostly a Howto for me with a tiny bit of explanations; usually you would not build Wavepacket yourself, but download a Windows package instead. In particular, file locations may be slightly incorrect and so on.
+
+=====Prepare the system=====
+
+* Install VisualStudio
+    * side remark: we only need the BuildTools, but, curiously, its license prohibits OSS development. Visual Studio Community, which is a superset of the build tools, allows OSS development.
+    * install the " Desktop Development with C++"  workload; not sure which of the packages are needed, but something I would not have thought about.
+    * You might want to fine-tune, for example, use the oldest possible Windows SDK for maximum backwards compatibility
+* Download the ninja build tool
+* Download and unzip boost
+* clone the repos for vcpkg, the tensor library and Wavepacket; see the Vagrantfile for the repo locations and commit IDs.
+
+=====Installing vcpkg dependencies=====
+
+* Go to the vcpkg directory
+* Run "bootstrap-vcpkg.bat"
+* Run "vcpkg.exe install lapack blas fftw3 gtest"
+* Fix the lapack package
+    * not required once https://github.com/microsoft/vcpkg/issues/38009 is resolved
+    * in share/lapack-3.11.0/lapack-targets-(debug|release).cmake, change the filenames "liblapack" -> "lapack" (2 occurences in each file)