WindowsBuild

Preface to understand the complex workflow

If you build things under Windows, you face some problems

1. Ignoring many edge cases, a library / program under Windows must have all the .dll files that it depends on in the same directory. At the same time, the standard Windows toolchains provide zero support for tracking and copying dependencies (vcpkg has a CMake hack, but that does not help too much here). That means, for every stupid test you need to copy all the dependencies to the test folder before you can run it.
2. Windows DLLs behave slightly differently with respect to exporting symbols than Linux. If, as we do here, we use libraries developed primarily under Linux, we need to directly hack the code a little to get them to work.
3. Quite a few other differences. For example, dlls usually use different naming schemes, files have different line endings etc. All kinds of bugs that we can trigger here. Altogether, the system is also simply less friendly to developing applications, you typically have no package manager to just install boost libraries that work and so on and so forth.

This page is mostly a Howto for me with a tiny bit of explanations; usually you would not build Wavepacket yourself, but download a Windows package instead. In particular, file locations may be slightly incorrect and so on.

Prepare the system

• Install VisualStudio
  • side remark: we only need the BuildTools, but, curiously, its license prohibits OSS development. Visual Studio Community, which is a superset of the build tools, allows OSS development.
  • install the " Desktop Development with C++" workload; not sure which of the packages are needed, but something I would not have thought about.
  • You might want to fine-tune, for example, use the oldest possible Windows SDK for maximum backwards compatibility
• Download the ninja build tool
• Download and unzip boost
• Install the target Python
  • check the switch "install py.exe" or whatever it is called (should be on by default)
• clone the repos for vcpkg, the tensor library and Wavepacket; see the Vagrantfile for the repo locations and commit IDs.

Installing vcpkg dependencies

• Go to the vcpkg directory
• Run "bootstrap-vcpkg.bat"
• Run "vcpkg.exe install lapack blas fftw3 gtest"
• Fix the lapack package
  • not required once https://github.com/microsoft/vcpkg/issues/38009 is resolved
  • in share/lapack-3.11.0/lapack-targets-(debug|release).cmake, change the filenames "liblapack" -> "lapack" (2 occurences in each file)

Building the tensor library

You might have to run the build commands in a "native tools x64 prompt".

• Change the build scripts
  • top-level CMakeLists.txt
    • LAPACK -> lapack
    • LAPACK::LAPACK -> lapack
    • FFTW3 -> FFTW3::fftw3
    • add set_target_properties(tensor PROPERTIES WINDOWS_EXPORT_ALL_SYMBOLS ON)
    • add target_compile_definitions(tensor PRIVATE TENSOR_EXPORT)
  • test/CMakeLists.txt
    • add set_target_properties(testmain PROPERTY WINDOWS_EXPORT_ALL_SYMBOLS ON)
    • add target_compile_definitions(testmain PRIVATE TESTMAIN_EXPORT)
• Modify source code
  • include/tensor/ranges.h
    • replace extern const Range _; by
      #ifdef TENSOR_EXPORT __declspec(dllexport) extern const Range _; #else __declspec(dllimport) extern const Range _; endif
  • include/tensor/flags.h
    • replace extern Flags FLAGS; by same pattern
    • and the TENSOR_DEBUG_BLOCK_SVD as well
  • test/alloc_informer.h
    • replace allocations and deallocations by same pattern (use TESTMAIN_EXPORT instead of TENSOR_EXPORT)
• Run CMake commands similar to Vagrantfile
  • additional flags: -G Ninja -D CMAKE_MAKE_PROGRAM=<path_to_ninja.exe>
• Go to build directory and run ninja -k 0
  • ignore the build errors; the build tries to run the tests, which fails due to missing dependencies
• Check that all tests run through
  • copy all dlls from vcpkg/installed/x64-windows/bin to the "tests/" subdirectory
  • copy tensor.dll from the top-level build folder to the "tests/" subdirectory
  • run ctest in the tests/ subdirectory
• Install everything ninja install
• Modify installed tensorConfig.cmake
  • remove the line find_dependency(lapack)

Building Wavepacket

You might have to run the build commands in a "native tools x64 prompt".

• Install the Python venv; not as in Vagrantfile (Linux-specific)
  • py -3.11 -m venv my_venv
  • my_venv/Scripts/pip install pybind11 matplotlib numpy
• Modify cpp/CMakeLists.txt
  • Replace everything below 2. (Boost) by
    add_library(Boost::headers INTERFACE IMPORTED GLOBAL) target_include_directories(Boost::headers INTERFACE <path_to_unzipped_boost>)
• Run cmake as in Vagrantfile; additional option -D WP_BUILD_DOC=OFF -G Ninja -D CMAKE_MAKE_PROGRAM=<path to ninja exe>
• Run ninja, ninja install
• Try out the tests / demos
  • Copy all dlls to each test subdir: wavepacket.dll, tensor.dll, all vcpkg dlls
    `for demo in $(find Demos -name 'demo.exe'); do d=$(dirname $demo); cp cpp/wavepacket.dll $d; done
  • Also copy all dlls to the cpp subdir
  • In root directory, run tests, ignoring Python tests ctest -R '.*_Test'

Assemble Python package

• open "x64 native tools console", go to ${install_dir}/lib/wavepacket_python/wavepacket
• Check dependencies dumpbin /DEPENDENTS wavepacket_python.pyd
  • python311.dll, vcruntime140.dll should be provided by Python
  • ignore api-ms-win-crt-* (Windows system libs)
  • probably missing -> install recursively: tensor.dll, msvcp140.dll
    • msvcp*.dll found in a directory like C:/Program Files (x86)/Microsoft Visual Studio/2022/(BuildTools|Professional|Community)/Redist/MSVC/.../x64/Microsoft.VC143.CRT
• Copy all licenses to a licenses/ subfolder, rename to "$dependency.$extension"
  • most comes from the vcpkg install folder; note that you should copy everything from "vcpkg-gfortran"
  • tensor lib from the tensor sources
  • EULA for the MSVC library can be taken from https://visualstudio.microsoft.com/en/license-terms/vs2022-cruntime
• add the Readme_PythonPackage.md from the python/ subdirectory
• copy the Demos folder from the checkout: Prune, copy only demo.py and required files, add an output/ directory to all Python demos, so that they can be run in-place

• compress

• Now we can run the Python tests of Wavepacket

  • copy the Python package (wavepacket/ directory) to all demo directories in the source folder, and to the pyhon/test folder in the build directory
  • install FFMpeg and add ffmpeg.exe to the PATH
  • run ctest -R '.*Python.*'