#2 ruamel_yaml_clibz-0.3.7 Windows issue

[Guðni Már Gilbert]

Noticed this in a pull request https://github.com/zephyrproject-rtos/zephyr/actions/runs/20660902067/job/59323021422 and issue appeared very suddenly. Noticed that 0.3.7 was released an hour ago, so maybe something went wrong?

Building wheels for collected packages: docopt, ruamel.yaml.clibz
  Building wheel for docopt (pyproject.toml): started
  Building wheel for docopt (pyproject.toml): finished with status 'done'
  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13858 sha256=d0b24760b7ce69cfe17b3d1194e19d23ce3ffac34c39f93d39004d7f6faa6b29
  Stored in directory: c:\users\runneradmin\appdata\local\pip\cache\wheels\1a\bf\a1\4cee4f7678c68c5875ca89eaccf460593539805c3906722228
  Building wheel for ruamel.yaml.clibz (pyproject.toml): started
  Building wheel for ruamel.yaml.clibz (pyproject.toml): finished with status 'error'
  error: subprocess-exited-with-error

  Building wheel for ruamel.yaml.clibz (pyproject.toml) did not run successfully.
  exit code: 1

  [6 lines of output]
  running bdist_wheel
  running build
  running build_ext
  cmd zig build-lib -dynamic -fallow-shlib-undefined -femit-bin=C:\Users\runneradmin\AppData\Local\Temp\pip-install-vo4rlbal\ruamel-yaml-clibz_44c543115deb4f7a8b056ad0713742d4\build\lib.win-amd64-cpython-312\_ruamel_yaml_clibz.cp312-win_amd64.pyd -lc -I C:\hostedtoolcache\windows\Python\3.12.10\x64\include -L C:\hostedtoolcache\windows\Python\3.12.10\x64\libs -L C:\hostedtoolcache\windows\Python\3.12.10\x64 -O ReleaseFast -lpython312 _ruamel_yaml_clibz.c api.c writer.c dumper.c loader.c reader.c scanner.c parser.c emitter.c

  error: [WinError 2] The system cannot find the file specified
  [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for ruamel.yaml.clibz

[Guðni Már Gilbert]

I can reproduce this on Python 3.14.2 on my local machine:

pip install ruamel_yaml_clibz
Collecting ruamel_yaml_clibz
  Using cached ruamel_yaml_clibz-0.3.7.tar.gz (231 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: ruamel_yaml_clibz
  Building wheel for ruamel_yaml_clibz (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for ruamel_yaml_clibz (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      running bdist_wheel
      running build
      running build_ext
      cmd zig build-lib -dynamic -fallow-shlib-undefined -femit-bin=C:\Users\Gudni\AppData\Local\Temp\pip-install-2fccyxex\ruamel-yaml-clibz_c626df41f5134e02bb45a6bd2a70cf4c\build\lib.win-amd64-cpython-314\_ruamel_yaml_clibz.cp314-win_amd64.pyd -lc -I C:\Users\Gudni\AppData\Local\Programs\Python\Python314\include -L C:\Users\Gudni\AppData\Local\Programs\Python\Python314\libs -L C:\Users\Gudni\AppData\Local\Programs\Python\Python314 -O ReleaseFast -lpython314 _ruamel_yaml_clibz.c api.c writer.c dumper.c loader.c reader.c scanner.c parser.c emitter.c

      error: [WinError 2] The system cannot find the file specified
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for ruamel_yaml_clibz
Failed to build ruamel_yaml_clibz
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
╰─> ruamel_yaml_clibz

[Anthon van der Neut]

0.3.7 was released because some people did not follow the README, and did not have setuptools installed.

It will take time to look into this, Windows is not one of my daily drivers, although I did test this and know that others succesfully installed ruamel.yaml.clibz on Windows.

Please use the suggestion in the README ( using --no-deps) if this blocks your work, for now.

[Guðni Már Gilbert]

Sorry for the noise Anthon, I found that installing Zig resolves the issue. I guess the Github pipeline needs to install it when using Windows.

[Anthon van der Neut]

actually ziglang is a build dependency, there should be no need to separately install it to get zig. You can of course do so if you need to proceed, but I rather find out what is going on and get this to work as intended.

[Guðni Már Gilbert]

I'm not familiar with Zig. But for reference I used winget install zig.zig to install it and then the pip install command worked.

[Guðni Már Gilbert]

If there is anything I can do to help debug this, please let me know. :)

[Anthon van der Neut]

Thanks, I first need to try and reproduce this. If I can't I'll get back to you to see what specific setup you have. If I can I'll try'l to solve and get back to see if the solution works for you as well.
I am glad you can proceed for now.

[amalachowski]

The setuptools-zig dependency needs a .exe extension when discovering the zig compiler from the ziglang import. e.g

diff --git a/setuptools_zig.py b/setuptools_zig.py
index 10f4730..af15a10 100644
--- a/setuptools_zig.py
+++ b/setuptools_zig.py
@@ -60,7 +60,7 @@ try:
     import ziglang  # NOQA
     zig_dir = Path(ziglang.__file__).parent
     # setting env didn't work for Unifi alpine, as zig did not have executable bit set
-    zig_path = zig_dir / 'zig'
+    zig_path = zig_dir / 'zig' + ('.exe' if platform.system() == 'Windows' else '')
     zig_path.chmod(0o755)
     debug('zig executable', zig_path)
     debug('zig executable permissions:', oct(zig_path.stat().st_mode))

[Anthon van der Neut]

Thanks, that makes perfect sense.

Originally I just added the directory where I new ziglang was installed to the PATH, and that was most likely what I tested on Windows. And I did not retest after making the changes required for Alpine.

[amalachowski]

Sorry for the delay on testing this. I realized logical error which causes an unsupported operand type(s) for +: 'WindowsPath' and 'str' exception. Moving the extension handling to the zig variable is tested as working:

--- a/setuptools_zig.py
+++ b/setuptools_zip.py
@@ -55,13 +55,13 @@

 debug = Debug()

-zig = 'zig'  # if nothing else just try the command
+# as indicated by amalachowski https://sourceforge.net/p/ruamel-yaml-clibz/tickets/2/#9468
+zig = 'zig' + ('.exe' if platform.system() == 'Windows' else '')  # if nothing else just try the command
 try:
     import ziglang  # NOQA
     zig_dir = Path(ziglang.__file__).parent
     # setting env didn't work for Unifi alpine, as zig did not have executable bit set
-    # as indicated by amalachowski https://sourceforge.net/p/ruamel-yaml-clibz/tickets/2/#9468
-    zig_path = zig_dir / 'zig' + ('.exe' if platform.system() == 'Windows' else '')
+    zig_path = zig_dir / zig
     zig_path.chmod(0o755)
     debug('zig executable', zig_path)
     debug('zig executable permissions:', oct(zig_path.stat().st_mode))

A follow on is that I am realizing compilation errors on Windows amd64.

setuptools_zig.ZigCompilerError: _ruamel_yaml_clibz.c:396:39: error: expression is not an integer constant expression
    enum { __pyx_check_sizeof_voidp = 1 / (int)(SIZEOF_VOID_P == sizeof(void*)) };

Digging around a bit I think there is a misconfiguration when using Zig to compile the C source in this instance.
cpython defines SIZEOF_VOID_P based upon a preprocessor definition, https://github.com/python/cpython/blob/main/PC/pyconfig.h#L352. This is handled correctly for MSVC built extensions internal to pyconfig.h, https://github.com/python/cpython/blob/main/PC/pyconfig.h#L119, but Zig looks to fallback to a mingw toolchain if MSVC isn't available. Adding the MS_WIN64 for this case fixes the compilation errors but I think that logic ends up being dependant upon how Zig's build is run.

[amalachowski]

To follow up on the information above. CPython added support for configuring with MINGW variants in https://github.com/python/cpython/pull/100137 (3.12.0a4). This causes the package to fail to build when targeting older releases (testing above was with cpython3.10.19) without explicitly configuring the preprocessor definition provided.

[Anthon van der Neut]

Thanks for this update, as I was not able to reproduce this (but then I did limited testing (of only Python 3.14 IIRC) on Windows.
It should be relatively easy to conditionally add '-I', 'MS_WIN64' to extra_compile_args in the sertup.py to test this, and then to add it to setuptools-zig. Finding the right Python version for which this is necessary, and installing those verson will be the bigger challenge (for me)