#20 ruamel.yaml.clib-0.2.7 macos universal is not really universal

[Sorin Sbarnea]

I think that the universal build is broken as it does only include the intern binaries inside, take a look at what happens on macos with arm64 architecture:
`
$ pip install ruamel-yaml-clib
Collecting ruamel-yaml-clib
Using cached ruamel.yaml.clib-0.2.7-cp311-cp311-macosx_10_9_universal2.whl (139 kB)
Installing collected packages: ruamel-yaml-clib
Successfully installed ruamel-yaml-clib-0.2.7

import ruamel.yaml.cyaml
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/ssbarnea/.pyenv/versions/3.11.3/lib/python3.11/site-packages/ruamel/yaml/cyaml.py", line 3, in <module>
from _ruamel_yaml import CParser, CEmitter # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ImportError: dlopen(/Users/ssbarnea/.pyenv/versions/3.11.3/lib/python3.11/site-packages/_ruamel_yaml.cpython-311-darwin.so, 0x0002): tried: '/Users/ssbarnea/.pyenv/versions/3.11.3/lib/python3.11/site-packages/_ruamel_yaml.cpython-311-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/ssbarnea/.pyenv/versions/3.11.3/lib/python3.11/site-packages/_ruamel_yaml.cpython-311-darwin.so' (no such file), '/Users/ssbarnea/.pyenv/versions/3.11.3/lib/python3.11/site-packages/_ruamel_yaml.cpython-311-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))
`</module></module></stdin>

I should mention that when trying to installing on py310, pip did pick the arm64 wheel instead of universal one and that one worked. I am not sure why on 3.11 is picking the universal one but for sure that universal build is not universal, so broken.

As a first measure before trying to fix the broken universal build it would be a good idea to remove or yank these from pypi, so pip would not pip them.

I tried to manually install the arm64 wheel on python 3.11 but pip would report it as not being compatible with current platform:

$ pip install ruamel.yaml.clib-0.2.7-cp311-cp311-macosx_12_6_arm64.whl
ERROR: ruamel.yaml.clib-0.2.7-cp311-cp311-macosx_12_6_arm64.whl is not a supported wheel on this platform

Time to debug it:
$ python -m pip debug --verbose|grep py311
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
py311-none-macosx_13_0_arm64
py311-none-macosx_13_0_universal2
py311-none-macosx_12_0_arm64
py311-none-macosx_12_0_universal2
py311-none-macosx_11_0_arm64
py311-none-macosx_11_0_universal2
py311-none-macosx_10_16_universal2
py311-none-macosx_10_15_universal2
py311-none-macosx_10_14_universal2
py311-none-macosx_10_13_universal2
py311-none-macosx_10_12_universal2
py311-none-macosx_10_11_universal2
py311-none-macosx_10_10_universal2
py311-none-macosx_10_9_universal2
py311-none-macosx_10_8_universal2
py311-none-macosx_10_7_universal2
py311-none-macosx_10_6_universal2
py311-none-macosx_10_5_universal2
py311-none-macosx_10_4_universal2
py311-none-any

So basically pip fails to find it because we have a 12_6 in the name instead of 12_0 or even 13_0, this being the root cause for the failure to install the wheel.

$ pip --version
pip 23.1.2 from /Users/ssbarnea/.pyenv/versions/3.11.3/lib/python3.11/site-packages/pip (python 3.11)
^ as of today, that is the latest pip, and I kinda doubt is a bug that it does not find it. IMHO, starting with 12 series, there is no sub-versioning. so only 12_0 and 13_0 would be valid names.

I will also contact packaging folks for confirmation about this. Still, the universal being broken is clearly a bug and likely the second one too.

As a side note, most of the multi-platform compiled python libraries I know already started using https://cibuildwheel.readthedocs.io for building as that makes the process much easier and less linkely to produce broken/incomplete artifacts.

[Anthon van der Neut]

I am not going to change to cibuildwheel

[Sorin Sbarnea]

Please read https://github.com/pypa/packaging/issues/435 to understand that the current build process is broken. You are not expected to produce anything else than 11_0, 12_0 or 13_0 for macos. Only 10_x has minor versions numbers due to change in ABI.

The alternative build system was only an idea, but the bug is still present. While I do not expect to see it fixed in already made release, I hope that the issue would not happen with next build.

I would offer to help with packaging and testing but I have no idea how you are producing these builds. I just wanted to mention that, in case help would be welcomed, especially as with open-source stuff, I am used with provide-your-patch approach ;)