#11 Build fails on Alpine docker image 2.7

[Anthon van der Neut]

https://gitlab.com/jayvdb/mobans/-/jobs/278684791

It is using latest setuptools, but being Alpine it does not have gcc by default.

DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Ignoring moban: markers 'python_version > "3.0"' don't match your environment
Collecting moban
  Downloading https://files.pythonhosted.org/packages/78/8b/451d95478fd9e9e5a2b4fc27e7d3777f6f6a0da306b4d153e78aa7d7ed07/moban-0.5.0-py2.py3-none-any.whl
Collecting ruamel.yaml>=0.15.5; python_version != "3.4" and python_version < "3.7" (from moban)
  Downloading https://files.pythonhosted.org/packages/fa/90/ecff85a2e9c497e2fa7142496e10233556b5137db5bd46f3f3b006935ca8/ruamel.yaml-0.16.5-py2.py3-none-any.whl (123kB)
Collecting GitPython>=2.0.0 (from moban)
  Downloading https://files.pythonhosted.org/packages/2e/93/c26103b14766b83d6c2f7d2a29aa8540058c2970f2c80b13c25720764b70/GitPython-2.1.14-py2.py3-none-any.whl (452kB)
Collecting git-url-parse>=1.2.2 (from moban)
  Downloading https://files.pythonhosted.org/packages/7e/fd/755d2abf15615c575c53d07a6cd65a66e4f9e0c07466e1598f7f74763af9/git_url_parse-1.2.2-py2-none-any.whl
Collecting crayons>=0.1.0 (from moban)
  Downloading https://files.pythonhosted.org/packages/b6/ee/1a4f411c66f2788baf82a91b6f726473d9f4075ec7b86c860ba15294ce90/crayons-0.2.0-py2.py3-none-any.whl
Collecting jinja2>=2.7.1 (from moban)
  Downloading https://files.pythonhosted.org/packages/1d/e7/fd8b501e7a6dfe492a433deb7b9d833d39ca74916fa8bc63dd1a4947a671/Jinja2-2.10.1-py2.py3-none-any.whl (124kB)
Collecting lml>=0.0.9 (from moban)
  Downloading https://files.pythonhosted.org/packages/90/e5/eaa5a1c8a9adbac956daa6027789ff13e30362a03cc0d010d88bd867c73f/lml-0.0.9-py2.py3-none-any.whl
Collecting appdirs>=1.2.0 (from moban)
  Downloading https://files.pythonhosted.org/packages/56/eb/810e700ed1349edde4cbdc1b2a21e28cdf115f9faf263f6bbf8447c1abf3/appdirs-1.4.3-py2.py3-none-any.whl
Collecting ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" (from ruamel.yaml>=0.15.5; python_version != "3.4" and python_version < "3.7"->moban)
  Downloading https://files.pythonhosted.org/packages/6a/6c/7b461053ce5be0d7c8b12dcae9a7c10e8012238a00f6fcd98643ee66d2de/ruamel.yaml.clib-0.1.2.tar.gz (177kB)
Collecting ruamel.ordereddict; platform_python_implementation == "CPython" and python_version <= "2.7" (from ruamel.yaml>=0.15.5; python_version != "3.4" and python_version < "3.7"->moban)
  Downloading https://files.pythonhosted.org/packages/7d/fd/522c4722347aaf01e6bc8cce486130d83c52c72d3c3c2bbe3cbb02dddd4d/ruamel.ordereddict-0.4.14.tar.gz (60kB)
Collecting gitdb2>=2.0.0 (from GitPython>=2.0.0->moban)
  Downloading https://files.pythonhosted.org/packages/da/30/a407568aa8d8f25db817cf50121a958722f3fc5f87e3a6fba1f40c0633e3/gitdb2-2.0.5-py2.py3-none-any.whl (62kB)
Collecting pbr (from git-url-parse>=1.2.2->moban)
  Downloading https://files.pythonhosted.org/packages/f9/d8/bd657bfa0e89eb71ad5e977ed99a9bb2b44e5db68d9190970637c26501bb/pbr-5.4.2-py2.py3-none-any.whl (110kB)
Collecting colorama<0.4 (from -c test-requirements.txt (line 5))
  Downloading https://files.pythonhosted.org/packages/db/c8/7dcf9dbcb22429512708fe3a547f8b6101c0d02137acbd892505aee57adf/colorama-0.3.9-py2.py3-none-any.whl
Collecting MarkupSafe>=0.23 (from jinja2>=2.7.1->moban)
  Downloading https://files.pythonhosted.org/packages/b9/2e/64db92e53b86efccfaea71321f597fa2e1b2bd3853d8ce658568f7a13094/MarkupSafe-1.1.1.tar.gz
Collecting smmap2>=2.0.0 (from gitdb2>=2.0.0->GitPython>=2.0.0->moban)
  Downloading https://files.pythonhosted.org/packages/55/d2/866d45e3a121ee15a1dc013824d58072fd5c7799c9c34d01378eb262ca8f/smmap2-2.0.5-py2.py3-none-any.whl
Building wheels for collected packages: ruamel.yaml.clib, ruamel.ordereddict, MarkupSafe
  Building wheel for ruamel.yaml.clib (setup.py): started
  Building wheel for ruamel.yaml.clib (setup.py): finished with status 'done'
  Created wheel for ruamel.yaml.clib: filename=ruamel.yaml.clib-0.1.2-cp27-none-any.whl size=2897 sha256=fd5d7c496324cbfbf1f11592834fbce31d57fd4f3116d292f16599a91409fa0b
  Stored in directory: /builds/jayvdb/mobans/.cache/pip/wheels/28/88/2a/70c4579e647724d98a52b856110558d37e66054af6de5f1795
  Building wheel for ruamel.ordereddict (setup.py): started
  Building wheel for ruamel.ordereddict (setup.py): finished with status 'error'
  ERROR: Command errored out with exit status 1:
   command: /usr/local/bin/python2 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-KPWdVW/ruamel.ordereddict/setup.py'"'"'; __file__='"'"'/tmp/pip-install-KPWdVW/ruamel.ordereddict/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-6uRWHA --python-tag cp27
       cwd: /tmp/pip-install-KPWdVW/ruamel.ordereddict/
  Complete output (14 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-2.7
  creating build/lib.linux-x86_64-2.7/ruamel
  creating build/lib.linux-x86_64-2.7/ruamel/ordereddict
  copying ./__init__.py -> build/lib.linux-x86_64-2.7/ruamel/ordereddict
  running build_ext
  building '_ordereddict' extension
  creating build/temp.linux-x86_64-2.7
  gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/local/include/python2.7 -c ordereddict.c -o build/temp.linux-x86_64-2.7/ordereddict.o
  unable to execute 'gcc': No such file or directory
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for ruamel.ordereddict
  Running setup.py clean for ruamel.ordereddict
  Building wheel for MarkupSafe (setup.py): started
  Building wheel for MarkupSafe (setup.py): finished with status 'done'
  Created wheel for MarkupSafe: filename=MarkupSafe-1.1.1-cp27-none-any.whl size=12631 sha256=1d2b205121ebd054347f0baa060d135c4cf7810298a7217c8e25b4ab79a00ea3
  Stored in directory: /builds/jayvdb/mobans/.cache/pip/wheels/f2/aa/04/0edf07a1b8a5f5f1aed7580fffb69ce8972edc16a505916a77
Successfully built ruamel.yaml.clib MarkupSafe
Failed to build ruamel.ordereddict
Installing collected packages: colorama, ruamel.yaml.clib, ruamel.ordereddict, ruamel.yaml, smmap2, gitdb2, GitPython, pbr, git-url-parse, crayons, MarkupSafe, jinja2, lml, appdirs, moban
  Running setup.py install for ruamel.ordereddict: started
    Running setup.py install for ruamel.ordereddict: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python2 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-KPWdVW/ruamel.ordereddict/setup.py'"'"'; __file__='"'"'/tmp/pip-install-KPWdVW/ruamel.ordereddict/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-VGjiAR/install-record.txt --single-version-externally-managed --compile
         cwd: /tmp/pip-install-KPWdVW/ruamel.ordereddict/
    Complete output (14 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-2.7
    creating build/lib.linux-x86_64-2.7/ruamel
    creating build/lib.linux-x86_64-2.7/ruamel/ordereddict
    copying ./__init__.py -> build/lib.linux-x86_64-2.7/ruamel/ordereddict
    running build_ext
    building '_ordereddict' extension
    creating build/temp.linux-x86_64-2.7
    gcc -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -DTHREAD_STACK_SIZE=0x100000 -fPIC -I/usr/local/include/python2.7 -c ordereddict.c -o build/temp.linux-x86_64-2.7/ordereddict.o
    unable to execute 'gcc': No such file or directory
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python2 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-KPWdVW/ruamel.ordereddict/setup.py'"'"'; __file__='"'"'/tmp/pip-install-KPWdVW/ruamel.ordereddict/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-VGjiAR/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.

I can play around with this, if the solution is not immediately obvious. I suspect you may have already solved this for yaml.clib, since that builds OK.

(originally posted on 2019-08-23 at 10:39:01 by John Vandenberg <John Vandenberg@bitbucket>)

[Anthon van der Neut]

I added verbose for the clib build, and it is also failing, but the build failure is hidden

Added ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" from https://files.pythonhosted.org/packages/6a/6c/7b461053ce5be0d7c8b12dcae9a7c10e8012238a00f6fcd98643ee66d2de/ruamel.yaml.clib-0.1.2.tar.gz#sha256=cee86ecc893a6a8ecaa7c6a9c2d06f75f614176210d78a5f155f8e78d6989509 (from ruamel.yaml>=0.15.5; python_version != "3.4" and python_version < "3.7"->moban) to build tracker '/tmp/pip-req-tracker-gCDy8O'
    Running setup.py (path:/tmp/pip-install-eO45HU/ruamel.yaml.clib/setup.py) egg_info for package ruamel.yaml.clib
    Running command python setup.py egg_info
    sys.argv ['/tmp/pip-install-eO45HU/ruamel.yaml.clib/setup.py', 'egg_info', '--egg-base', 'pip-egg-info']
    unable to execute 'gcc': No such file or directory
    test compiling test_ruamel_yaml
    compile error: /tmp/tmp_ruamel_r1SqTn/test_ruamel_yaml.c
    running egg_info
    creating pip-egg-info/ruamel.yaml.clib.egg-info
    writing pip-egg-info/ruamel.yaml.clib.egg-info/PKG-INFO
    writing namespace_packages to pip-egg-info/ruamel.yaml.clib.egg-info/namespace_packages.txt
    writing top-level names to pip-egg-info/ruamel.yaml.clib.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/ruamel.yaml.clib.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/ruamel.yaml.clib.egg-info/SOURCES.txt'
    reading manifest file 'pip-egg-info/ruamel.yaml.clib.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/ruamel.yaml.clib.egg-info/SOURCES.txt'
  Source in /tmp/pip-install-eO45HU/ruamel.yaml.clib has version 0.1.2, which satisfies requirement ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" from https://files.pythonhosted.org/packages/6a/6c/7b461053ce5be0d7c8b12dcae9a7c10e8012238a00f6fcd98643ee66d2de/ruamel.yaml.clib-0.1.2.tar.gz#sha256=cee86ecc893a6a8ecaa7c6a9c2d06f75f614176210d78a5f155f8e78d6989509 (from ruamel.yaml>=0.15.5; python_version != "3.4" and python_version < "3.7"->moban)
  Removed ruamel.yaml.clib>=0.1.2; platform_python_implementation == "CPython" and python_version < "3.8" from https://files.pythonhosted.org/packages/6a/6c/7b461053ce5be0d7c8b12dcae9a7c10e8012238a00f6fcd98643ee66d2de/ruamel.yaml.clib-0.1.2.tar.gz#sha256=cee86ecc893a6a8ecaa7c6a9c2d06f75f614176210d78a5f155f8e78d6989509 (from ruamel.yaml>=0.15.5; python_version != "3.4" and python_version < "3.7"->moban) from build tracker '/tmp/pip-req-tracker-gCDy8O'

https://gitlab.com/jayvdb/mobans/-/jobs/278858621

(originally posted on 2019-08-23 at 14:06:30 by John Vandenberg <John Vandenberg@bitbucket>)

[Anthon van der Neut]

in the commen setup.py this is solved, but once I found that the dict class had changed as much as it did in 3.3 (or 3.4) and I both realized that my almost working Python 3.1/3.2 version of ruamel.ordereddict needed further rewrite and that someone was redoing the whole thing starting with a tiny subset of what I had implemented I minimalized my effort. That minmal effort included not upgrading to the common setup.py for the latest “release” a month ago.

Essentially the setup.py for ruamel.ordereddict doesn’t do the optional test compile that for clib stops the ext_modules from being set, and I don’t know if it would make much sense to switch to the common setup.py, or just add the relevant code directly to the current setup.py, given Python 27 lifespan.

(originally posted on 2019-08-23 at 14:35:11)

[Anthon van der Neut]

Is ruamel.ordereddict strictly necessary on Python 2.7? Or could it be made an optional extra?

The code suggests that py27 collections.OrderedDict is enough, because iirc ordereddict.OrderedDict is the same as py27 collections.OrderedDict .

If so, then pip install --no-deps ruamel.yaml again is a good enough workaround for Alpine usage if size of disk image is more important than speed.

(originally posted on 2019-08-23 at 14:53:19 by John Vandenberg <John Vandenberg@bitbucket>)

[Anthon van der Neut]

collections.OrderedDict is pure Python IIRC, the OrderedDict in 3.x is C based. Not using ruamel.ordereddict makes things slower, but if that is a concern, YAML is not a good choice anyway, so --no-deps should certainly work.

It could be that some 3rd party lib relies on some ruamel.ordereddict feature not available in collections.OrderedDict. (after loading a mapping). But that lib would then only work on 2.7 unless the extended the ordereddict class in ruamel.yaml's compat.py like I did for .insert()

(originally posted on 2019-08-23 at 15:55:25)