LilyIssues / Issues / #5384 Various fixes to GUB: - update broken download links - add missing dependencies - fix building of 'make' on glibc 2.27 systems - teach our ancient glibc that make 4.x exists

Knut Petersen - 2018-07-16

This allows GUB to be used on an ubuntu 14.04-5 system.

Install ubuntu 14.04-5, boot it, login as root, then execute

apt-get update apt-get install file gcc git g++ make texlive-fonts-recommended texlive-metapost texlive-xetex unzip zip

To use GUB you have to create a user gub. 'make lilypond' will only succeed if GUB is installed in '/home/gub/NewGub/gub' [Reason: our regtest results are erroneously built with dgs-load-fonts]

Login as gub, then

cd ~ mkdir NewGub cd NewGub git clone git://github.com/gperciva/gub.git cd gub

apply the patch, then

mkdir regtests cd regtests wget http://lilypond.org/downloads/binaries/test-output/lilypond-2.19.81-1.test-output.tar.bz2 touch ignore cd .. make lilypond LILYPOND_BRANCH=stable/2.20

IF none of the download links are broken (checked 2018-07-16) make takes about 160 minutes to finish on my i4790K system.

At the moment building of lilypond master is broken because of Alexander Myltsev's recent patch to musicexp.py that is incompatible to our ancient toolchain.

You may want to use an image of ubuntu with chroot. If the ubuntu system is installed on /dev/sdc2 and you want the image to be located at e.g. /home/knut/sources/ubuntu-14.04-5-server/, you could use something like

cd /home/knut/sources mkdir ubuntu-14.04-5-server cd ubuntu-14.04-5-server su mount /dev/sdc2 /mnt rsync -a /mnt/* . echo 'nameserver 8.8.8.8' > etc/resolv.conf mount --bind /tmp /home/knut/sources/ubuntu-14.04-5-server/tmp mount --bind /proc /home/knut/sources/ubuntu-14.04-5-server/proc mount --bind /dev /home/knut/sources/ubuntu-14.04-5-server/dev chroot /home/knut/sources/ubuntu-14.04-5-server

immediately after installing ubuntu to /dev/sdc2 and rebooting to you normal system. After that continue with 'apt-get update' etc as described above.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Knut Petersen - 2018-07-17

With the discussed patch GUB also succeeds to 'make lilypond ...' on Ubuntu 16.04.4 LTS.

Knut

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-07-17

Description has changed:

Diff:

Needs: -->

Patch: new --> review
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-07-17

Note: I cannot test this using the usual patch-test scripts. We have no specific procedure in place for GUB patches as it were. So I will set this to review but this can probably bypass the normal patch test countdown and get 'pushed' via comments from Phil/Masamichi et al who 'do' GUB-stuff

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Federico Bruni - 2018-07-19

Knut, I guess you'd better send a pull request to Github repository so Phil can easily test it and merge it. Or send him the patch file by email.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Knut Petersen - 2018-07-19

2nd version of GUB patch set, fix gmake segfaulting on glibc 2.27

http://codereview.appspot.com/369750043

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-07-19

summary: Various fixes to GUB: - update broken download links - add missing dependencies - fix building of 'make' on glibc 2.27 systems - teach our ancient glibc that make 4.x exists - minor fixes --> GUB: Various Fixes - update broken download links - add missing dependencies - fix building of 'make' on glibc 2.27 systems - teach our ancient glibc that make 4.x exists - minor fixes

Needs: -->

Patch: new --> review

Type: --> Enhancement
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

summary: GUB: Various Fixes - update broken download links - add missing dependencies - fix building of 'make' on glibc 2.27 systems - teach our ancient glibc that make 4.x exists - minor fixes --> Various fixes to GUB: - update broken download links - add missing dependencies - fix building of 'make' on glibc 2.27 systems - teach our ancient glibc that make 4.x exists - minor fixes
Description has changed:

Diff:

--- old
+++ new
@@ -1,4 +1,22 @@
-Various fixes to GUB:  - update broken download links  - add missing dependencies  - fix building of 'make' on glibc 2.27 systems  - teach our ancient glibc that make 4.x exists  - minor fixes
+Various fixes to GUB:
+  - update broken download links 
+  - add missing dependencies 
+  - fix 'make' for glibc 2.27 systems
+  - teach our ancient glibc that make 4.x exists
+
+Status on some selected distributions:
+
+ubuntu-14.04.5 LTS:
+  'make lilypond LILYPOND_BRANCH=stable/2.20' succeeds
+
+ubuntu-16.04.4 LTS:
+  'make lilypond LILYPOND_BRANCH=stable/2.20' succeeds
+
+ubuntu-18.04 LTS:
+  'make bootstrap' succeeds, but crossbuilding guile fails during 'make lilypond LILYPOND_BRANCH=stable/2.20'
+
+openSuSE Tumbleweed:
+  make segfaults fixed, but here we see that some more dependencies need to be fixed.

 Signed-off-by: Knut Petersen <knut_petersen@t-online.de>

Patch: review --> new
Type: Enhancement -->

Anonymous - 2018-07-19

Patch: new --> review

Type: --> Enhancement
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-07-22

Patch: review --> push
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-07-22

Knut sned a pull request please to the GUB github repo.

https://github.com/gperciva/gub

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Knut Petersen - 2018-07-27

Fix the 'db delete failing' problem. Needed at least for openSuSE Tumbleweed.

http://codereview.appspot.com/369750043

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Description has changed:

Diff:

--- old
+++ new
@@ -3,6 +3,7 @@
   - add missing dependencies 
   - fix 'make' for glibc 2.27 systems
   - teach our ancient glibc that make 4.x exists
+  - fix 'db delete failing' problem on openSuSE Tumbleweed

 Status on some selected distributions:

@@ -12,11 +13,8 @@
 ubuntu-16.04.4 LTS:
   'make lilypond LILYPOND_BRANCH=stable/2.20' succeeds

-ubuntu-18.04 LTS:
-  'make bootstrap' succeeds, but crossbuilding guile fails during 'make lilypond LILYPOND_BRANCH=stable/2.20'
-
-openSuSE Tumbleweed:
-  make segfaults fixed, but here we see that some more dependencies need to be fixed.
+ubuntu-18.04 LTS and openSuSE Tumbleweed
+  'make bootstrap' succeeds, but crossbuilding guile still fails during 'make lilypond LILYPOND_BRANCH=stable/2.20'

 Signed-off-by: Knut Petersen <knut_petersen@t-online.de>

Needs: -->
Type: -->

Anonymous - 2018-07-29

Patch: new --> review

Type: --> Maintainability
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-08-01

Patch: review --> countdown
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-08-04

Patch: countdown --> push
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-08-04

Knut,

This has been 'counted down' can you make a pull request to the approrpiate repo please?

You'll have to do this yourself.

James

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-08-07

Knut,

This has been 'counted down' can you make a pull request to the approrpiate repo please?

You'll have to do this yourself.

James

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Anonymous - 2018-08-10

Knut,

Make a pull request please at

https://github.com/gperciva/gub

James

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Federico Bruni - 2018-08-27

labels: --> GUB
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Knut, I'm testing your patch on Fedora 28.

When building glibc (during the make bootstrap step), it seems that it's still using my system make (version 4.2.1) instead of the version (3.81) downloaded by gub. This is a problem that was supposed to be fixed by your patch?
I've downloaded the 3rd patchset from Rietveld, applied it with patch -p1 and committed to this branch:
https://github.com/fedelibre/gub/tree/5384-knut-patch

$ target/tools/root/usr/bin/make --version
GNU Make 3.81

$ make --version
GNU Make 4.2.1

These lines are taken from target/linux-64/build/glibc-core-2.3/config.log (full file attached):

configure:4112: checking for gmake
configure:4128: found /usr/bin/gmake
configure:4138: result: gmake
configure:4152: checking version of gmake
configure:4162: result: 4.2.1, bad
...
configure:4261: result: makeinfo
configure:4275: checking version of makeinfo
configure:4285: result: 6.1, bad
...
*** These critical programs are missing or too old: make
*** Check the INSTALL file for required versions.
...
MAKE='gmake'
MAKEINFO=':'

And this is the tail of make bootstrap:

building package: linux-64::cross/gcc-core
 *** Stage: download (cross/gcc-core, linux-64)
 *** Stage: untar (cross/gcc-core, linux-64)
 *** Stage: patch (cross/gcc-core, linux-64)
 *** Stage: autoupdate (cross/gcc-core, linux-64)
 *** Stage: configure (cross/gcc-core, linux-64)
 *** Stage: compile (cross/gcc-core, linux-64)
 *** Stage: install (cross/gcc-core, linux-64)
 *** Stage: package (cross/gcc-core, linux-64)
 *** Stage: clean (cross/gcc-core, linux-64)
 *** Stage: pkg_install (cross/gcc-core, linux-64)

This compile has previously been interrupted.
To ensure a repeatable build, this will not be packaged.

Run with

    --fresh # or issue
              rm /home/fede/src/gub/target/linux-64/status/glibc-core-2.3-glibc-2.3-20070416

to force a full package rebuild, or

    --force-package

to skip this check and risk a defective build.
building package: linux-64::glibc-core
 *** Stage: download (glibc-core, linux-64)
 *** Stage: configure (glibc-core, linux-64)
Command barfed: cd /home/fede/src/gub/target/linux-64/build/glibc-core-2.3 && chmod +x /home/fede/src/gub/target/linux-64/src/glibc-core-2.3/configure && BUILD_CC=gcc  sh /home/fede/src/gub/target/linux-64/src/glibc-core-2.3/configure --prefix=/usr --cache-file=config.cache --enable-shared --disable-static --disable-silent-rules --build=x86_64-linux --host=x86_64-linux --target=x86_64-linux --prefix=/usr --sysconfdir=/usr/etc --includedir=/usr/include --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib  --disable-profile --disable-debug --without-cvs --without-gd --with-headers=/home/fede/src/gub/target/linux-64/root/usr/include  --enable-add-ons=linuxthreads --without-tls --without-__thread 

Tail of target/linux-64/log/glibc-core.log >>>>>>>>
    configure: error:
    *** These critical programs are missing or too old: make
    *** Check the INSTALL file for required versions.
    Command barfed: cd /home/fede/src/gub/target/linux-64/build/glibc-core-2.3 && chmod +x /home/fede/src/gub/target/linux-64/src/glibc-core-2.3/configure && BUILD_CC=gcc  sh /home/fede/src/gub/target/linux-64/src/glibc-core-2.3/configure --prefix=/usr --cache-file=config.cache --enable-shared --disable-static --disable-silent-rules --build=x86_64-linux --host=x86_64-linux --target=x86_64-linux --prefix=/usr --sysconfdir=/usr/etc --includedir=/usr/include --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib  --disable-profile --disable-debug --without-cvs --without-gd --with-headers=/home/fede/src/gub/target/linux-64/root/usr/include  --enable-add-ons=linuxthreads --without-tls --without-__thread
<<<<<<<< Tail of target/linux-64/log/glibc-core.log

*** Failed target: linux-64::glibc-core
make: *** [compilers.make:44: cross-compilers] Error 1

The patches/glibc-allow-make4.patch file seems to be applied. I've checked the relevant lines in this file:

nano +4154 src/glibc-core-2.3/configure

config.log

Federico Bruni - 2018-09-12

Find attached also the file target/linux-64/log/glibc-core.log.

glibc-core.log

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Federico Bruni - 2018-09-12

Sorry, actually the patch has not been applied! The 4.x regexp is missing.
Tomorrow I'll try a clean build.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Federico Bruni - 2018-09-12

I've edited manually the file src/glibc-core-2.3/configure and run make bootstrap again. This time I've managed to pass the glibc-core build step:

building package: linux-64::glibc-core *** Stage: download (glibc-core, linux-64) *** Stage: untar (glibc-core, linux-64) *** Stage: patch (glibc-core, linux-64) *** Stage: autoupdate (glibc-core, linux-64) *** Stage: configure (glibc-core, linux-64) *** Stage: compile (glibc-core, linux-64) *** Stage: install (glibc-core, linux-64) *** Stage: package (glibc-core, linux-64) *** Stage: clean (glibc-core, linux-64) *** Stage: pkg_install (glibc-core, linux-64)

even though above lines would suggest that all the files have been downloaded again from scratch and therefore the patch would have been applied.
I'm confused...

Anyway let's see if I manage to complete the bootstrap step and then I'll make a clean build in another directory.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

make bootstrap completed successfully.

Now make lilypond is failing in the compile stage of tools::glib:

building package: tools::glib
 *** Stage: download (glib, tools)
 *** Stage: untar (glib, tools)
 *** Stage: patch (glib, tools)
 *** Stage: autoupdate (glib, tools)
 *** Stage: configure (glib, tools)
 *** Stage: compile (glib, tools)
Command barfed: cd /home/fede/src/gub/target/tools/build/glib-2.44.1 && make -j8  

Tail of target/tools/log/glib.log >>>>>>>>
    make[1]: *** [all-recursive] Error 1
    make[1]: Leaving directory `/home/fede/src/gub/target/tools/build/glib-2.44.1'
    make: *** [all] Error 2
    Command barfed: cd /home/fede/src/gub/target/tools/build/glib-2.44.1 && make -j8
<<<<<<<< Tail of target/tools/log/glib.log

*** Failed target: tools::glib
make[1]: *** [gub.make:63: packages] Error 1
make[1]: uscita dalla directory "/home/fede/src/gub"
make: *** [GNUmakefile:26: lilypond] Error 2

The last lines of glib.log (full file attached) are:

make[4]: Leaving directory `/home/fede/src/gub/target/tools/build/glib-2.44.1/glib'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/home/fede/src/gub/target/tools/build/glib-2.44.1/glib'
make[2]: *** [all] Error 2
make[2]: Leaving directory `/home/fede/src/gub/target/tools/build/glib-2.44.1/glib'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/fede/src/gub/target/tools/build/glib-2.44.1'
make: *** [all] Error 2
Command barfed: cd /home/fede/src/gub/target/tools/build/glib-2.44.1 && make -j8  
Traceback (most recent call last):
  File "bin/gub", line 233, in exceptional_build
    build (settings, options, files)
  File "bin/gub", line 229, in build
    b.build_source_packages (names)
  File "bin/../gub/buildrunner.py", line 334, in build_source_packages
    self.spec_build (spec_name)
  File "bin/../gub/buildrunner.py", line 262, in spec_build
    deferred_runner.execute_deferred_commands ()
  File "bin/../gub/runner.py", line 167, in execute_deferred_commands
    cmd.execute (self.logger)
  File "bin/../gub/commands.py", line 75, in execute
    ignore_errors=self.ignore_errors)
  File "bin/../gub/loggedos.py", line 93, in system
    raise misc.SystemFailed (m)
SystemFailed: Command barfed: cd /home/fede/src/gub/target/tools/build/glib-2.44.1 && make -j8

glib.log

Various fixes to GUB: - update broken download links - add missing...

Issue Tracker for LilyPond

Searches

Help

#5384 Various fixes to GUB: - update broken download links - add missing dependencies - fix building of 'make' on glibc 2.27 systems - teach our ancient glibc that make 4.x exists - minor fixes

Related

Discussion