LilyIssues / Issues / #3577 Patch: Build: Fix out-of-tree build from tarball.

Google Importer - 2013-09-24

Originally posted by: julien.r...@gmail.com

(No comment was entered for this change.)

Owner: julien.r...@gmail.com

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-09-24

Originally posted by: julien.r...@gmail.com

Remove echo, leftover from debugging.

http://codereview.appspot.com/13854043

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-09-24

Originally posted by: pkx1...@gmail.com

Patchy the autobot says: passes make, make check and a full make doc.

Labels: -Patch-new Patch-review

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-09-27

Originally posted by: pkx1...@gmail.com

Patch on countdown for October 1 - 06:00 GMT

Labels: -Patch-review Patch-countdown

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-09-27

Originally posted by: dak@gnu.org

I'm not convinced this "bison-less build" is making for a lot of happiness. It means that we need to distribute autogen.sh or at least configure in a form where it won't complain about a missing Bison, but where any attempt to change the respective sources will result in compilation failure.

Maybe we should just ditch the entire concept (including all EXTRA_DIST files). It only makes sense if you scratch the "preferred form for modification" idea from the GPL and instead consider the tarball the "prepared form for compilation", namely something more portable than a binary distribution.

Perhaps one litmus test of "should we reasonably expect Bison to be there?" would be whether our GUB process bothers with compiling Bison or not.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-09-27

Originally posted by: dak@gnu.org

As an additional remark: there is a roughly similar situation regarding the presence of autoconf and a pregeneration of configure for the tarball.

However, this is a manual process and such not dependent on something as fragile as file modification dates. People choose to run autogen.sh or not.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-09-27

Originally posted by: julien.r...@gmail.com

Is it making a lot of unhappiness? Why?

While we do ship autogen.sh, building from tarball does not require autotools at all, so you're not expected to run autogen. For configure, it already doesn't complain about missing bison, it merely suggests installing it as optional. So, no changes needed here.

Gub does compile his own bison, then uses it to build lilypond (and lilypond tarballs) from a git branch. Building from git does require you to have all dependencies. If some other process were to produce the tarballs and Gub would be building from those tarballs, then Gub could do without Bison, but that's not the way it currently operates (and without the patch from this issue, it would fail anyway).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-09-30

Originally posted by: pkx1...@gmail.com

Not sure on this one. David if there is nothing you want Julien to do or if you are unsure can you set this patch to either needs work or push - going to leave on countdown for now.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: dak@gnu.org

We are currently having the problem that the released tar ball for 2.17.27 does not build with simple

./autogen.sh --noconf
mkdir build
cd build
../configure
make

I've spent already more than a day with attempts of reproducing this behavior in-tree. One key to the failure appears to be the presence of Documentation/out/ly-grammar.txt which is not found when building in a separate build directory. Creating such a file (even if empty) seems to make the failure appear even with older tarballs.

This is reproducible from a _fresh_ checkout (add an mkdir Documentation/out; touch Documentation/out/ly-grammar.txt to the front of the recipe). "make clean" or "make distclean" are not sufficient for getting back to a pristine state; instead you need to use git clean -x -d -f for the cleanup.

This "separate build directory" problem is _unique_ to the bison-less build as far as I can tell: autoconf is _always_ run in the _original_ directory.

Now there is more involved than just the build directory: clearly the dependencies on ly-grammar.txt are broken in the wake of issue 3015 which failed to adjust them. The German translation still includes the grammar for some reason, however.

And I think there was work added for generating dependencies for includes. That stuff is likely fragile against moving the build directory, and pregenerated files do not help with that and get minimal testing.

So my vote is strongly on stopping Bison being an optional dependency. We don't make Flex optional, either.

Without the pregenerated grammar files, we have a whole dark corner regarding separate build dir, source/derived distinction and file modification date dependency _gone_. Sure, it is an interesting intellectual challenge to get all that right, but it does not buy us anything important, and we have lousy test coverage for it.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: julien.r...@gmail.com

I actually noticed that the tarball out-of-tree build was failing, but though that it might have been the case for a long time. I think issue 3530 or issue 3559 or a combination of the two is probably to blame. Make will see the file Documentation/out/ly-grammar.txt in the source tree and won't regenerate it, but in that case it should pass the correct -I command-line to makeinfo, which it apparently doesn't. There's no point trying to reproduce in-tree.

OK, I am now convinced that there's too much subtle trouble with this optional dependency that I'll turn this issue into making bison a required dependency. Bison-less build seems to have been broken for a while; it cannot have been that common for people to rely on it, although it was made optional 8 years ago: http://git.savannah.gnu.org/gitweb/?p=lilypond.git;a=commit;h=ace9039a15a1595a13e58927842647e7e4c90e78

Labels: -Patch-countdown Patch-needs_work

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: thomasmo...@gmail.com

@10
Compiling 2.16.2, 2.17.25 and 2.17.26 from the tarball worked fine!

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: dak@gnu.org

I have no actual idea why the presence of Documentation/out/ly-grammar.txt should make _any_ difference for an out-of-tree build.

Or why deleting the autoconf cache should make a difference. Now clearly we also have the problem that the dependencies for ly-grammar.txt are wrong: I see
git grep ly-grammar.txt
Documentation/GNUmakefile:OUT_DIST_FILES = $(outdir)/ly-grammar.txt
Documentation/GNUmakefile:$(outdir)/notation.texi: $(outdir)/ly-grammar.txt
Documentation/GNUmakefile:$(outdir)/ly-grammar.txt: $(top-src-dir)/lily/parser.yy
Documentation/contributor.texi:@verbatiminclude ly-grammar.txt
Documentation/de/notation.tely:@verbatiminclude ly-grammar.txt
So we have an outdated dependency of notation.texi on ly-grammar.txt but it turns out that even that outdated dependency simply was wrong since notation.texi will _still_ contain the @verbatiminclude and consequently _not_ depend on ly-grammar.txt. What actually depends on ly-grammar.txt is any target compiled _from_ notation.texi. Or now, contributor.texi.

We still need to fix that. But in the mean time, we can reduce the number of points we need to take care of by forcing the Bison dependency. That also avoids Heisenbugs based on whether Bison is run or not.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: dak@gnu.org

@10: try
mkdir Documentation/out
touch Documentation/out/ly-grammar.txt

right after unpacking the tar file and before the build procedure starting with ./autogen.sh --noconf. Do the build _only_ using "make" (no parallel build).

I get problems with a number of versions, not just 2.17.27. I wanted to bisect between master and 2.17.26 and accidentally bisected to 2.17.16 instead, and got build failures with that recipe (using git clean -d -x -f between tries) rather consistently.

Any less thorough cleanup or different setup, and things might work. Quite elusive. After wasting the better part of a day on it, I lean towards obliterating rather than solving that problem.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: julien.r...@gmail.com

@11: yes, because you have bison installed, so the necessary files can be generated.
@12: look up VPATH in the make manual.
@13: so that clearly points to issue 3530.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: julien.r...@gmail.com

Build: Fix out-of-tree build from tarball.

http://codereview.appspot.com/13854043

Summary: Patch: Build: Fix out-of-tree build from tarball.
Labels: -Patch-needs_work Patch-new

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: dak@gnu.org

@14@13: yes, issue 3530 likely was what pushed a fragile setup over the edge. But in principle, it should be in the same ballpark as lily/out/parser.cc and lily/out/parser.hh which we also distribute.

Taking a look at the new patch.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: julien.r...@gmail.com

Remove additional dependencies in Documentation/de, let issue 3588 take care of the rest.

http://codereview.appspot.com/13854043

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-01

Originally posted by: dak@gnu.org

Patchy the autobot says: passes tests.

Labels: -Patch-new Patch-review

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-02

Originally posted by: julien.r...@gmail.com

I'll venture putting this back on countdown.

Labels: -Patch-review Patch-countdown

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-04

Originally posted by: pkx1...@gmail.com

Patch counted down - please push

Labels: -Patch-countdown Patch-push

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-08

Originally posted by: julien.r...@gmail.com

[rd5b553119c63ebb4bd9d62adb8f4a200a2c4ad92]
[r862e6c059dc9487e921e7c48e356e848eae1877b]

Labels: -Patch-push Fixed_2_17_29
Status: Fixed

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Google Importer - 2013-10-21

Originally posted by: Elu...@gmail.com

(No comment was entered for this change.)

Status: Verified

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Patch: Build: Fix out-of-tree build from tarball.

Issue Tracker for LilyPond

Searches

Help

#3577 Patch: Build: Fix out-of-tree build from tarball.

Discussion