libmesh-devel — General information for libmesh developers

Re: [Libmesh-devel] Still Really Dislike Automake

[Paul T. B. <ptb...@gm...>]

On Thu, Aug 30, 2018 at 3:28 PM Derek Gaston <fri...@gm...> wrote:

>
> On Thu, Aug 30, 2018 at 3:09 PM Paul T. Bauman <ptb...@gm...> wrote:
>
>> On Thu, Aug 30, 2018 at 3:04 PM Paul T. Bauman <ptb...@gm...>
>> wrote:
>>
>>>
>>>
>>> On Thu, Aug 30, 2018 at 3:02 PM Derek Gaston <fri...@gm...> wrote:
>>>
>>>> In general you guys are talking about "features" though: when in my
>>>> estimation the "core" capability of the development cycle is broken.  I
>>>> don't think that maintaining "nice" features is worth it at the cost of
>>>> hurting the main development cycle
>>>> (build->fix->build->fix->add_file->build->fix->test).
>>>>
>>>
>> I should've added here, this is entirely how our build system currently
>> works. I've had a build tree. I make a modification in the source. I hit
>> `make` in the build tree. I edit the source. I hit `make` in the build
>> tree. I hit make install. (With install -C so GRINS only rebuilds what's
>> necessary)
>>
>
> Oh really?  How do you do that when you add a file?
>

I add the file in the source tree. And then I type `make` in my build tree.


>   I think you missed the steps where you have to hunt down the scripts and
> get the Makefiles modified.
>

That doesn't matter. You don't need to reconfigure.


> Then you have makefile modifications mixed in with your changes.  Do you
> try to save those off in separate commits - or do you just smash everything
> together?
>

Yes, the build system changes are in separate commits. Like we've been
doing for years.


>
> edit->make->test->add_file->make->test
>
>
No need for "install" at all.
>

The only time I need to install is if I'm dropping in a change into libMesh
that I'm consuming in GRINS. If I have a self-contained libMesh feature, I
do not install it until I'm testing downstream.


> No need to modify Makefiles.  No need to run scripts - all done within my
> editor with source linking to compile errors.  Not currently possible.
>

Again, the building within the editor may be an issue. I'm willing to bet
it's fixable given the number of developers that use emacs together with
the literally thousands of Linux packages that use autotools as their build
system.


>
> Derek
>

Re: [Libmesh-devel] Still Really Dislike Automake

[Paul T. B. <ptb...@gm...>]

On Thu, Aug 30, 2018 at 3:24 PM Derek Gaston <fri...@gm...> wrote:

>
>
> On Thu, Aug 30, 2018 at 3:19 PM Derek Gaston <fri...@gm...> wrote:
>
>>
>> -1. Rebuilding/linking of all the small executables every time
>> -2. In editor building broken (in multiple ways)
>> -3. Pain to add any single new file (especially headers)
>> -4. Thousands of Makefiles
>> -4b. Makefiles in include directories cause in-editor building to get
>> hung up (I have to switch to a .C file and build from there)
>> -5. Creating new examples is a HUGE pain
>> -6. Trying to use an example to try out a new feature (or editing an
>> existing example) also sucks.  You have to use pretty funky command-lines
>> just to get a single executable re-run.
>>
>
> Forgot one:
>
> -7. Extraneous diffs that always come along with any change because tons
> of makefile garbage changed just by adding one file / directory / example.
>

You guys get the blame for this one. There was insistence from MOOSE
developers that the bootstrapped build system be included in the master
tree. I was against it.


> In general: I just don't understand why the _build system_ should ever be
> changing!  This my main complaint with what we have with Automake and my
> main complaint against CMake as well.  The build system should only ever
> change if a change is actually needed to the _build system_.  It shouldn't
> change because of normal development activity (like adding a file!).
>
> Derek
>

Re: [Libmesh-devel] Still Really Dislike Automake

[Paul T. B. <ptb...@gm...>]

On Thu, Aug 30, 2018 at 3:20 PM Derek Gaston <fri...@gm...> wrote:

> On Thu, Aug 30, 2018 at 3:02 PM Paul T. Bauman <ptb...@gm...> wrote:
>
>> On Thu, Aug 30, 2018 at 1:32 PM Derek Gaston <fri...@gm...> wrote:
>>
>>> On Thu, Aug 30, 2018 at 11:54 AM Roy Stogner <roy...@ic...>
>>>
>>
>> I strongly disagree with this statement. It is *vital* for development.
>> On my CIVET server I have multiple builds with different
>> options/configurations and I would have to reconfigure and rebuild the
>> entire bloody tree for one commit change instead of going to the build
>> directory and hitting `make` and only building the files and dependencies
>> for each configuration. It is a *HUGE* time saver. Let alone on my own box.
>>
>
> I sincerely doubt it - even the tiniest change (like adding a header file
> that is only included in one .C file triggers a complete reconfigure.
>

That shouldn't happen. I'm betting that you made another change along side
or it's an artifact of the symlinking bits (which needs to be fixed). The
only time configure ever needs to be rerun is if you change your
configuration you or you change how you configure your build (configure.ac).
When I add new files and hit `make` in my build tree, automake reruns and
then proceeds with the build, regardless of .h or .C file additions. Even
changing Makefile.am does not need a reconfigure.


> Any tiny change to mesh / equation_systems / etc. creates a whole
> rebuild.  I'd love to see statistics on how much this saves you.
>

Sure, because there is a lot of dependency in those files, but I claim most
of the libMesh changes happen much farther downstream of that. In fact,
rarely do I have to recompile more than a handful of libMesh files.


> On the other end, users don't need it at all because they are usually just
>>> happy to get something to build for _one_ environment configuration (and
>>> many never, ever want to rebuild after that).  I seriously think it's just
>>> not worth it.
>>>
>>
>> *Users* may not care, but it is *vital* for developers. And I'm a
>> developer, not just a user.
>>
>
> How many developers are using it?  I know that I _never_ do so "vital" is
> a bit too far.  We developed libMesh for MANY years without it as well.
>

I perish to think how many needless complete rebuilds of the stack was
required for a small change.


> In addition: MOOSE has never had it and I've only ever heard one person
> complain (Roy).
>
> Most of the MOOSE developers (besides myself and John) simply run the
> MOOSE script that we have to rebuild libMesh because they can't even be
> bothered to deal with it manually.  Before Automake that script never even
> existed!
>

Again, in-source builds just worked for me literally just now. `configure;
make` or just `make`. I'm not sure what else you're doing your script.


> Now, "install" I can see a need for.  I don't personally care about it -
>>> but the arguments are convincing enough.  It's something that both
>>> developers and users might want to use.
>>>
>>> In my mind "install" also obviates a lot of the need for out-of-tree
>>> builds: if you need multiple configurations built - just install them to
>>> different places.
>>>
>>
>> From a user point of view, sure. From a developer point of view, this is
>> simply false. I want my different build configurations sitting around so I
>> only have to rebuild what's necessary for that configuration without having
>> to reconfigure and rebuild *all* files, not just dependencies for each
>> configuration.
>>
>
> Again - I simply don't see it.  Even in my own workflow I trigger complete
> rebuilds of most of libMesh fairly often.  Do you really make one tiny
> change then go rebuild 5 configurations every time?
>

No, usually two or three (mostly devel, then opt, but then sometimes dbg
with stdcxx-debug enabled for really detailed tracing). But sometimes
different versions of PETSc with the previous variations. But yes, in those
cases, usually only a handful of files get rebuilt. Of course if you change
something in mesh_base.h you're basically rebuilding the whole library, but
most of the time we are touching something much less "central" (solvers,
elements, FEMSystem).


>   I would think that you would work with one configuration for quite a
> while - then go try others.  At the point you make many edits and want to
> try another configuration you're going to end up rebuilding most (if not
> all) of the tree - in addition you're probably going to trigger a
> re-configure.
>

No. Triggering a reconfigure is very rare for me.


> Also - with CI this is less important - work in one configuration... let
> CI handle the others.  Only build a "non-standard" configuration when CI
> shows that you've botched it.
>

Huh? There are plenty of times already where devel passed and dbg didn't.
Running these configurations in tandem always is critical.


>
>
>> If out-of-tree-builds are a requirement: I would like them off by default.
>>>
>>
>> I strongly disagree. In fact, I do not understand what the problem is
>> here regarding in vs. out-of-source builds. I just did an in-source build
>> of libMesh master and it worked fine (I recall we made sure that this
>> worked for you guys). If you're having to do any maintenance for in-source
>> builds I don't know what because it's not on the libMesh side.
>>
>
> It "kinda" works - I have to maintain my own "install" script that gets
> broken every now and again.
>

How? It literally just did it and it worked fine. What is in your script
besides `make install`?


>   Also: the pathing is broken on compile errors in this mode so that Emacs
> can't figure out where the source files are.
>
>
>> In fact, other that your original complaint about adding the file to the
>> build system (for which John created a wiki page so we can all remember),
>> I've not seen any other actual issue other that just being unhappy with
>> Autotools.
>>
>
> Literally every time I have to do development in libMesh I run into the
> friction.  The fact that we have to have a wiki page about how to add a new
> file just shows that it's broken.
>

It shows adding a file should be streamlined. It's hyperbole to extrapolate
that to saying the build system is broken when literally every feature of
the current build system is exercised and is passing on CI.


> Positives:
>>
>> 1. Out-of-source builds just work. In-source builds work. You can do
>> either. And this requires no maintenance for files compiled into the
>> library (there is a little work for new examples, but it's mostly
>> automated).
>> 2. make install just works. For both in-source and out-of-source builds.
>> 3. make check just works. For both in-source and out-of-source builds.
>> 4. make dist just works for preparing release tar balls.
>> 5. make distcheck just works for checking said release tar balls will
>> actually configure/make/make check/make install.
>> 6. It will work on any system that supports make and a shell script
>> (configure).
>>
>> 1,2,3,4,5,6 are vital, non-negotiable features for me.
>>
>> Negatives:
>>
>> 1. The automation of adding a new file should be more straight forward
>> (which I think it would be if we just did away with the symlinks, but I've
>> forgotten why we needed that in the first place, so maybe we can't).
>> 2. ... What else?
>>
>
> -1. Rebuilding/linking of all the small executables every time
>

I agree, but that's easy to enable configure option for (which you'd need
to do in any other build system anyway).


> -2. In editor building broken (in multiple ways)
>

This may be valid. I don't do in editor building.


> -3. Pain to add any single new file (especially headers)
>

This was number 1 under negatives, no double-counting allowed.


> -4. Thousands of Makefiles
>

(A handful) That you never need to look at, touch, or otherwise modify?
This is not a valid criticism.


> -4b. Makefiles in include directories cause in-editor building to get hung
> up (I have to switch to a .C file and build from there)
>

I believe this is only because of the symlinking which I agree should go
away.


> -5. Creating new examples is a HUGE pain
>

It is annoying to get it to `make install` properly (and not hugely
annoying it is mostly automated). Creating a new example is easy.


> -6. Trying to use an example to try out a new feature (or editing an
> existing example) also sucks.  You have to use pretty funky command-lines
> just to get a single executable re-run.
>

Huh? I'm not sure how it gets any easier then literally running the script
that is installed along side the example.

Re: [Libmesh-devel] Still Really Dislike Automake

[Jed B. <je...@je...>]

Derek Gaston <fri...@gm...> writes:

> Or: like I said earlier, maybe we just maintain two systems.  PETSc has had
> several different build systems all at the same time because developers
> wanted different things.  

It's important to note that all build systems have used the same
underlying specification.  The GNU make and CMake systems both use/used
a generator script that read the file lists from legacy makefiles.
We've ditched CMake and are phasing out the legacy makefiles because GNU
make is so much faster, more reliable, and less code.

I'm with Paul that out-of-source builds are essential, but it isn't hard
to put build products in a directory of the user's choosing (a la
PETSC_ARCH).  I have 35 PETSC_ARCHes at the moment and I appreciate
being able to switch between them and update builds in a couple seconds.

> It definitely could be true that a really simple pure Make system for
> everyone that doesn't need the fancy features of Automake and then
> Automake for Roy and Paul might be the way to go.

The hard part is _not_ make (unless you absolutely need to work with the
intersection of GNU and BSD makes).  Configuration is hard and all
solutions are terrible.

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

On Thu, Aug 30, 2018 at 4:02 PM Jed Brown <je...@je...> wrote:

> Derek Gaston <fri...@gm...> writes:
>
> > Or: like I said earlier, maybe we just maintain two systems.  PETSc has
> had
> > several different build systems all at the same time because developers
> > wanted different things.
>
> It's important to note that all build systems have used the same
> underlying specification.  The GNU make and CMake systems both use/used
> a generator script that read the file lists from legacy makefiles.
> We've ditched CMake and are phasing out the legacy makefiles because GNU
> make is so much faster, more reliable, and less code.
>

Thanks Jed - definitely appreciate your insight!


> I'm with Paul that out-of-source builds are essential, but it isn't hard
> to put build products in a directory of the user's choosing (a la
> PETSC_ARCH).  I have 35 PETSC_ARCHes at the moment and I appreciate
> being able to switch between them and update builds in a couple seconds.
>

I agree - I do think that a GNU Make solution can do out-of-source builds.
And if that's what's needed then we can make it happen here.


> The hard part is _not_ make (unless you absolutely need to work with the
> intersection of GNU and BSD makes).  Configuration is hard and all
> solutions are terrible.
>

Luckily we have a really good set of configure scripts.  "configure" has
its own problems... but we (mostly because of hard work by John) have a
pretty good handle on it.  We wouldn't need to re-engineer that for this
new effort.

Derek

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

Also: let me bring up one more huge inefficiency that our users deal with:
METHODS.

Currently METHODS has to be set at configure time - and there is no way to
configure for all the METHODS you want and then only compile a particular
METHOD.  This kills new MOOSE users because we want them to be able to run
both opt and dbg... and the only way to to do that is to force them to
compile libMesh twice (which can take a very long time).

That wasn't true in the pre-Automake days - and I wish it weren't true
now.  It's definitely something I would change with the new build system.

Derek

On Thu, Aug 30, 2018 at 3:52 PM Roy Stogner <roy...@ic...>
wrote:

>
>
> On Thu, 30 Aug 2018, Derek Gaston wrote:
>
> > However, it's obvious that you're both passionate about these... so
> > I will relent and agree that whatever build system we come up with
> > has to have these features.  Fine.
>
> Thanks.
>
> > Can you agree with me that the current build system adds friction
> > for everyone that doesn't use these features?
>
> Yup.  I think adding new files is the worst-case scenario, but there's
> flaws (actually, "friction" is the perfect word) in every case.
>
> > I think you guys are so used to how slow and laborious it is that
> > you don't even realize how much better it could be.
>
> No, I remember pre-automake libMesh too, and it was certainly better
> in some ways, just much worse in some others.  I still think the move
> to automake was an improvement but we're definitely not at an optimum.
> ---
> Roy

Re: [Libmesh-devel] Still Really Dislike Automake

[Roy S. <roy...@ic...>]

On Thu, 30 Aug 2018, Derek Gaston wrote:

> However, it's obvious that you're both passionate about these... so
> I will relent and agree that whatever build system we come up with
> has to have these features.  Fine.

Thanks.

> Can you agree with me that the current build system adds friction
> for everyone that doesn't use these features?

Yup.  I think adding new files is the worst-case scenario, but there's
flaws (actually, "friction" is the perfect word) in every case.

> I think you guys are so used to how slow and laborious it is that
> you don't even realize how much better it could be.

No, I remember pre-automake libMesh too, and it was certainly better
in some ways, just much worse in some others.  I still think the move
to automake was an improvement but we're definitely not at an optimum.
---
Roy

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

What I mean is: what if you guys could have the things you like... and we
could also have a sane, clean system that doesn't require all the BS we
currently have to deal with when modifying libMesh?

It is definitely possible.

Or: like I said earlier, maybe we just maintain two systems.  PETSc has had
several different build systems all at the same time because developers
wanted different things.  It definitely could be true that a really simple
pure Make system for everyone that doesn't need the fancy features of
Automake and then Automake for Roy and Paul might be the way to go.

Derek

On Thu, Aug 30, 2018 at 3:44 PM Derek Gaston <fri...@gm...> wrote:

> Both of you claim to use these features - fine - I believe that you do...
> but how many others?
>
> However, it's obvious that you're both passionate about these... so I will
> relent and agree that whatever build system we come up with has to have
> these features.  Fine.
>
> Can you agree with me that the current build system adds friction for
> everyone that doesn't use these features?  I think you guys are so used to
> how slow and laborious it is that you don't even realize how much better it
> could be.
>
> This happened the other day with John and I: I was complaining about how
> slow "make install" is for libMesh.  John said "what do you mean, seems
> fine to me"... after a bit of chatting it was clear that he was just so
> used to it that it didn't phase him (he just types "make install" and let's
> it do its thing)... but that doesn't mean that the inefficiency doesn't
> exist!
>
> Derek
>
> On Thu, Aug 30, 2018 at 3:15 PM Roy Stogner <roy...@ic...>
> wrote:
>
>>
>> On Thu, 30 Aug 2018, Derek Gaston wrote:
>>
>> > I would rather fix the core development cycle - then backfill features
>> based on priority (install > check > dist > out-of-tree, etc.)
>>
>> out-of-tree > install > check > dist.
>>
>> > completely chucked a sane development flow for the sake of a few
>> features that are rarely actually used.
>>
>> By "rarely" do you mean "literally all the time, for years,
>> indispensibly"?  Being able to easily build multiple configurations of
>> the same working tree is incredibly useful.  The inability to do this
>> as easily with MOOSE has cost in man-hours of both workarounds (when I
>> maintain multiple trees to test different software stacks) and errors
>> (when I don't have room to do so or time to go back-and-forth and one
>> configuration or stack regresses).
>> ---
>> Roy
>>
>

Re: [Libmesh-devel] Still Really Dislike Automake

[Roy S. <roy...@ic...>]

On Thu, 30 Aug 2018, Derek Gaston wrote:

> I would rather fix the core development cycle - then backfill features based on priority (install > check > dist > out-of-tree, etc.)

out-of-tree > install > check > dist.

> completely chucked a sane development flow for the sake of a few features that are rarely actually used.

By "rarely" do you mean "literally all the time, for years,
indispensibly"?  Being able to easily build multiple configurations of
the same working tree is incredibly useful.  The inability to do this
as easily with MOOSE has cost in man-hours of both workarounds (when I
maintain multiple trees to test different software stacks) and errors
(when I don't have room to do so or time to go back-and-forth and one
configuration or stack regresses).
---
Roy

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

Both of you claim to use these features - fine - I believe that you do...
but how many others?

However, it's obvious that you're both passionate about these... so I will
relent and agree that whatever build system we come up with has to have
these features.  Fine.

Can you agree with me that the current build system adds friction for
everyone that doesn't use these features?  I think you guys are so used to
how slow and laborious it is that you don't even realize how much better it
could be.

This happened the other day with John and I: I was complaining about how
slow "make install" is for libMesh.  John said "what do you mean, seems
fine to me"... after a bit of chatting it was clear that he was just so
used to it that it didn't phase him (he just types "make install" and let's
it do its thing)... but that doesn't mean that the inefficiency doesn't
exist!

Derek

On Thu, Aug 30, 2018 at 3:15 PM Roy Stogner <roy...@ic...>
wrote:

>
> On Thu, 30 Aug 2018, Derek Gaston wrote:
>
> > I would rather fix the core development cycle - then backfill features
> based on priority (install > check > dist > out-of-tree, etc.)
>
> out-of-tree > install > check > dist.
>
> > completely chucked a sane development flow for the sake of a few
> features that are rarely actually used.
>
> By "rarely" do you mean "literally all the time, for years,
> indispensibly"?  Being able to easily build multiple configurations of
> the same working tree is incredibly useful.  The inability to do this
> as easily with MOOSE has cost in man-hours of both workarounds (when I
> maintain multiple trees to test different software stacks) and errors
> (when I don't have room to do so or time to go back-and-forth and one
> configuration or stack regresses).
> ---
> Roy
>

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

On Thu, Aug 30, 2018 at 3:09 PM Paul T. Bauman <ptb...@gm...> wrote:

> On Thu, Aug 30, 2018 at 3:04 PM Paul T. Bauman <ptb...@gm...> wrote:
>
>>
>>
>> On Thu, Aug 30, 2018 at 3:02 PM Derek Gaston <fri...@gm...> wrote:
>>
>>> In general you guys are talking about "features" though: when in my
>>> estimation the "core" capability of the development cycle is broken.  I
>>> don't think that maintaining "nice" features is worth it at the cost of
>>> hurting the main development cycle
>>> (build->fix->build->fix->add_file->build->fix->test).
>>>
>>
> I should've added here, this is entirely how our build system currently
> works. I've had a build tree. I make a modification in the source. I hit
> `make` in the build tree. I edit the source. I hit `make` in the build
> tree. I hit make install. (With install -C so GRINS only rebuilds what's
> necessary)
>

Oh really?  How do you do that when you add a file?  I think you missed the
steps where you have to hunt down the scripts and get the Makefiles
modified.  Then you have makefile modifications mixed in with your
changes.  Do you try to save those off in separate commits - or do you just
smash everything together?

edit->make->test->add_file->make->test

No need for "install" at all.  No need to modify Makefiles.  No need to run
scripts - all done within my editor with source linking to compile errors.
Not currently possible.

Derek

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

On Thu, Aug 30, 2018 at 3:19 PM Derek Gaston <fri...@gm...> wrote:

>
> -1. Rebuilding/linking of all the small executables every time
> -2. In editor building broken (in multiple ways)
> -3. Pain to add any single new file (especially headers)
> -4. Thousands of Makefiles
> -4b. Makefiles in include directories cause in-editor building to get hung
> up (I have to switch to a .C file and build from there)
> -5. Creating new examples is a HUGE pain
> -6. Trying to use an example to try out a new feature (or editing an
> existing example) also sucks.  You have to use pretty funky command-lines
> just to get a single executable re-run.
>

Forgot one:

-7. Extraneous diffs that always come along with any change because tons of
makefile garbage changed just by adding one file / directory / example.

In general: I just don't understand why the _build system_ should ever be
changing!  This my main complaint with what we have with Automake and my
main complaint against CMake as well.  The build system should only ever
change if a change is actually needed to the _build system_.  It shouldn't
change because of normal development activity (like adding a file!).

Derek

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

On Thu, Aug 30, 2018 at 3:02 PM Paul T. Bauman <ptb...@gm...> wrote:

> On Thu, Aug 30, 2018 at 1:32 PM Derek Gaston <fri...@gm...> wrote:
>
>> On Thu, Aug 30, 2018 at 11:54 AM Roy Stogner <roy...@ic...>
>>
>
> I strongly disagree with this statement. It is *vital* for development. On
> my CIVET server I have multiple builds with different
> options/configurations and I would have to reconfigure and rebuild the
> entire bloody tree for one commit change instead of going to the build
> directory and hitting `make` and only building the files and dependencies
> for each configuration. It is a *HUGE* time saver. Let alone on my own box.
>

I sincerely doubt it - even the tiniest change (like adding a header file
that is only included in one .C file triggers a complete reconfigure.  Any
tiny change to mesh / equation_systems / etc. creates a whole rebuild.  I'd
love to see statistics on how much this saves you.


> On the other end, users don't need it at all because they are usually just
>> happy to get something to build for _one_ environment configuration (and
>> many never, ever want to rebuild after that).  I seriously think it's just
>> not worth it.
>>
>
> *Users* may not care, but it is *vital* for developers. And I'm a
> developer, not just a user.
>

How many developers are using it?  I know that I _never_ do so "vital" is a
bit too far.  We developed libMesh for MANY years without it as well.  In
addition: MOOSE has never had it and I've only ever heard one person
complain (Roy).

Most of the MOOSE developers (besides myself and John) simply run the MOOSE
script that we have to rebuild libMesh because they can't even be bothered
to deal with it manually.  Before Automake that script never even existed!


> Now, "install" I can see a need for.  I don't personally care about it -
>> but the arguments are convincing enough.  It's something that both
>> developers and users might want to use.
>>
>> In my mind "install" also obviates a lot of the need for out-of-tree
>> builds: if you need multiple configurations built - just install them to
>> different places.
>>
>
> From a user point of view, sure. From a developer point of view, this is
> simply false. I want my different build configurations sitting around so I
> only have to rebuild what's necessary for that configuration without having
> to reconfigure and rebuild *all* files, not just dependencies for each
> configuration.
>

Again - I simply don't see it.  Even in my own workflow I trigger complete
rebuilds of most of libMesh fairly often.  Do you really make one tiny
change then go rebuild 5 configurations every time?  I would think that you
would work with one configuration for quite a while - then go try others.
At the point you make many edits and want to try another configuration
you're going to end up rebuilding most (if not all) of the tree - in
addition you're probably going to trigger a re-configure.

Also - with CI this is less important - work in one configuration... let CI
handle the others.  Only build a "non-standard" configuration when CI shows
that you've botched it.


> If out-of-tree-builds are a requirement: I would like them off by default.
>>
>
> I strongly disagree. In fact, I do not understand what the problem is here
> regarding in vs. out-of-source builds. I just did an in-source build of
> libMesh master and it worked fine (I recall we made sure that this worked
> for you guys). If you're having to do any maintenance for in-source builds
> I don't know what because it's not on the libMesh side.
>

It "kinda" works - I have to maintain my own "install" script that gets
broken every now and again.  Also: the pathing is broken on compile errors
in this mode so that Emacs can't figure out where the source files are.


> In fact, other that your original complaint about adding the file to the
> build system (for which John created a wiki page so we can all remember),
> I've not seen any other actual issue other that just being unhappy with
> Autotools.
>

Literally every time I have to do development in libMesh I run into the
friction.  The fact that we have to have a wiki page about how to add a new
file just shows that it's broken.


> Positives:
>
> 1. Out-of-source builds just work. In-source builds work. You can do
> either. And this requires no maintenance for files compiled into the
> library (there is a little work for new examples, but it's mostly
> automated).
> 2. make install just works. For both in-source and out-of-source builds.
> 3. make check just works. For both in-source and out-of-source builds.
> 4. make dist just works for preparing release tar balls.
> 5. make distcheck just works for checking said release tar balls will
> actually configure/make/make check/make install.
> 6. It will work on any system that supports make and a shell script
> (configure).
>
> 1,2,3,4,5,6 are vital, non-negotiable features for me.
>
> Negatives:
>
> 1. The automation of adding a new file should be more straight forward
> (which I think it would be if we just did away with the symlinks, but I've
> forgotten why we needed that in the first place, so maybe we can't).
> 2. ... What else?
>

-1. Rebuilding/linking of all the small executables every time
-2. In editor building broken (in multiple ways)
-3. Pain to add any single new file (especially headers)
-4. Thousands of Makefiles
-4b. Makefiles in include directories cause in-editor building to get hung
up (I have to switch to a .C file and build from there)
-5. Creating new examples is a HUGE pain
-6. Trying to use an example to try out a new feature (or editing an
existing example) also sucks.  You have to use pretty funky command-lines
just to get a single executable re-run.

Re: [Libmesh-devel] Still Really Dislike Automake

[Paul T. B. <ptb...@gm...>]

On Thu, Aug 30, 2018 at 3:04 PM Paul T. Bauman <ptb...@gm...> wrote:

>
>
> On Thu, Aug 30, 2018 at 3:02 PM Derek Gaston <fri...@gm...> wrote:
>
>> In general you guys are talking about "features" though: when in my
>> estimation the "core" capability of the development cycle is broken.  I
>> don't think that maintaining "nice" features is worth it at the cost of
>> hurting the main development cycle
>> (build->fix->build->fix->add_file->build->fix->test).
>>
>
I should've added here, this is entirely how our build system currently
works. I've had a build tree. I make a modification in the source. I hit
`make` in the build tree. I edit the source. I hit `make` in the build
tree. I hit make install. (With install -C so GRINS only rebuilds what's
necessary).


>
>> I would rather fix the core development cycle - then backfill features
>> based on priority (install > check > dist > out-of-tree, etc.).  We
>> completely chucked a sane development flow for the sake of a few features
>> that are rarely actually used.
>>
>
> "Rarely used" is totally and completely wrong. Every single one of those
> is a core feature of the build system and a vital part of my entire groups
> workflow.
>
>
>> As for symlinking: that's done so that we can maintain code organization
>> - and allow for the "libmesh/" prefix for #includes (namespacing the header
>> file names essentially).
>>
>
> So why don't just `git mv` them and be done with it?
>
>
>>
>> Derek
>>
>> On Thu, Aug 30, 2018 at 2:50 PM Paul T. Bauman <ptb...@gm...>
>> wrote:
>>
>>> This all got started while I was lecturing for my two undergrad classes,
>>> so I'm going to piggy back on Roy's response to start and then reply to
>>> some others since history was getting trimmed in some replies.
>>>
>>> On Thu, Aug 30, 2018 at 12:28 PM Roy Stogner <roy...@ic...>
>>> wrote:
>>>
>>>>
>>>> On Thu, 30 Aug 2018, Derek Gaston wrote:
>>>>
>>>> > After all of these years: I still dislike the Automake build system
>>>> > in libMesh.
>>>>
>>>> So do I.
>>>>
>>>
>>> All build systems suck. As Ben pointed out, one advantage with autotools
>>> is not having to build the build system. (CMake, SCons, etc.)
>>>
>>>
>>>> > I still can't believe that we threw out a perfectly
>>>> > decent build system and saddled ourselves with this thing.
>>>>
>>>
>>> Decent, but feature incomplete.
>>>
>>>
>>>> > In hindsight: do you guys TRULY think it was worth it?
>>>>
>>>> For install/dist/distcheck targets, and out-of-source builds?  Yeah.
>>>>
>>>
>>> I'll add 'check' target as well, but emphatically yes on all counts.
>>> Especially check, install, and out-of-source builds.
>>>
>>>
>>>> > All I'm trying to do right now is add ONE damn header file - and I
>>>> > can't seem to find the magic script / sauce to get everything
>>>> > updated.  I ran include/rebuild_include_HEADERS.sh ... nope.  I ran
>>>> > bootstrap... still nope.  Oh!  Now I remember... I need to run
>>>> > include/libmesh/rebuild_makefile.sh... NOPE!
>>>>
>>>
>>> https://github.com/libMesh/libmesh/wiki/Adding-or-removing-source-files
>>>
>>> Should we put one magic add_files.sh at the top level?
>>>>
>>>> But the two include shell scripts should have done it... that's always
>>>> worked for me.  I guess I usually manually bootstrap afterwards too,
>>>> but in my experience automake is smart enough to rerun itself if it
>>>> sees Makefile.am newer than Makefile.in.
>>>>
>>>
>>> But I do agree this seems like a needless hassle to symlink the header
>>> files. Were we just trying to preserve some transition between svn and git
>>> or something? Why not just `git mv` the headers and get rid of the need for
>>> symlinking. I'm sure I'm missing something, but I don't remember what it is.
>>>
>>>
>>>> > Ok - 100% serious here.  If I redo the libMesh build system so that
>>>> > it's autoconf and make based again - would you guys consider
>>>> > switching to it?
>>>>
>>>> Consider?  Yes.  Anticipate agreeing to with such a high probability
>>>> that the work would be worth your time?  No.
>>>>
>>>
>>> Pretty much this.  I don't think it's worth the time because none of the
>>> features we have with automake I'm willing to give up.
>>>
>>>
>>>> Except... if you're going full Focus on creating a new build system,
>>>> presumably you'd want to put it in MOOSE too...
>>>>
>>>> And I'd really love to be able to do out-of-source builds in MOOSE
>>>> too.  If "autoderek" handled that (plus "make install"; I don't care
>>>> as much about dist/distcheck)
>>>
>>>
>>> I disagree, I think dist and distcheck are important. Also `make check`.
>>>
>>>
>>>> it would be totally worth the switch
>>>> from my point of view.  You'd still have to convince others though.
>>>>
>>>> (It's not really graft if I let my decisions get swayed for code
>>>> rather than money, right? ...)
>>>> ---
>>>>
>>>
>>>> Roy------------------------------------------------------------------------------
>>>
>>>
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>> _______________________________________________
>>>> Libmesh-devel mailing list
>>>> Lib...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>>>>
>>>

Re: [Libmesh-devel] Still Really Dislike Automake

[Paul T. B. <ptb...@gm...>]

On Thu, Aug 30, 2018 at 3:02 PM Derek Gaston <fri...@gm...> wrote:

> In general you guys are talking about "features" though: when in my
> estimation the "core" capability of the development cycle is broken.  I
> don't think that maintaining "nice" features is worth it at the cost of
> hurting the main development cycle
> (build->fix->build->fix->add_file->build->fix->test).
>
> I would rather fix the core development cycle - then backfill features
> based on priority (install > check > dist > out-of-tree, etc.).  We
> completely chucked a sane development flow for the sake of a few features
> that are rarely actually used.
>

"Rarely used" is totally and completely wrong. Every single one of those is
a core feature of the build system and a vital part of my entire groups
workflow.


> As for symlinking: that's done so that we can maintain code organization -
> and allow for the "libmesh/" prefix for #includes (namespacing the header
> file names essentially).
>

So why don't just `git mv` them and be done with it?


>
> Derek
>
> On Thu, Aug 30, 2018 at 2:50 PM Paul T. Bauman <ptb...@gm...> wrote:
>
>> This all got started while I was lecturing for my two undergrad classes,
>> so I'm going to piggy back on Roy's response to start and then reply to
>> some others since history was getting trimmed in some replies.
>>
>> On Thu, Aug 30, 2018 at 12:28 PM Roy Stogner <roy...@ic...>
>> wrote:
>>
>>>
>>> On Thu, 30 Aug 2018, Derek Gaston wrote:
>>>
>>> > After all of these years: I still dislike the Automake build system
>>> > in libMesh.
>>>
>>> So do I.
>>>
>>
>> All build systems suck. As Ben pointed out, one advantage with autotools
>> is not having to build the build system. (CMake, SCons, etc.)
>>
>>
>>> > I still can't believe that we threw out a perfectly
>>> > decent build system and saddled ourselves with this thing.
>>>
>>
>> Decent, but feature incomplete.
>>
>>
>>> > In hindsight: do you guys TRULY think it was worth it?
>>>
>>> For install/dist/distcheck targets, and out-of-source builds?  Yeah.
>>>
>>
>> I'll add 'check' target as well, but emphatically yes on all counts.
>> Especially check, install, and out-of-source builds.
>>
>>
>>> > All I'm trying to do right now is add ONE damn header file - and I
>>> > can't seem to find the magic script / sauce to get everything
>>> > updated.  I ran include/rebuild_include_HEADERS.sh ... nope.  I ran
>>> > bootstrap... still nope.  Oh!  Now I remember... I need to run
>>> > include/libmesh/rebuild_makefile.sh... NOPE!
>>>
>>
>> https://github.com/libMesh/libmesh/wiki/Adding-or-removing-source-files
>>
>> Should we put one magic add_files.sh at the top level?
>>>
>>> But the two include shell scripts should have done it... that's always
>>> worked for me.  I guess I usually manually bootstrap afterwards too,
>>> but in my experience automake is smart enough to rerun itself if it
>>> sees Makefile.am newer than Makefile.in.
>>>
>>
>> But I do agree this seems like a needless hassle to symlink the header
>> files. Were we just trying to preserve some transition between svn and git
>> or something? Why not just `git mv` the headers and get rid of the need for
>> symlinking. I'm sure I'm missing something, but I don't remember what it is.
>>
>>
>>> > Ok - 100% serious here.  If I redo the libMesh build system so that
>>> > it's autoconf and make based again - would you guys consider
>>> > switching to it?
>>>
>>> Consider?  Yes.  Anticipate agreeing to with such a high probability
>>> that the work would be worth your time?  No.
>>>
>>
>> Pretty much this.  I don't think it's worth the time because none of the
>> features we have with automake I'm willing to give up.
>>
>>
>>> Except... if you're going full Focus on creating a new build system,
>>> presumably you'd want to put it in MOOSE too...
>>>
>>> And I'd really love to be able to do out-of-source builds in MOOSE
>>> too.  If "autoderek" handled that (plus "make install"; I don't care
>>> as much about dist/distcheck)
>>
>>
>> I disagree, I think dist and distcheck are important. Also `make check`.
>>
>>
>>> it would be totally worth the switch
>>> from my point of view.  You'd still have to convince others though.
>>>
>>> (It's not really graft if I let my decisions get swayed for code
>>> rather than money, right? ...)
>>> ---
>>>
>>
>>> Roy------------------------------------------------------------------------------
>>
>>
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> Libmesh-devel mailing list
>>> Lib...@li...
>>> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>>>
>>

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

In general you guys are talking about "features" though: when in my
estimation the "core" capability of the development cycle is broken.  I
don't think that maintaining "nice" features is worth it at the cost of
hurting the main development cycle
(build->fix->build->fix->add_file->build->fix->test).

I would rather fix the core development cycle - then backfill features
based on priority (install > check > dist > out-of-tree, etc.).  We
completely chucked a sane development flow for the sake of a few features
that are rarely actually used.

As for symlinking: that's done so that we can maintain code organization -
and allow for the "libmesh/" prefix for #includes (namespacing the header
file names essentially).

Derek

On Thu, Aug 30, 2018 at 2:50 PM Paul T. Bauman <ptb...@gm...> wrote:

> This all got started while I was lecturing for my two undergrad classes,
> so I'm going to piggy back on Roy's response to start and then reply to
> some others since history was getting trimmed in some replies.
>
> On Thu, Aug 30, 2018 at 12:28 PM Roy Stogner <roy...@ic...>
> wrote:
>
>>
>> On Thu, 30 Aug 2018, Derek Gaston wrote:
>>
>> > After all of these years: I still dislike the Automake build system
>> > in libMesh.
>>
>> So do I.
>>
>
> All build systems suck. As Ben pointed out, one advantage with autotools
> is not having to build the build system. (CMake, SCons, etc.)
>
>
>> > I still can't believe that we threw out a perfectly
>> > decent build system and saddled ourselves with this thing.
>>
>
> Decent, but feature incomplete.
>
>
>> > In hindsight: do you guys TRULY think it was worth it?
>>
>> For install/dist/distcheck targets, and out-of-source builds?  Yeah.
>>
>
> I'll add 'check' target as well, but emphatically yes on all counts.
> Especially check, install, and out-of-source builds.
>
>
>> > All I'm trying to do right now is add ONE damn header file - and I
>> > can't seem to find the magic script / sauce to get everything
>> > updated.  I ran include/rebuild_include_HEADERS.sh ... nope.  I ran
>> > bootstrap... still nope.  Oh!  Now I remember... I need to run
>> > include/libmesh/rebuild_makefile.sh... NOPE!
>>
>
> https://github.com/libMesh/libmesh/wiki/Adding-or-removing-source-files
>
> Should we put one magic add_files.sh at the top level?
>>
>> But the two include shell scripts should have done it... that's always
>> worked for me.  I guess I usually manually bootstrap afterwards too,
>> but in my experience automake is smart enough to rerun itself if it
>> sees Makefile.am newer than Makefile.in.
>>
>
> But I do agree this seems like a needless hassle to symlink the header
> files. Were we just trying to preserve some transition between svn and git
> or something? Why not just `git mv` the headers and get rid of the need for
> symlinking. I'm sure I'm missing something, but I don't remember what it is.
>
>
>> > Ok - 100% serious here.  If I redo the libMesh build system so that
>> > it's autoconf and make based again - would you guys consider
>> > switching to it?
>>
>> Consider?  Yes.  Anticipate agreeing to with such a high probability
>> that the work would be worth your time?  No.
>>
>
> Pretty much this.  I don't think it's worth the time because none of the
> features we have with automake I'm willing to give up.
>
>
>> Except... if you're going full Focus on creating a new build system,
>> presumably you'd want to put it in MOOSE too...
>>
>> And I'd really love to be able to do out-of-source builds in MOOSE
>> too.  If "autoderek" handled that (plus "make install"; I don't care
>> as much about dist/distcheck)
>
>
> I disagree, I think dist and distcheck are important. Also `make check`.
>
>
>> it would be totally worth the switch
>> from my point of view.  You'd still have to convince others though.
>>
>> (It's not really graft if I let my decisions get swayed for code
>> rather than money, right? ...)
>> ---
>>
>
>> Roy------------------------------------------------------------------------------
>
>
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Libmesh-devel mailing list
>> Lib...@li...
>> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>>
>

Re: [Libmesh-devel] Still Really Dislike Automake

[Paul T. B. <ptb...@gm...>]

On Thu, Aug 30, 2018 at 1:32 PM Derek Gaston <fri...@gm...> wrote:

> On Thu, Aug 30, 2018 at 11:54 AM Roy Stogner <roy...@ic...>
> wrote:
>
>>
>> On Thu, 30 Aug 2018, Derek Gaston wrote:
>>
>> Should we put one magic add_files.sh at the top level?
>>
>
> I guess?  I still don't understand why these are even necessary - there
> should just be a build rule for the symlinks!
>
>
>> Except... if you're going full Focus on creating a new build system,
>> presumably you'd want to put it in MOOSE too...
>>
>
> It already _is_ in MOOSE.  Our build system is pretty awesome.  Now:
> libMesh doesn't need that level of complexity (we have added complexity for
> the cascading build capability that doesn't need to be in libMesh) - but
> the basic ideas are already there.  We even have stuff that I have no idea
> how you would do with Automake like Unity Builds.
>
>
>> And I'd really love to be able to do out-of-source builds in MOOSE
>> too.  If "autoderek" handled that (plus "make install"; I don't care
>> as much about dist/distcheck) it would be totally worth the switch
>> from my point of view.  You'd still have to convince others though.
>>
>
> This is where we start to have a problem because I just seriously don't
> see compelling enough reasons for out of tree builds.  It adds a layer of
> complexity that is annoying - and it's not really compatible with any
> "in-editor" building (like using Emacs) making it pretty not useful for
> development.
>

I strongly disagree with this statement. It is *vital* for development. On
my CIVET server I have multiple builds with different
options/configurations and I would have to reconfigure and rebuild the
entire bloody tree for one commit change instead of going to the build
directory and hitting `make` and only building the files and dependencies
for each configuration. It is a *HUGE* time saver. Let alone on my own box.


> On the other end, users don't need it at all because they are usually just
> happy to get something to build for _one_ environment configuration (and
> many never, ever want to rebuild after that).  I seriously think it's just
> not worth it.
>

*Users* may not care, but it is *vital* for developers. And I'm a
developer, not just a user.


> Now, "install" I can see a need for.  I don't personally care about it -
> but the arguments are convincing enough.  It's something that both
> developers and users might want to use.
>
> In my mind "install" also obviates a lot of the need for out-of-tree
> builds: if you need multiple configurations built - just install them to
> different places.
>

>From a user point of view, sure. From a developer point of view, this is
simply false. I want my different build configurations sitting around so I
only have to rebuild what's necessary for that configuration without having
to reconfigure and rebuild *all* files, not just dependencies for each
configuration.


>
> If out-of-tree-builds are a requirement: I would like them off by default.
>

I strongly disagree. In fact, I do not understand what the problem is here
regarding in vs. out-of-source builds. I just did an in-source build of
libMesh master and it worked fine (I recall we made sure that this worked
for you guys). If you're having to do any maintenance for in-source builds
I don't know what because it's not on the libMesh side.

In fact, other that your original complaint about adding the file to the
build system (for which John created a wiki page so we can all remember),
I've not seen any other actual issue other that just being unhappy with
Autotools.

Positives:

1. Out-of-source builds just work. In-source builds work. You can do
either. And this requires no maintenance for files compiled into the
library (there is a little work for new examples, but it's mostly
automated).
2. make install just works. For both in-source and out-of-source builds.
3. make check just works. For both in-source and out-of-source builds.
4. make dist just works for preparing release tar balls.
5. make distcheck just works for checking said release tar balls will
actually configure/make/make check/make install.
6. It will work on any system that supports make and a shell script
(configure).

1,2,3,4,5,6 are vital, non-negotiable features for me.

Negatives:

1. The automation of adding a new file should be more straight forward
(which I think it would be if we just did away with the symlinks, but I've
forgotten why we needed that in the first place, so maybe we can't).
2. ... What else?

Paul


>
> Derek
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Libmesh-devel mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>

Re: [Libmesh-devel] Still Really Dislike Automake

[Paul T. B. <ptb...@gm...>]

This all got started while I was lecturing for my two undergrad classes, so
I'm going to piggy back on Roy's response to start and then reply to some
others since history was getting trimmed in some replies.

On Thu, Aug 30, 2018 at 12:28 PM Roy Stogner <roy...@ic...>
wrote:

>
> On Thu, 30 Aug 2018, Derek Gaston wrote:
>
> > After all of these years: I still dislike the Automake build system
> > in libMesh.
>
> So do I.
>

All build systems suck. As Ben pointed out, one advantage with autotools is
not having to build the build system. (CMake, SCons, etc.)


> > I still can't believe that we threw out a perfectly
> > decent build system and saddled ourselves with this thing.
>

Decent, but feature incomplete.


> > In hindsight: do you guys TRULY think it was worth it?
>
> For install/dist/distcheck targets, and out-of-source builds?  Yeah.
>

I'll add 'check' target as well, but emphatically yes on all counts.
Especially check, install, and out-of-source builds.


> > All I'm trying to do right now is add ONE damn header file - and I
> > can't seem to find the magic script / sauce to get everything
> > updated.  I ran include/rebuild_include_HEADERS.sh ... nope.  I ran
> > bootstrap... still nope.  Oh!  Now I remember... I need to run
> > include/libmesh/rebuild_makefile.sh... NOPE!
>

https://github.com/libMesh/libmesh/wiki/Adding-or-removing-source-files

Should we put one magic add_files.sh at the top level?
>
> But the two include shell scripts should have done it... that's always
> worked for me.  I guess I usually manually bootstrap afterwards too,
> but in my experience automake is smart enough to rerun itself if it
> sees Makefile.am newer than Makefile.in.
>

But I do agree this seems like a needless hassle to symlink the header
files. Were we just trying to preserve some transition between svn and git
or something? Why not just `git mv` the headers and get rid of the need for
symlinking. I'm sure I'm missing something, but I don't remember what it is.


> > Ok - 100% serious here.  If I redo the libMesh build system so that
> > it's autoconf and make based again - would you guys consider
> > switching to it?
>
> Consider?  Yes.  Anticipate agreeing to with such a high probability
> that the work would be worth your time?  No.
>

Pretty much this.  I don't think it's worth the time because none of the
features we have with automake I'm willing to give up.


> Except... if you're going full Focus on creating a new build system,
> presumably you'd want to put it in MOOSE too...
>
> And I'd really love to be able to do out-of-source builds in MOOSE
> too.  If "autoderek" handled that (plus "make install"; I don't care
> as much about dist/distcheck)


I disagree, I think dist and distcheck are important. Also `make check`.


> it would be totally worth the switch
> from my point of view.  You'd still have to convince others though.
>
> (It's not really graft if I let my decisions get swayed for code
> rather than money, right? ...)
> ---
>
> Roy------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Libmesh-devel mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

BTW: I do realize that supporting both still means that I have to deal with
Automake when modifying libMesh... but I would only have to deal with it at
the _end_ after I'm done working on something.  Not _constantly_ :-)

Derek

On Thu, Aug 30, 2018 at 1:35 PM Derek Gaston <fri...@gm...> wrote:

> I suppose another option is to just support both build systems
> indefinitely.  People who want/need the stuff automake does (out of tree,
> dist, install) can use automake.  Everyone else that just wants to build
> (98%+) can just type "make" and be happy...
>
> Derek
>
> On Thu, Aug 30, 2018 at 1:31 PM Derek Gaston <fri...@gm...> wrote:
>
>> On Thu, Aug 30, 2018 at 11:54 AM Roy Stogner <roy...@ic...>
>> wrote:
>>
>>>
>>> On Thu, 30 Aug 2018, Derek Gaston wrote:
>>>
>>> Should we put one magic add_files.sh at the top level?
>>>
>>
>> I guess?  I still don't understand why these are even necessary - there
>> should just be a build rule for the symlinks!
>>
>>
>>> Except... if you're going full Focus on creating a new build system,
>>> presumably you'd want to put it in MOOSE too...
>>>
>>
>> It already _is_ in MOOSE.  Our build system is pretty awesome.  Now:
>> libMesh doesn't need that level of complexity (we have added complexity for
>> the cascading build capability that doesn't need to be in libMesh) - but
>> the basic ideas are already there.  We even have stuff that I have no idea
>> how you would do with Automake like Unity Builds.
>>
>>
>>> And I'd really love to be able to do out-of-source builds in MOOSE
>>> too.  If "autoderek" handled that (plus "make install"; I don't care
>>> as much about dist/distcheck) it would be totally worth the switch
>>> from my point of view.  You'd still have to convince others though.
>>>
>>
>> This is where we start to have a problem because I just seriously don't
>> see compelling enough reasons for out of tree builds.  It adds a layer of
>> complexity that is annoying - and it's not really compatible with any
>> "in-editor" building (like using Emacs) making it pretty not useful for
>> development.  On the other end, users don't need it at all because they are
>> usually just happy to get something to build for _one_ environment
>> configuration (and many never, ever want to rebuild after that).  I
>> seriously think it's just not worth it.
>>
>> Now, "install" I can see a need for.  I don't personally care about it -
>> but the arguments are convincing enough.  It's something that both
>> developers and users might want to use.
>>
>> In my mind "install" also obviates a lot of the need for out-of-tree
>> builds: if you need multiple configurations built - just install them to
>> different places.
>>
>> If out-of-tree-builds are a requirement: I would like them off by default.
>>
>> Derek
>>
>

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

I suppose another option is to just support both build systems
indefinitely.  People who want/need the stuff automake does (out of tree,
dist, install) can use automake.  Everyone else that just wants to build
(98%+) can just type "make" and be happy...

Derek

On Thu, Aug 30, 2018 at 1:31 PM Derek Gaston <fri...@gm...> wrote:

> On Thu, Aug 30, 2018 at 11:54 AM Roy Stogner <roy...@ic...>
> wrote:
>
>>
>> On Thu, 30 Aug 2018, Derek Gaston wrote:
>>
>> Should we put one magic add_files.sh at the top level?
>>
>
> I guess?  I still don't understand why these are even necessary - there
> should just be a build rule for the symlinks!
>
>
>> Except... if you're going full Focus on creating a new build system,
>> presumably you'd want to put it in MOOSE too...
>>
>
> It already _is_ in MOOSE.  Our build system is pretty awesome.  Now:
> libMesh doesn't need that level of complexity (we have added complexity for
> the cascading build capability that doesn't need to be in libMesh) - but
> the basic ideas are already there.  We even have stuff that I have no idea
> how you would do with Automake like Unity Builds.
>
>
>> And I'd really love to be able to do out-of-source builds in MOOSE
>> too.  If "autoderek" handled that (plus "make install"; I don't care
>> as much about dist/distcheck) it would be totally worth the switch
>> from my point of view.  You'd still have to convince others though.
>>
>
> This is where we start to have a problem because I just seriously don't
> see compelling enough reasons for out of tree builds.  It adds a layer of
> complexity that is annoying - and it's not really compatible with any
> "in-editor" building (like using Emacs) making it pretty not useful for
> development.  On the other end, users don't need it at all because they are
> usually just happy to get something to build for _one_ environment
> configuration (and many never, ever want to rebuild after that).  I
> seriously think it's just not worth it.
>
> Now, "install" I can see a need for.  I don't personally care about it -
> but the arguments are convincing enough.  It's something that both
> developers and users might want to use.
>
> In my mind "install" also obviates a lot of the need for out-of-tree
> builds: if you need multiple configurations built - just install them to
> different places.
>
> If out-of-tree-builds are a requirement: I would like them off by default.
>
> Derek
>

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

On Thu, Aug 30, 2018 at 11:54 AM Roy Stogner <roy...@ic...>
wrote:

>
> On Thu, 30 Aug 2018, Derek Gaston wrote:
>
> Should we put one magic add_files.sh at the top level?
>

I guess?  I still don't understand why these are even necessary - there
should just be a build rule for the symlinks!


> Except... if you're going full Focus on creating a new build system,
> presumably you'd want to put it in MOOSE too...
>

It already _is_ in MOOSE.  Our build system is pretty awesome.  Now:
libMesh doesn't need that level of complexity (we have added complexity for
the cascading build capability that doesn't need to be in libMesh) - but
the basic ideas are already there.  We even have stuff that I have no idea
how you would do with Automake like Unity Builds.


> And I'd really love to be able to do out-of-source builds in MOOSE
> too.  If "autoderek" handled that (plus "make install"; I don't care
> as much about dist/distcheck) it would be totally worth the switch
> from my point of view.  You'd still have to convince others though.
>

This is where we start to have a problem because I just seriously don't see
compelling enough reasons for out of tree builds.  It adds a layer of
complexity that is annoying - and it's not really compatible with any
"in-editor" building (like using Emacs) making it pretty not useful for
development.  On the other end, users don't need it at all because they are
usually just happy to get something to build for _one_ environment
configuration (and many never, ever want to rebuild after that).  I
seriously think it's just not worth it.

Now, "install" I can see a need for.  I don't personally care about it -
but the arguments are convincing enough.  It's something that both
developers and users might want to use.

In my mind "install" also obviates a lot of the need for out-of-tree
builds: if you need multiple configurations built - just install them to
different places.

If out-of-tree-builds are a requirement: I would like them off by default.

Derek

Re: [Libmesh-devel] Still Really Dislike Automake

[Roy S. <roy...@ic...>]

On Thu, 30 Aug 2018, Derek Gaston wrote:

> After all of these years: I still dislike the Automake build system
> in libMesh.

So do I.

> I still can't believe that we threw out a perfectly
> decent build system and saddled ourselves with this thing.

> In hindsight: do you guys TRULY think it was worth it?

For install/dist/distcheck targets, and out-of-source builds?  Yeah.

> All I'm trying to do right now is add ONE damn header file - and I
> can't seem to find the magic script / sauce to get everything
> updated.  I ran include/rebuild_include_HEADERS.sh ... nope.  I ran
> bootstrap... still nope.  Oh!  Now I remember... I need to run
> include/libmesh/rebuild_makefile.sh... NOPE!

Should we put one magic add_files.sh at the top level?

But the two include shell scripts should have done it... that's always
worked for me.  I guess I usually manually bootstrap afterwards too,
but in my experience automake is smart enough to rerun itself if it
sees Makefile.am newer than Makefile.in.

> Ok - 100% serious here.  If I redo the libMesh build system so that
> it's autoconf and make based again - would you guys consider
> switching to it? 

Consider?  Yes.  Anticipate agreeing to with such a high probability
that the work would be worth your time?  No.

Except... if you're going full Focus on creating a new build system,
presumably you'd want to put it in MOOSE too...

And I'd really love to be able to do out-of-source builds in MOOSE
too.  If "autoderek" handled that (plus "make install"; I don't care
as much about dist/distcheck) it would be totally worth the switch
from my point of view.  You'd still have to convince others though.

(It's not really graft if I let my decisions get swayed for code
rather than money, right? ...)
---
Roy

Re: [Libmesh-devel] Still Really Dislike Automake

[Kirk, B. (JSC-EG311) <ben...@na...>]

> On Aug 30, 2018, at 11:09 AM, Roy Stogner <roy...@ic...> wrote:
> 
> Would it be better to try and figure out how to ameliorate the suck in
> automake, though?  I don't mean to start a flame war but "the rebuild
> scripts are too confusing" is a fair criticism and if the list of
> other criticisms isn't too long or too foundational then we might be
> able to reduce the damage.

For sure. Also there’s been some movement in automake and rumors of 2.0, so it’s entirely possible there are better solutions that have been revealed. 

Re: [Libmesh-devel] Still Really Dislike Automake

[Roy S. <roy...@ic...>]

On Thu, 30 Aug 2018, Derek Gaston wrote:

> Maybe.  Certainly the "configure" that we have is really good... and
> that won't change.  It _may_ be possible to make them coexist for a
> bit.

I'd be much more open to experimentation if we can get things to
coexist; I've done that in the past with CMake-vs-autotools (aka
we-have-some-Windows-users vs this-works-better-for-everyone-else) on
another project and it was okay.  I don't know if it would be as
smooth with autotools-vs-autotools_minus_automake or if we'd have name
clash headaches, though.

> Hah - I do agree with having to build build-systems.

There doesn't yet seem to be a build system that doesn't suck, so it's
not *entirely* ridiculous to envision reinventing the wheel here.

Would it be better to try and figure out how to ameliorate the suck in
automake, though?  I don't mean to start a flame war but "the rebuild
scripts are too confusing" is a fair criticism and if the list of
other criticisms isn't too long or too foundational then we might be
able to reduce the damage.
---
Roy

Re: [Libmesh-devel] Still Really Dislike Automake

[Derek G. <fri...@gm...>]

Maybe.  Certainly the "configure" that we have is really good... and that
won't change.  It _may_ be possible to make them coexist for a bit.

Hah - I do agree with having to build build-systems.

At any rate: I won't be working on this until after my dissertation is
complete (this fall)... so we have some time to debate about it / generate
ideas.  I truly think that we can create one static Makefile that doesn't
need to get autogenerated at all (will "include" some things from one that
automake creates though) that can do all the things we need.

Derek

On Thu, Aug 30, 2018 at 11:16 AM Kirk, Benjamin (JSC-EG311) <
ben...@na...> wrote:

> Any chance your modifications could create a build system that can coexist?
>
> What I dislike more than automake is having to spend a good chunk of time
> building the build system on a machine that’s not particularly modern…
>
> -Ben
>
>
>
> > On Aug 30, 2018, at 9:47 AM, Derek Gaston <fri...@gm...> wrote:
> >
> > After all of these years: I still dislike the Automake build system in
> libMesh.  I still can't believe that we threw out a perfectly decent build
> system and saddled ourselves with this thing.
> >
> > In hindsight: do you guys TRULY think it was worth it?
> >
> > I still maintain a set of scripts that allow me to do in-tree builds.
> In addition - I often prototype things in MOOSE / a new MOOSE application
> before moving it down to libMesh (if I ever do - I am also just
> increasingly NOT moving stuff to libMesh!).  All of this because of the
> build system.
> >
> > All I'm trying to do right now is add ONE damn header file - and I can't
> seem to find the magic script / sauce to get everything updated.  I ran
> include/rebuild_include_HEADERS.sh ... nope.  I ran bootstrap... still
> nope.  Oh!  Now I remember... I need to run
> include/libmesh/rebuild_makefile.sh... NOPE!
> >
> > Ok - 100% serious here.  If I redo the libMesh build system so that it's
> autoconf and make based again - would you guys consider switching to it?  I
> am willing to even code up a "make install" (which I believe was the
> original reason for wanting to use Automake).
> >
> > Derek
> >
> > PS - if anyone could tell me what I need to do to get the symlink built
> in include/libmesh for a new header file I would greatly appreciate it!
> >
> ------------------------------------------------------------------------------
> > Check out the vibrant tech community on one of the world's most
> > engaging tech sites, Slashdot.org!
> http://sdm.link/slashdot_______________________________________________
> > Libmesh-devel mailing list
> > Lib...@li...
> > https://lists.sourceforge.net/lists/listinfo/libmesh-devel
>
>