On Wed, Nov 21, 2012 at 9:53 PM, Kirk, Benjamin (JSC-EG311) <benjamin.kirk-1@nasa.gov> wrote:
On Nov 21, 2012, at 7:21 PM, Derek Gaston <friedmud@gmail.com> wrote:
Thanks for the vote of confidence, and I apologize for getting short with you over this…

No problem at all.  God knows I've gotten short with enough people on mailing lists ;-)
My goal would be to have libMesh integrated into the popular package managers, and that many users would be able to install the code that way.  And the old build system was not getting us there.  One major motivation for the new build system is to enable packaging in a standard way.

I suspected that this was the motivation.  I don't think it is a bad motivation, but I'm not sure it's the right path either.  libMesh is fundamentally different from Firefox or Gimp... you don't just "use it"... you create applications with it.  In that application creation process libMesh gets completely intertwined in the application.  There is no separation.  This changes the game.  It means that small changes in libMesh have big impact on the applications.

BTW - This is one of the reasons why the word "Framework" is a dirty word in a lot of scientific computing circles these days.  A lot of scientists have completely written off the idea of a framework... saying that it is simply infeasible.  At Sandia they have completely changed to "toolkits"... functional pieces that are loosely coupled to your application and are interchangeable.  Every piece of the toolkit talks through agnostic interfaces so that no piece is sacred and applications are, more or less, insulated from changes to the functional pieces.  This has also caused very real issues that need to be considered like interface bloat and requiring quite a bit of "glue" code in the applications.  There is no silver bullet here.

If you read some of those papers I linked to from Ross you'll see that the relationship between the "scientific framework" and the application fundamentally changes the way software needs to be developed.  The higher degree of "coupling" (for lack of a better term) means that a tighter development cycle is required (ie, iterations between application and framework need to be fast).
Do you point your users to gcc/trunk, or petsc-dev?  As libMesh matures, why would it be different?

Firstly, yes, we are starting to point our users at petsc-dev.  We're starting to have a faster cycle with the PETSc guys because we're starting to more directly incorporate their "framework" bits.  PETSc is both a "toolkit" and a "framework".  You can choose to use it either way.

Toolkit bits include the solvers... you just fill their data structures and let it solve.  It has very little impact on the overall structure of your application and the interface bits are small.

Framework bits include their Mesh datastructures... and (to a lesser extent) their time integrators.  If you adopt these PETSc stuff starts to show up in a lot more of your code.  If you only use PETSc releases... ignoring "dev" you are in for a whole lot of pain when you go to update PETSc because of how much impact it's going to have on your application.

We can now (optionally at this point) use PETSc-TS... the time integration package in PETSc... and because of some of the changes necessary to PETSc in order to enable that capability in MOOSE we now have a petsc-dev module in our standard install package... complete with an "update" script that grabs a new version of petsc-dev.  Most of our users are not using this new time integration capability yet... so they are still using a PETSc release.  However, going forward that is most likely going to change and our users will get a "vetted" version of petsc-dev with MOOSE, just like they now get libMesh.

Back to Debian packages.  I don't personally think that we should be encouraging users to tie themselves to a particular version of libMesh that comes down through a package manager.  It does not work like Gimp.  Their package manager is going to happily "update" libMesh to 0.9.0 one day... and all of their stuff is going to break.  Not only is it going to break, but since they haven't kept up with development they are REALLY far off from working with 0.9.0... causing them to spend quite a bit of time integrating all the new changes.

There are two other options in this situation:

1.  They never update to a newer version.  This sucks because they don't get any of the benefit of the ongoing development.  Also, if they make changes to libMesh we won't get those back....

2.  libMesh must be a slave to backwards compatibility.  Backwards compatibility is always a good idea... when possible... or when it makes sense.  For a library to move forward sometimes things have to change (although maybe not as much as the PETSc guys like to change it.. ;-).  To be a slave to backwards compatibility is to stagnate the library...

I completely understand the draw to want libMesh as a package.  I mean, what open source developer doesn't dream of seeing their project in the main linux distro package lists?  It has a certain amount of validation to it.  That said, I don't believe that the nature of libMesh lends itself to that method of distribution.  It's going to cause more problems for our users and more problems for libmesh developers (how many times have we seen someone mail the list about a problem and the first thing we tell them is.... use the SVN version to get the fix!).

Thanks for having an open dialog about this... let's continue that going forward.  In the meantime we'll continue to try to integrate the new build system and work with you guys to smooth out any rough edges we hit...