[Mupad-combinat-users] MuPAD-Combinat future and SAGE

[Nicolas M. T. <Nic...@u-...>]

	Dear MuPAD-Combinat community,

This e-mail starts an essential discussion for the future of MuPAD-Combinat: 

			Should we switch to SAGE?

Please read carefully, and get involved!

SAGE is a completely open source mathematical system which was
launched around 2004 by William Stein. It's based on a general purpose
language (python), and readily has a large range of functionality
thanks to the integration of existing components (GMP, GAP, singular,
linbox, R, just to name a few):

			http://www.sagemath.org/

One year ago, we got contacted by Mike Hansen. Since then, he has
ported 30k lines of code from MuPAD-Combinat into SAGE. With Fran�ois,
Jason, and Anne, we were invited at the SAGE days 7 last week in Los
Angeles (see http://mupad-combinat.sourceforge.net/#sage_days_7 for my
presentation slides). Our goal was to play hard with the system and
discuss seriously with the SAGE developers to evaluate how good an
alternative SAGE would be. This is a report about it.

In short, I have been amazed by the dynamism of the growing SAGE
community. Furthermore, our experiments (partial reimplementation of
our crystal code) seem to indicate that there would be no major
technical issues to port our code: most of the idioms and paradigms we
use have some natural equivalent. On the other hand, porting 7 years
worth of work in MuPAD, and more importantly migrating the community
will be a lot of work and worst may cause the loss of users and developers.

Now is the time to take a decision:

 - Should we switch to SAGE?
 - If yes, what should be the time line?

The answer is far from clear to me, and anyway this is really a
decision that we have to take all together. There are lots of pros and
cons, which I have tried to summaries below.

Your feedback is essential. Please answer by e-mail, or edit our Wiki:

	http://mupad-combinat.sourceforge.net/Wiki/Sage

 - Would you be ready to invest some time into learning a new system?
 - Would you be ready to invest some time into the porting itself?
 - What are the features that you would need to be ported before you switched?
 - What are, for you, the prominent pros and cons of both platforms?
 - Altogether, do you think we should switch?

Independently of when the switch would occur, here is my feeling about
the time line:

 - We should leave MuPAD-Combinat in a good state. If not just so that
   the MuPAD people can reuse whatever they see fit in there. We owe
   them this for all the support they provided us.

 - We don't necessarily need to switch all at the same time. But none
   of us want to have one foot in each system, even if the sage-MuPAD
   interface may level things a bit.

 - As Anne suggested, we have an opportunity now, since Jason, Mike,
   Anne and myself are altogether at MSRI, and Florent,
   Jean-Christophe and Fran�ois will visit us at some point.

------------------------------------------------------------------------------

Here is a summary of pros and cons (please edit!)

Development model and community:

+ Completely open source
+ Free (as in free beer)
+ (Hyper)Active fast growing community, with members in all areas of mathematics
+ Regular SAGE days meetings (3 more planned for 2008, including one in France)
+ Excellent community organization (see sagetrac.org)
+ A new release comes out quite often (about every two weeks)
+ Releases are managed by a release manager which is paid for this,
  and takes administrative burden away (think running tests, making
  sure everything builds on all platforms, ...)
+ The quality of the code is enforced by systematic peer-reviews
  before integration into SAGE itself. This may motivate people to
  submit more stable/correct/easy-to-read/tested/documented code.
+ Mercurial distributed version control (better than subversion when
  it comes to have several lines of development in parallel).
- A Windows version is not yet fully available!

What about the long run? Will they manage to continue grow at the same
speed? Will the NSF/other funds follow?

Documentation:

+ easier to write
+ close to the code (written in the source file, within each function)
+ the doc does compile easily on all platforms
- Not as structured as in MuPAD (latex instead of XML)
  The platform by itself wont impose high quality standards
- syntactic tests instead of semantic tests
  needs to update the results if the pretty printing changes

Tests:

+ close to the code (as the doc)

Technical level:

+ Python has a huge community. In particular:
  + The interpreter, debugger, ... have been highly optimized and debugged
  + There are thousands of general purpose python libraries
+ Continuations (yield) which makes it much easier to write
  generators. However: it is not sure that you can do copies of those
  generators and we use this intensively (see generators::subset)
+ parent/element: sets are objects like the others
  (see the discussion about posets last June)
+ Cython compiler: low level critical sections in python can be
  compiled without rewrite into C (up to some restrictions). It can be
  further optimized by using C-types instead of python types (think
  integer lists lex tools).
+ much easier integration of external C/C++ code
  (can drop MAPITL)
- some issues with multiple inheritance
- no standard mantra for input and output type checking
- the coercion model may need some adaptation


To do list (Mike, please update!):

 - Basic combinatorial classes      [ 75%]
 - Decomposable objects             [    ] (needs complete rewrite anyway)
 - Symmetric functions              [ 25%]
 - k-Schur & the like               [  0%]
 - Root systems / ...               [ 10%]
 - Crystals                         [  5%]
 - Free modules & such              [  0%]
 - Algebra (desosseur, ...)         [  0%]
 - Operads                          [  0%]
 - Linbox interface                 [100%] (compares to 10% in MuPAD)
 - GAP interface                    [100%] (compares to  1% in MuPAD)
 - Interface for fast Gr�bner basis [100%] (compares to  0% in MuPAD)
 - Nauty                            [100%]
 - Symmetrica                       [ 60%]
 - GLIP                                    Deprecated by Robert's generic code
 - dot2tex                          [ 10%] Delegated to the SAGE graph theorists
 - Database access                  [100%] 
 - MachineIntegerListsLex           [  0%] Will be easy via cython
 - Rigged configurations            [  0%]

-- 
Nicolas M. Thiery "Isil" <nt...@us...>
http://Nicolas.Thiery.name/