Re: [Sbcl-devel] Batteries-included SBCL

[Nikodemus S. <nik...@ra...>]

2009/12/19  <dhe...@te...>:

I was hoping you'd show up. :)

> IMO, SBCL-s`pecific libs should be included similarly to the current
> contribs.  SVN remotes or git submodules could help with the versioning;

> IMO, portable libraries should not appear in an SBCL-specific bundle.  I
> will hold ASDF as an example of what happens when each implementation
> independently maintains a portable layer (others have mentioned LOOP;
> PCL/CLOS is another).  Actions tend to reflect organizational structure;
> separate maintenance leads to divergence, and that's something avoid.

I don't think this quite applies here. We are explicitly not
interested in maintaining the extra libraries, which is part of the
reason why I think we must talk to upstream, no matter the license.
It's the way to get whatever changes we might need into the upstream,
so that there are no local patches.

ASDF is a case in point. SBCL has always shipped with the upstream
copy of ASDF. (Modulo human errors and lagging a bit behind like now.)

> One goal of LibCL is to encourage portable library development; there will
> be no overlap from my side.  ;)

I can't guarantee there isn't going to be overlap from SBCL side...

Let's talk contents -- not to pick on libcl, just using it as a list of things:

INSTALL.txt		cl-utilities		local-time
LICENSE.txt		cl-vectors		mel-base
alexandria		clem			metabang-bind
anaphora		closer-mop		metatilities
array-operations	closure-common		metatilities-base
asdf			compile-libcl.lisp	misc-extensions
asdf-binary-locations	configure-libcl.lisp	moptilities
asdf-system-connections	cxml			packages.txt
babel			defsystem-compatibility	parse-declarations
bordeaux-threads	dynamic-classes		platforms
cffi			ffa			png-read
ch-image		flexi-streams		puri
ch-util			flexichain		salza2
chipz			fset			skippy
chunga			git			spatial-trees
cl-base64		html-template		stefil
cl-cont			ieee-floats		tinaa
cl-containers		imago			trees
cl-fad			index.html		trivial-features
cl-graph		install.inf		trivial-gray-streams
cl-jpeg			ironclad		trivial-shell
cl-l10n			iterate			vecto
cl-markdown		kmrcl			zlib
cl-numlib		libcl-compat		zpb-ttf
cl-pdf			libcl-init		zpng
cl-ppcre		lml2

This is *way* more than I would like to ship with SBCL:

1) Multiple libraries for the same purpose: Alexandria, cl-utilities,
ch-util, KMRCL, metatilities, and possibly other generic utility
libraries as well. (Also overlap with PNG tools, I think -- maybe
others as well.)

2) Stuff I am unfamiliar with: unless an SBCL hacker puts their hand
up and says that eg. CL-NUMLIB is cool beans, I would not want to ship
with it since I don't know anything about it. I think this is
essential for reasonable handling of bugreports.

3) TRIVIAL-FOO... I would personally prefer not to ship with any of these.

Now, I know that otherwise perfectly fine libraries can end up pulling
in a crapload of dependencies you'd rather not have. To a degree
that's life... but part of the reason for doing the batteries included
thing is to shake the ecosystem up a bit so that we can have less of
these sort of redundancies.

I think something like libcl (or clbuild) makes the correct choice
when erring on the side of completeness, but for SBCL extras I think
we should wear some ideologically guided blinders -- not too narrow,
but some.

(It is also in some ways *less* than I would like to ship with, as
there are some portable libraries on my personal "maybe we should ship
with this?" list that are not there.)

> 1) By releasing a project under an open-source license, you explicitly
> allow others to redistribute your work.

Sure. I still think talking to upstream is essential: software is social.

> 2) By bundling someone else's code, you get to choose which headaches to

No disagreement here.

> 3) Best practice is something like the following.  End-users report bugs
> to the packager; the packager triages these bugs and submits them upstream
> as appropriate, often with patches added.

I think this really works only if a single packager is responsible for
a limited number of packages, and is reasonably familiar with all of
them. Otherwise triage can become too hard and time consuming. (Not
always of course: obvious things remain obvious, but things like
user-errors become harder to separate out when you are unfamiliar with
the software.)

I strongly believe it is most time-efficient to get the upstream eyes
on bugreports ASAP. As long as the packager doesn't introduce any
local changes to the code, pretty much all bugs reported are upstream
bugs. More eyes are better, but...

>> * License: libcl doesn't guarantee MIT/BSD compatibility right now,
>> and I think we want that?
>
> Technically, its up to the end-user to pick which libraries they use...
> It is a stated goal of LibCL to get people moving towards a common
> license; but LibCL is dealing with other issues right now.

I think this is perfectly understandable.

However, I do want to be able to tell users that if they install
SBCL+Batteries, they are 100% under BSD/MIT. I don't want to even
imagine the issues that some of our commercial users could run into
because an intern accidentally pulled in a bundled extra that was
GPL'd -- which updating SBCL ninjaed into a supposedly GPL-free
environment. "Ouch"

>> [ While from our users perspective the difference between libcl and
>> SBCL extras may be minimal, I think from our perspective there are
>> several points where goals and constraints diverge somewhat. ]
>
> And this is why I think both types of system server a purpose.  LibCL is
> no nirvana -- feel free to build something better -- but we need some
> (implementation-independent) community to maintain portable libraries.

\o/

>> * I would be nice if APROPOS (or equivalent) knew about the contents
>> of these libraries, so that people could figure which library to look
>> at using an interactive tool instead of a webpage.
>
> I've already talked to Alastair about the possibility of stripping debug
> info from fasls and optionally loading it later.  This sounds like the
> reverse; load some debug info without the code...

I was thinking of something even more trivial: after building
everything load everything, and generate a simple binary-searchable DB
file that looks something like this:

 FOO:*QUUX* BOUNDP
 FOO:BAR FBOUNDP

which an apropos-like tool could read without having to load stuff
into the core.

Cheers,

 -- Nikodemus