On Wed, Jan 09, 2008 at 10:03:38PM +0200, Nikodemus Siivola wrote:
> I like our monthly release policy, but I think we're kidding ourselves
> if we pretend they are tested even remotely solidly. More then a random
> revision mid-month, sure, but I don't think as many users as we hope
> put SBCL put through its paces during the freeze period -- and we don't
> have a solid list of things to shake at it ourselves either before we call
> it good.
I basically agree that we shouldn't kid ourselves about how heavily
tested the releases are. I have only a small cynical disagreement: I
suspect that this is not so much a characteristic problem of frequent
releases, because my impression is that the phenomenon tends to
persist pretty strongly even with less frequent releases.
> Monthly release cycle is also extremely rapid (I think) for people for whom
> it means updating applications to when there are incompatible changes, and
> even if there is nothing to fix, just testing a new SBCL for use with their
> applications every month is probably a bit much a sizable percentage of our
> users (users, please comment on this!)
I agree that this is an extremely rapid cycle from most end users'
points of view, but I'm not sure that that's a problem. My guess is
that a large fraction of SBCL users probably cope with it by upgrading
only on some chosen fraction of releases. That's certainly what I do
with most of the software I use, and it seems to be what most other
people do with most of the software they use. It would become a nasty
problem if have flaky noncommutative changes (like where you want to
upgrade from 1.2.3 to 1.2.8 for a key bugfix, but unfortunately 1.2.5
introduced a showstopper-for-you bug which hasn't been fixed) but as
far as I know such noncommutativity is present for only a very small
fraction of people's problems with SBCL.
> I'm thinking that maybe we should timebox the second digit of our version
> number as well. This seems topical to me since I think there has been
> enough time and features since 1.0 to call in 1.1...
> I'm not sure what period would be good. Maybe every 4-8, or 5-9 months?
> What I would like to see would be a "real beta period":
> * Freeze 1.1.beta.1. Announce it.
> * Let people test it for a month or so. If there are no regressions
> or major complaints, call it 1.1.0. During this time bleeding edge
> development proceeds normally.
> * If there ("if", hah!) is anything worth fixing, make a new beta:
> 1.1.beta.2 -- either with backported fixes, or take the next monthly,
> whichever seems like a better bet at the time.
> * Repeat as necessary -- hopefully finishing the process in less then
> 4 iterations...
> How does this sound? Would the better testing be worth the extra hassle?
I honestly don't know. It's not a silly idea --- many projects have
done it for many years, and they've managed to avoid producing a
stream of embarrassing little revelations that the emperor has no
clothes and the upside is negligible. And I'm not even sure the signal
of extra hassle can easily be distinguished from the usual hassle
On the other hand, the usual hassle noise level is high enough that it
could conceal a fairly large signal, and I generally tend to weight
hassle fairly heavily.
Also, one thing that makes me unenthusiastic about direct
participation in an especially-reliable beta process is that to the
extent that the project is useful relative to the main development
line, it seems that it must involve a stream of serious judgment calls
about how much change to include in the solid version. Of course, all
software development involves lots of judgment calls about what should
be included. However, the subspecialty of judgment calls in
maintaining an especially-reliable branch is one where I don't have
much confidence. My tentative guess is that it's some combination of
me being relatively weak in this area and this area being more
absolutely difficult: my impression is that the population of people
who can maintain a working consensus over what should happen in an
especially-reliable branch is smaller than the population who can
maintain a working consensus on what should happen in ordinary
One possibility (which I seem to remember I've mentioned on the list
before, but which I'm not sure how to search for conveniently) is that
such a solid beta release process could be largely decoupled from the
main development and release process. As long as it is even minimally
culturally compatible with the main branch, a solid beta branch could
share space on key web pages (like SourceForge File Releases), and I'm
not sure it would need much else. And besides the usual MORE WNEWMAN
"COULDN'T THIS BE DECOUPLED?" CHANT reflex, doing it decoupled might
have actual advantages:
* It's a commonplace among people serious about reliability and quality
(not just in software) that it can be important to have testing and
reliability judgments made by people not too directly invested in
* If I'm right that it's harder to get a single technical consensus
for reliability tradeoffs than for generally developing the best
we can, then being decoupled could have an advantage of letting the
people who are primarily interested in the ordinary development go
on with what their business almost no matter what. (There are limits:
if it gets to the point where competing reliability branch teams win
bakeoffs by sabotaging each other's AC power, we'll probably be
> Just to clarify, I'm more interested in a solid beta period for every 1.X
> release then I am in timeboxing the same -- but I think there would be
> benefits in timeboxing the period last 1.X release and the start of beta
> period for next.
As to when carries should be propagated between subfields of version
numbers, version numbering is another art where I doubt my own ability
and where I am skeptical about anyone's ability to reliably build
consensus. It's just lucky that it seldom looms large enough to
actually break groups into bitterly feuding factions. Currently I'm
thinking of trying to avoid having sub-version numbers much exceed 10,
and thus releasing 1.1.0 soonish, but this plan is very weakly held,
so if I'm wrong and there *is* a consensus on how we should be
numbering releases, I'll probably be happy to go along.
William Harold Newman <william.newman@...>
PGP key fingerprint 85 CE 1C BA 79 8D 51 8C B9 25 FB EE E0 C3 E5 7C
a mudball of power, not a razor of willfull minimalism ;-)