Re: [Sbcl-devel] development and 0.7.x

[William H. N. <wil...@ai...>]

On Fri, Aug 03, 2001 at 03:39:19PM +0100, Daniel Barlow wrote:
> I hope your Go went well. 

Thanks for the kind wishes. Alas, it didn't actually go so well -- my
program lost all its games. Many of the interesting and important
parts of my program are still much too slow to play in competition
time, and what's left is terribly weak. Hopefully that will be
changing fairly soon, and when it does, the Computer Go Ladder
<http://www.cgl.ucsf.edu/go/ladder.html> will let me try it out even
without a face-to-face tournament.

> What kind of timeframes are there for a 0.7 release?  My orbit is
> gradually spinning back round to the position that I can do some more
> SBCL work: there are a few Alpha infelicities that need fixing, the
> posix-environ thing if it's not already done, and then I want to start
> thinking about MP stuff.  To be honest I'd rather start with the last
> one, but if 0.7 release engineering stuff is planned during the next
> few weeks, the other bits had better come first.  Hence the question.
> 
> I believe that 0.7 will be the first actual release with Alpha support...

I plan to put out 0.6.13 any time now, so that should be the first
release with Alpha support. After that my plan was to start doing the
various incompatible or large changes that I've talked about before,
and put some large fraction of them into 0.7.0.

I've fixed all the bugs on my old gotta-do-before-0.7.0 list, and the
only things pending before I can release 0.6.13 are the DEFKNOWN
weirdness that I wrote about in the last CVS commit message, and MNA's
INSPECT patch. And since a test compile of the DEFKNOWN-tidied version
finished successfully while I was working on this message, the
DEFKNOWN stuff shouldn't take too long. However, I like to play with
the system for a day or more I actually do a release, so 0.6.13 will
probably be no earlier than Monday.

My guess is that the interval between 0.6.13 and 0.7.0 will be on the
order of a month. I'd like 0.7.0 to have in it at least
  * the incompatible changes anticipated in BUGS (e.g. ".fasl" instead
    of ".x86f", and probably INTERNAL-TIME-UNITS-PER-SECOND = 1000)
  * miscellaneous packaging changes (e.g. old CMU CL docs no longer
    bundled with SBCL sources)
  * IR1 interpreter gone, replaced with a trivial interpreter and
    FUNCALL of byte-compiled LAMBDA for the hard stuff
  * most of the renaming (e.g. systematizing names DEFINE-FOO,
    DEFBAR) that I talked about long ago
  * the code which has been accumulating in contrib/*-extras.lisp
Then sometime early in 0.7.x, but probably not in 0.7.0, I'd like
to straighten out the IR1-X bugs.

> With regard to MP: I've found the messages from Raymond Wiker saying
> that he's successfully ported the cmucl co-operative MP, and I've been
> wondering how much work it would take to make it pre-empt.  Here's an
> off-the-top-of-the-head idea -
> 
> 1) define locking macros that compile to no-ops if MP is not compiled
> in (keep the overhead out when we don't need it)
> 
> 2) rearrange the memory map to have _two_ static spaces: one of them
> for MP-safe code and the other for unsafe code.  Mangle the core
> creation or purify stages (<handwave>somehow</handwave>) so that we
> can specify into which space each file goes.  Initally all files are
> tagged as unsafe.
> 
> 3) mprotect() the unsafe space at startup time so that accesses to it
> trap.  Create a trap handler (in C; on Linux it would probably be
> another if clause in the sigsegv handler) that notices this, acquires
> a global lock on behalf of the current thread, and unprotects the
> unsafe space
> 
> 4) the thread scheduler needs to check if it's changing to/from the 
> thread with the global lock, and mprotect/unprotect as appropriate
> 
> 5) over time and as we find the bits with the biggest performance hit
> from this approach, rewrite them with appropriate fine-granularity locks
> and rebuild them into the safe static space.  
> 
> I have no idea how well this would work in practice, but it would
> allow a thread-safe (if slow) lisp right from the start, so at least
> has the advantage that it could be developed incrementally.

I suspect you'll still be subject to various kinds of problems in
the dynamic space, so that the mprotect() scheme won't be enough:
you'll need some hand-coded locks right from the start. E.g. when you
increase the size of a package enough, its data will end up in the
dynamic space; and the package operations aren't thread safe.

But although I've done some threaded development, it was always
threaded from the start: I have no experience with this kind of
transition from single-threaded code, and I can't really guess how
smoothly it might go.

> A propos of nothing, ww.telent.net (and thus CLiki) is now running on
> SBCL and has been for a couple of weeks.

Cool.

-- 
William Harold Newman <wil...@ai...>
Communication would be much more reliable if people would turn off the
gainy decompression. -- Del Cotter
PGP key fingerprint 85 CE 1C BA 79 8D 51 8C  B9 25 FB EE E0 C3 E5 7C