[Sbcl-devel] SBCL/Alpha progress

[Daniel Barlow <dan@te...>]

People might be interested to know that SBCL 0.6.5 on Linux/Alpha now
successfully starts up with cold-sbcl.core, and makes it through
cold-init to the * prompt.

It blows up very quickly afterwards (LET and QUIT are among the forms
that can make it die) but I thought that rated as progress anyway.
I can evaluate "(+ 1 2 3)" with no problems :-)

I've just reviewed a diff file, which is pleasingly short to far
(after removing a gazillion "%primitive print" calls).  So far I've
needed to make the following general categories of changes:

1) "semi-mechanical" changes to the cmucl alpha backend - package
   renaming, asterisks on the names of specials, and so on.

2) invocations of copiers for structure-based classes get replaced
   with calls to COPY-STRUCTURE as they're found to cause problems -
   e.g. COPY-NUMERIC-TYPE in code/late-type.lisp.  I find it vaguely
   disturbing that this was necessary, given that it apparently isn't
   on the x86

3) I hacked up the signals stuff in an if-it-builds-ship-it fashion -
   linux/alpha supports posix signals so I thought I'd wait for 0.6.7
   instead of expending effort trying to understand what 0.6.5 did

4) random changes: e.g.

- the appropriate bit of DO-COLD-FIXUP needed fixing to not use SAPs
- SYMBOL-HASH has no VOP on the Alpha, so calls to it tended to spin
  on CPU.  I had to write a real function: 
   (%sxhash-simple-string (symbol-name function))
- code/debug-int.lisp has references to a %ALPHA package; changed to
  use SB!VM instead

5) weird stuff: e.g.

- the DECLAIM of DO-COLD-FIXUP causes :JMP-HINT relocations to break.
  Commented out

- "(def-type-translator or" on the alpha uses REDUCE, which
  crashes.  Rewritten to use DOLIST instead

Signals are still set to bite me.  GC I expect will bite me (it looks
like CMUCL on the Alpha uses gc.c which is neither conservative or
generational).  Floating point modes likewise as soon as I uncomment
the appropriate call in.  Dynamic loading, when I get that far, I'll
make work like the x86 Linux version works.
  
I'm not sure if it gets easier or harder from here ...

-dan

-- 
  http://ww.telent.net/cliki/ - CLiki: CL/Unix free software link farm

[Sbcl-devel] SBCL/Alpha progress

[Daniel Barlow <dan@te...>]

People might be interested to know that SBCL 0.6.5 on Linux/Alpha now
successfully starts up with cold-sbcl.core, and makes it through
cold-init to the * prompt.

It blows up very quickly afterwards (LET and QUIT are among the forms
that can make it die) but I thought that rated as progress anyway.
I can evaluate "(+ 1 2 3)" with no problems :-)

I've just reviewed a diff file, which is pleasingly short to far
(after removing a gazillion "%primitive print" calls).  So far I've
needed to make the following general categories of changes:

1) "semi-mechanical" changes to the cmucl alpha backend - package
   renaming, asterisks on the names of specials, and so on.

2) invocations of copiers for structure-based classes get replaced
   with calls to COPY-STRUCTURE as they're found to cause problems -
   e.g. COPY-NUMERIC-TYPE in code/late-type.lisp.  I find it vaguely
   disturbing that this was necessary, given that it apparently isn't
   on the x86

3) I hacked up the signals stuff in an if-it-builds-ship-it fashion -
   linux/alpha supports posix signals so I thought I'd wait for 0.6.7
   instead of expending effort trying to understand what 0.6.5 did

4) random changes: e.g.

- the appropriate bit of DO-COLD-FIXUP needed fixing to not use SAPs
- SYMBOL-HASH has no VOP on the Alpha, so calls to it tended to spin
  on CPU.  I had to write a real function: 
   (%sxhash-simple-string (symbol-name function))
- code/debug-int.lisp has references to a %ALPHA package; changed to
  use SB!VM instead

5) weird stuff: e.g.

- the DECLAIM of DO-COLD-FIXUP causes :JMP-HINT relocations to break.
  Commented out

- "(def-type-translator or" on the alpha uses REDUCE, which
  crashes.  Rewritten to use DOLIST instead

Signals are still set to bite me.  GC I expect will bite me (it looks
like CMUCL on the Alpha uses gc.c which is neither conservative or
generational).  Floating point modes likewise as soon as I uncomment
the appropriate call in.  Dynamic loading, when I get that far, I'll
make work like the x86 Linux version works.
  
I'm not sure if it gets easier or harder from here ...

-dan

-- 
  http://ww.telent.net/cliki/ - CLiki: CL/Unix free software link farm

Re: [Sbcl-devel] SBCL/Alpha progress

[William Harold Newman <william.newman@ai...>]

On Sun, Sep 10, 2000 at 04:23:04PM +0100, Daniel Barlow wrote:
> 
> People might be interested to know that SBCL 0.6.5 on Linux/Alpha now
> successfully starts up with cold-sbcl.core, and makes it through
> cold-init to the * prompt.
> 
> It blows up very quickly afterwards (LET and QUIT are among the forms
> that can make it die) but I thought that rated as progress anyway.
> I can evaluate "(+ 1 2 3)" with no problems :-)

This sounds very good to me. (Now all I need is an Alpha.:-)
 
> 3) I hacked up the signals stuff in an if-it-builds-ship-it fashion -
>    linux/alpha supports posix signals so I thought I'd wait for 0.6.7
>    instead of expending effort trying to understand what 0.6.5 did

> Signals are still set to bite me.  GC I expect will bite me (it looks
> like CMUCL on the Alpha uses gc.c which is neither conservative or
> generational).  Floating point modes likewise as soon as I uncomment
> the appropriate call in.  Dynamic loading, when I get that far, I'll
> make work like the x86 Linux version works.

Signals are, if I may say so myself, *much* cleaner in 0.6.7 than in
0.6.5. Take a look, and hopefully you will agree. Of course, it won't
help that the code corresponds less closely to the CMU CL code,
including the Alpha code that you're starting with, but I hope that
most of the changes should be very straightforward. (E.g. writing a
new wrapper function context_eflags_addr(..) is not rocket science,
unless I've screwed up by making an implicit assumption which doesn't
make sense on the Alpha.)

GC will probably be a bit of a mess, since the #+GENGC/#+GENCGC/#+X86
stuff in CMU CL was fairly messy to begin with and I might have made
it worse, since I didn't have any way to test whether my changes make
sense on non-X86 systems. But the interface is not fundamentally all
that complicated, so it should probably be pretty manageable. And I
don't think anything I did to the CMU CL code base should have broken
the fundamental precise-vs-conservative compilation characteristics of
the system, so if gc.c is precise and worked on the Alpha on CMU CL
2.4.7, I'd expect it to be fundamentally OK on SBCL.

Dynamic loading looks pretty simple actually, now that I've read
the man pages for the appropriate system calls.

> I'm not sure if it gets easier or harder from here ...

Easier, I hope, but I can't say for sure. In my original port, by the
time I got to this point, most of the hard stuff was over, and much of
the stuff which wasn't over shouldn't be an issue in your port (e.g.
making PCL work). However, don't remember problems like what you
describe with LET and (presumably, as part of the implementation of
QUIT) THROW. If the compilation of special forms is screwed up, that
could be as simple as finding one or two messed up 32-bit-vs-64-bit
hacks, but could possibly be something much nastier. I hope it's the
former..

-- 
William Harold Newman <william.newman@...>
software consultant
PGP key fingerprint 85 CE 1C BA 79 8D 51 8C  B9 25 FB EE E0 C3 E5 7C

Re: [Sbcl-devel] SBCL/Alpha progress

[Daniel Barlow <dan@te...>]

William Harold Newman <william.newman@...> writes:

> On Sun, Sep 10, 2000 at 04:23:04PM +0100, Daniel Barlow wrote:

[Yeesh.  That long ago]
 
> > People might be interested to know that SBCL 0.6.5 on Linux/Alpha now
> > successfully starts up with cold-sbcl.core, and makes it through
> > cold-init to the * prompt.
> > 
> > It blows up very quickly afterwards (LET and QUIT are among the forms
> > that can make it die) but I thought that rated as progress anyway.
> > I can evaluate "(+ 1 2 3)" with no problems :-)

I found a couple of free evenings this week to work on it further.
SB!C::%DEF-REFFER does a JFIW (Jump Far Into Weeds) when I call it - 
the problem seems to be that (SETF SB!C::FUNCTION-INFO-IR2-CONVERT)
didn't get dumped.  At least, it shows as undefined in cold-sbcl.map

In fact:
:; grep FUNCTION-INFO ../../output/cold-sbcl.map 
0x308F76F8: SB!C::FUNCTION-INFO-P   #X308F76E1
0x308F85C8: SB!C::MAKE-FUNCTION-INFO   #X308F85B1
0x308FEAA8: SB!C::FUNCTION-INFO-OR-LOSE   #X308FEA91
0x314FFF20: SB!C::INLINE-FUNCTION-INFO-P   #X314FFF09
0x31500B80: SB!C::MAKE-INLINE-FUNCTION-INFO   #X31500B69
(SETF SB!C::FUNCTION-INFO-IR2-CONVERT)
308F7435: SB!C::FUNCTION-INFO[11]
314FFC55: SB!C::INLINE-FUNCTION-INFO[6]

I don't see any FUNCTION-INFO accessors there at all.  Should they
have been dumped?  How do they spring into existence if not?


-dan

-- 

  http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources 

Re: [Sbcl-devel] SBCL/Alpha progress

[Daniel Barlow <dan@te...>]

Daniel Barlow <dan@...> writes:
> I don't see any FUNCTION-INFO accessors there at all.  Should they
> have been dumped?  How do they spring into existence if not?

I should learn to read more carefully.  A long time ago in a galaxy
far away, in regard to a superficially different but I suspect
basically similar problem, WHN wrote

> Just because you see warnings at GENESIS time doesn't necessarily mean
> that there's a real problem. The existing DEFSTRUCT/DEF!STRUCT logic
> should generate toplevel forms to install the closures. GENESIS
> doesn't know how to magically do them before cold init starts, but
> that doesn't mean that they don't happen. As long as those toplevel
> forms are executed (which should happen partway through cold init)
> before any calls to COPY-SB or COPY-FINITE-SB, then everything should
> be fine. (The joys of late binding.:-) (Actually, I should probably
> add some text in the GENESIS warning output explaining this.)

So something somewhere in cold-init is busted.


-dan

-- 

  http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources 

Re: [Sbcl-devel] SBCL/Alpha progress

[William Harold Newman <william.newman@ai...>]

On Fri, Dec 01, 2000 at 12:29:05PM +0000, Daniel Barlow wrote:
> Daniel Barlow <dan@...> writes:
> > I don't see any FUNCTION-INFO accessors there at all.  Should they
> > have been dumped?  How do they spring into existence if not?
> 
> I should learn to read more carefully.  A long time ago in a galaxy
> far away, in regard to a superficially different but I suspect
> basically similar problem, WHN wrote

*I* should learn to read more carefully. Before I replied to your
previous message I scanned ahead to see whether anyone had
posted a reply. Too bad my vision and/or reading centers weren't
functioning.:-( I think I should probably drink a little coffee.

-- 
William Harold Newman <william.newman@...>
software consultant
PGP key fingerprint 85 CE 1C BA 79 8D 51 8C  B9 25 FB EE E0 C3 E5 7C

Re: [Sbcl-devel] SBCL/Alpha progress

[Daniel Barlow <dan@no...>]

Daniel Barlow <dan@...> writes:

> I should learn to read more carefully.  

Indeed.

Had I read runtime/runtime.c and runtime/alpha-arch.c, I'd have seen
that the latter has a non-empty arch_init() function that the former
doesn't call.  The primary purpose of this is to map a page at 0x10000
- so what I thought was a bug in the fixups was actually perfectly
legitimate code and it is supposed to jump there after all.

This time I get to the toplevel _and_ I can evaluate complicated stuff
like "IF".  LOAD blows up because PROBE-FILE returns NIL because -
surprise! - the layout of the "stat" structure is different again.  Yay
for hand-copying constants from header files!

I've written a wrapper, for what it's worth.  Given all the stat
macros in Linux it's hard enough to tell exactly which structure you
need anyway, so runtime/stat_wrapper.c does the actual call and shoves
the answer into a structure that we _do_ know the shape of.  "Fast" is
probably not the word, but I doubt it's significant compared to the
cost of the stat operation itself.

I'm now rebuilding again (it didn't like me attempting to shortcut by
changing the size of dev-t after loading output/after-xc.core ...) -
I'll send patches for stat when it looks like it works.


-dan

-- 

  http://ww.telent.net/cliki/ - Link farm for free CL-on-Unix resources 

Re: [Sbcl-devel] SBCL/Alpha progress

[William Harold Newman <william.newman@ai...>]

On Fri, Dec 01, 2000 at 11:05:43AM +0000, Daniel Barlow wrote:
> I found a couple of free evenings this week to work on it further.
> SB!C::%DEF-REFFER does a JFIW (Jump Far Into Weeds) when I call it - 
> the problem seems to be that (SETF SB!C::FUNCTION-INFO-IR2-CONVERT)
> didn't get dumped.  At least, it shows as undefined in cold-sbcl.map
> 
> In fact:
> :; grep FUNCTION-INFO ../../output/cold-sbcl.map 
> 0x308F76F8: SB!C::FUNCTION-INFO-P   #X308F76E1
> 0x308F85C8: SB!C::MAKE-FUNCTION-INFO   #X308F85B1
> 0x308FEAA8: SB!C::FUNCTION-INFO-OR-LOSE   #X308FEA91
> 0x314FFF20: SB!C::INLINE-FUNCTION-INFO-P   #X314FFF09
> 0x31500B80: SB!C::MAKE-INLINE-FUNCTION-INFO   #X31500B69
> (SETF SB!C::FUNCTION-INFO-IR2-CONVERT)
> 308F7435: SB!C::FUNCTION-INFO[11]
> 314FFC55: SB!C::INLINE-FUNCTION-INFO[6]
> 
> I don't see any FUNCTION-INFO accessors there at all.  Should they
> have been dumped?  How do they spring into existence if not?

To save space, the out-of-line versions of structure slot accessors
are defined not as ordinary functions, but as closures. The code
which installs them looks like
      (DOLIST (SLOT (DD-SLOTS INFO))
	(LET ((DSD SLOT))
	  (WHEN (AND (DSD-ACCESSOR SLOT)
		     (EQ (DSD-RAW-TYPE SLOT) T))
	    (PROTECT-CL (DSD-ACCESSOR SLOT))
	    (SETF (SYMBOL-FUNCTION (DSD-ACCESSOR SLOT))
		  (STRUCTURE-SLOT-GETTER LAYOUT DSD))
	    (UNLESS (DSD-READ-ONLY SLOT)
	      (SETF (FDEFINITION `(SETF ,(DSD-ACCESSOR SLOT)))
		    (STRUCTURE-SLOT-SETTER LAYOUT DSD))))))
and runs as a toplevel form. Since it doesn't use DEFUN, GENESIS
doesn't recognize it, and doesn't handle it specially. Thus,
structure slot accessors used early in cold init need to be compiled
inline, which basically means that the DEFSTRUCT needs to be compiled
before the slot accessor call. But once toplevel forms have been loaded,
this issue should go away. (You still won't see the structure
slot accessor in cold-sbcl.map, of course, but it should be there
in the Lisp image.) And since you said above that you're getting to a 
REPL prompt, I'd expect that the toplevel forms have been run.

-- 
William Harold Newman <william.newman@...>
software consultant
PGP key fingerprint 85 CE 1C BA 79 8D 51 8C  B9 25 FB EE E0 C3 E5 7C