Steel Bank Common Lisp

Edi Weitz wrote:
<snip SBCL slowdown with refactoring>

The problem is that it's significantly faster to access slots via
accessors than via SLOT-VALUE in SBCL.  Changing the WITH-SLOTS in
READ-BYTE* to WITH-ACCESSORS (and adding an accessor for the octet
stack) makes your "foo" test improve from 10--11 seconds on my machine
to under two seconds.
-- 
Robert Macomber
tbnl@...

larryl <larryl@...> writes:

> >   The value NIL
> >   is not of type
> >     (OR (VECTOR CHARACTER) (VECTOR NIL) BASE-STRING PATHNAME FILE-STREAM).
> >   
> sb-bsd-sockets uses sb-grovel.
> 
> NIL should be the compiler name that sb-grovel uses. Prior to 1.0 it was 
> hardcoded to "gcc" but now sb-grovel will call sb-posix:getenv
> 
> in contrib/sb-grovel/def-to-lisp.lisp change
> in row 188 (sb-ext:posix-getenv "CC")
> 
> to
> 
> (or (sb-ext:posix-getenv "CC") "gcc")
> 
> and sb-grovel will fallback on gcc if your CC env is not set.
> 
> Alternatively add an sbcl sb-ext:posix-setenv :)

However, the CC environment variable should always be set when the
contribs are initially built. So this problem should only appear if
one for some reason tries to recompile the installed constants.lisp at
some later date. My guess is that this is actually a case of
sb-bsd-sockets not having been installed, probably due to not passing
its tests. And the constants.lisp that's being compiled is a relic
from an older installation at the same location.

-- 
Juho Snellman

This is going to be a bit long, so please bear with me.

I got complaints from at least one user that Hunchentoot was
significantly slower than TBNL when large files were uploaded and I
figured this was mainly due to its usage of FLEXI-STREAMS.
FLEXI-STREAMS had never been optimized for speed and it showed: A lot
of decisions were done at runtime, for example STREAM-READ-CHAR for a
flexi stream looked at the external format of the stream and from
there called the corresponding function for this format.  In the case
of an 8-bit encoding, it also retrieved the encoding table and handed
it over to this function.  Likewise, whether or not end-of-line
handling was necessary was decided at runtime.  This is obviously
pretty extensive if you're reading a large file character by character
and you have to do this several million times.

So, I thought I was smart and re-factored FLEXI-STREAMS.  For 8-bit
encodings, the encoding table is now stored within the (Gray) stream
object itself and doesn't have to be looked up anymore.  Furthermore,
the various encodings now all translate to their own subclasses of
FLEXI-STREAM, so they can all have their own STREAM-READ-CHAR
implementation optimized for the specific encoding.  Plus, there's a
mixin for encodings with line-end conversion (CR and CRLF) which is an
optimization for the "common case" of no line-end conversion (LF).

I changed all this and IMHO the code has lost quite a bit of its
original elegance.  Anyway, it's also much faster now.  For the
(:LATIN-1 :EOL-STYLE :LF) encoding which I was mostly interested in,
reading and writing of files is about five times faster now and
consing was reduced by several orders of magnitude.  Cool... :)

For LispWorks, that is.  After I had done this, I also tried with SBCL
and had to find out that my "improvements" made the code actually
noticeably /slower/ on SBCL while it still consed considerably.
Hmmm...

I myself use LispWorks and I'm quite happy with the new version, but
I'm aware that the majority of users of my open source libs probably
use SBCL, so I don't really want to make matters worse for them.  I
tried to find out what caused the performance decrease for SBCL, but I
think I don't know enough about its compiler to come up with useful
results in my limited spare time.

So, the whole point of this email is: I'd be very happy if some SBCL
experts could look at the FLEXI-STREAMS release and point out how I
could make it faster without sacrificing the improvements I'm seeing
for LispWorks.  The current release (0.9.0 as of this writing) is at

  http://weitz.de/files/flexi-streams.tar.gz

The last "known good" release for SBCL (0.8.0) is available from

  http://weitz.de/files/flexi-streams-0.8.0.tar.gz

Typical code I use for benchmarking is at the end of this email.

Thanks a lot in advance,
Edi.




(in-package :cl-user)

(defun foo ()
  ;; use a large file like (at least) a big Linux kernel
  (with-open-file (in% "/tmp/big" :element-type '(unsigned-byte 8))
    (let ((in (flex:make-flexi-stream in%
                                      :external-format (flex:make-external-format :latin-1
                                                                                  :eol-style :lf))))
      (loop for char = (read-char in nil nil)
            while char))))

(defun bar ()
  (with-open-file (in% "/tmp/big" :element-type '(unsigned-byte 8))
    (let ((in (flex:make-flexi-stream in%
                                      :external-format (flex:make-external-format :latin-1
                                                                                  :eol-style :lf))))
      (with-open-file (out% "/tmp/big_out"
                            :element-type '(unsigned-byte 8)
                            :direction :output
                            :if-exists :supersede)
        (let ((out (flex:make-flexi-stream out%
                                           :external-format (flex:make-external-format :latin-1
                                                                                       :eol-style :lf))))
          (loop for char = (read-char in nil nil)
                while char
                do (write-char char out)))))))

I have a system I'd like to compile and restart without ok'ing a few warnings
from third part modules.  Is there a setting I can use to do that?

Jonathon McKitrick
--
My other computer is your Windows box.

sbcl-help@...様へ、招待期限になりましたのでご確認下さい。
至急、下記の招待カードからアクセスしていただき、
準備したおります専用スペースへ入室お願いします。

 >>>  招待カード  :::: 明日期限 ::::

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
【No.20024】瑞穂様から
┌──┐
│＼／│　　ご招待カードを開封する
　　　　　http://www.now-on-site.com/acfree/20024

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

ずべて無料の自由恋愛コミュニティは、健全セフレの募集板です。

This is with SBCL 1.0 on Linux/x86:

  CL-USER> (enough-namestring "/etc/passwd" "/etc")
  "/etc/passwd"
  CL-USER> (enough-namestring "/etc/passwd" "/etc/")
  "passwd"
  CL-USER> (enough-namestring "/etc/passwd" "/")
  "/etc/passwd"
  CL-USER> (merge-pathnames (parse-namestring "/etc/passwd" nil "/") "/")
  #P"/etc/passwd"
  CL-USER> (merge-pathnames "etc/passwd" "/")
  #P"/etc/passwd"

I think that the third result should be "etc/passwd" and not
"/etc/passwd".  It would be more consistent with "passwd" in the
second case, and the last two computations show that "/etc/passwd" is
not the "shortest reasonable string" which satisfies the criterion
described in the CLHS dictionary entry for ENOUGH-NAMESTRING.

I would /guess/ that the following patch could fix that, but I don't
know enough about SBCL's pathname internals to really know what I'm
doing.

edi@...$ diff -u src/code/unix-pathname.lisp.~1.6.~ src/code/unix-pathname.lisp
--- src/code/unix-pathname.lisp.~1.6.~  2006-10-26 18:07:54.000000000 +0200
+++ src/code/unix-pathname.lisp 2006-12-23 23:54:52.000000000 +0100
@@ -279,7 +279,7 @@
               (cond ((null pathname-directory) '(:relative))
                     ((eq (car pathname-directory) :relative)
                      pathname-directory)
-                    ((and (> prefix-len 1)
+                    ((and (plusp prefix-len)
                           (>= (length pathname-directory) prefix-len)
                           (compare-component (subseq pathname-directory
                                                      0 prefix-len)

Cheers,
Edi.

sub

How do I achieve __FILE__ funtionality in sbcl?

Florian

-- 
Florian Ebeling
florian.ebeling@...

Ok, I can't see it.

Why are these to lines (marked below) unreachable
to the compiler?

I want to use the init and update form for the loop var
to read stuff in from stream.

(defun main ()
  (with-open-file (log "my.log" :direction :output :if-exists :append
:if-does-not-exist :create)
    (handler-case
	(progn
	  (format log "Wait for fd 0~%")
	  (sb-sys:wait-until-fd-usable 0 :input)
	  (format log "Usable reached.~%")
	  (let ((stream (sb-sys:make-fd-stream 0 :element-type
'(unsigned-byte 8) :input t :output t :buffering :none))
		(buf (make-array 100 :element-type '(unsigned-byte 8))))
	    (format log "Stream created.~%")
	    (do ((b (read-byte stream nil 'eof) ; <-- UNREACHABLE
		    (read-byte stream nil 'eof)) ; <-- UNREACHABLE
		 (i 0 (1+ i)))
		((equal b 'eof))
	      (format log ">>>>'~2x' " b)
	      (when (= 0 (mod (1+ i) 8))
		(terpri log)))))
      (error (c) (format log ">>> Error caught:~%'~a'~%error type:
~a~% " c (type-of c))))))

Deignoration would be very welcome. Perhaps it's
obvious, please kick me.

(sbcl 0.9.14, slime current, Linux)

Florian



-- 
Florian Ebeling
florian.ebeling@...