Re: [Sbcl-help] speed problem reading binary file

[Stas B. <sta...@gm...>]

Jeff Cunningham <je...@jk...> writes:

> Hi.
>
> I've been working with some big binary files with mixed word sized data. 
> The read times are considerably slower than I would have expected (e.g. 
> much slower than Matlab or Octave can read them). So I started profiling 
> and reduced it to a common case which is giving me most of the trouble. 
> Here is some code that illustrates what I've tried and where I'm at:
>
> (declaim (inline read-u2le))
>
> (defun read-u2le (s)
>    (declare (optimize (speed 3) (compilation-speed 0) (safety 0) (debug 0)))
>    (let ((u 0))
>      (declare (type (unsigned-byte 16) u))
>      (setf (ldb (byte 8 0) u) (read-byte s))
>      (setf (ldb (byte 8 8) u) (read-byte s))
>      u))
Should be
(defun read-u2le (s)
  (let ((u 0))
    (setf (ldb (byte 8 0) u)
          (the (unsigned-byte 8) (read-byte s)))
    (setf (ldb (byte 8 8) u)
          (the (unsigned-byte 8) (read-byte s)))
    u))
> (progn
>    (sb-profile::profile read-byte read-u2le)
>
>    (let ((n  100000) ;; file is on actually on the order of 2GB, this is 
> just for testing
>          (h)
>          (a))
>      (with-open-file (s "bin.dat" :direction :input :element-type 
> '(unsigned-byte 8))
>        (dotimes (i 8) (push (read-byte s) h))
>        (dotimes (i n) (push (read-u2le s) a)))
>      (print h)
>      (print (length a)))
>
>    (sb-profile::report))
>
> When I do this I get the following:
>
> (8 7 6 5 4 3 2 1)
> 100000
>    seconds  |     gc     | consed |  calls  |  sec/call  |  name
> ------------------------------------------------------
>       0.004 |      0.000 |      0 | 200,177 |   0.000000 | READ-BYTE
> ------------------------------------------------------
>       0.004 |      0.000 |      0 | 200,177 |            | Total
>
> estimated total profiling overhead: 0.29 seconds
> overhead estimation parameters:
>    2.e-8s/call, 1.43e-6s total profiling, 6.98e-7s internal profiling
>
> These functions were not called:
>   READ-U2LE
>
> which is perplexing because I can set a break  in read-u2le and it stops 
> there. I don't understand why in this example it doesn't think it's 
> called and isn't profiling it.
It's inlined, of course it's not called.
-- 
With best regards, Stas.