[Sbcl-devel] purify failure when compact-info-env-entries-bits is too small

[Cyrus H. <ch...@bo...>]

Dear SBCL-dev,

I would like to resubmit a one-line patch to 
sbcl/src/compiler/globaldb.lisp that I consider to be rather useful. 
Allow me to give a little background on the motivation for the patch. I 
have been, for better or worse, attempting to generate FFI definitions 
for the MacOS Carbon APIs and have run into a problem attempting to run 
purify, which gets called automatically by save-lisp-and-die. The call 
to purify was failing and giving an error about 65536 not being an 
(integer 0 65535) or something similar. Before you write this off as 
some wacko ppc-specific thing, Xophe mentioned that he has seen 
something similar before in a different context with, presumably, lots 
of definitions in an environment.

Looking a bit at the code, it seems that purify (and perhaps other 
things?) attempt to create a compact environment that contains the same 
information as an existing environment, but (hopefully) in a more 
compact representation. There is a variable 
compact-info-env-entries-bits that is used to determine the size (in 
bits) of the compact-info-env-entries-index type, which, in turn, is 
used as the type of the compact-info-inv/cache-index and as the type of 
the simple-array compact-info-env/index.

As an environment gets more than 65535 entries, it becomes impossible 
to compact it do to the size of the index into the underlying data 
structures. So the obvious thing to try is to boost the size of the 
index. I made compact-info-env-entries-bits 32 and have been using this 
for a few weeks (on sbcl 0.8.20.xx on PPC) with no problems.

This all sounds reasonable to me, but there is one potential problem. 
At some point a table-size gets calculated based on the number of names 
in the environment. We call primify to get a prime number size for the 
table. primify declares its argument x as an (unsigned-byte x) and then 
iterates over the odd numbers >= x until it finds a prime number. But 
positive-prime declares its argument to be a fixnum, so it's possible 
that we could have a table size that is > most-positive-fixnum which 
cause positive-primep to fail. I haven't run into this in practice, but 
we might want to watch it for this and/or come up with a better 
approach for getting the size of the table. There's a comment about 
using almost-primify from hash-table.lisp, but this must be an 
out-of-date comment, as I can't find this function in hash-table.lisp.

In any event, I'd appreciate it if those who understand what's going on 
here better than I do would take a look at this and consider boosting 
the size of the compact-info-env-entries-bits.

The one-line patch is attached.

Thanks for all of your efforts in making SBCL such a joy to program 
with.

Sincerely,

Cyrus



Index: globaldb.lisp
===================================================================
RCS file: /cvsroot/sbcl/sbcl/src/compiler/globaldb.lisp,v
retrieving revision 1.36
diff -u -r1.36 globaldb.lisp
--- globaldb.lisp       12 Jun 2004 13:55:49 -0000      1.36
+++ globaldb.lisp       26 Mar 2005 15:03:13 -0000
@@ -476,7 +476,7 @@
  ;;;; compact info environments

  ;;; The upper limit on the size of the ENTRIES vector in a 
COMPACT-INFO-ENV.
-(def!constant compact-info-env-entries-bits 16)
+(def!constant compact-info-env-entries-bits 32)
  (deftype compact-info-entries-index () `(unsigned-byte 
,compact-info-env-entries-bits))

  ;;; the type of the values in COMPACT-INFO-ENTRIES-INFO