[Sbcl-devel] How To have my own integer types?

[Philipp M. <ph...@ma...>]

Hi everybody,

in one of my projects I have a large number of small objects;
for testing purposes, and to reduce memory usage and GC time
I wrote a (simple) DEF-COLUMN-STRUCT that (pre)allocates an
array for each slot in the structure, and provides the getter
& setter functions (as well a GF that allows to reset the storage)

There's also a "(typedef ,struct-name () `(unsigned-byte 27))",
so that other structures, classes, and arrays can use "struct-name"
for specialization..


Most of that worked out quite nicely.

+ only 3 (specialized) arrays instead of millions of objects
+ preallocation (for approx. known number of entries) is easy
+ "allocation" via an atomic-incf is quick
- no GC support (not needed here)
~ identification of "object" happens via the array index
- no support for type dispatch


The last two points is what I'd like to improve now --
by having (a few) own number types that are distinct from FIXNUM
etc., so that

   (etypecase input
     (fixnum ...)
     (struct-name ...)

and

   (defmethod print-object ((obj struct-name) ...) ...)

work as "expected".


Constraints/wishes:
- CHARACTER is not big enough -- there are already ~50M structure
   instances right now, and that'll get bigger over time.
   character is limited to ~1M.
- If 4 tag bits are taken up, but the "ID" (index) of the object
   could still be 28 bits long, I hope that references via other
   vectors would resolve to a 32bit element type -- that would save
   some memory again.


I read doc/internals/objects-in-memory.texinfo about
sb-vm:n-fixnum-tag-bits, and looked at the referenced
src/compiler/generic/early-objdef.lisp.

The latter lists a few OTHER-IMMEDIATE-x-LOWTAG and PADx-LOWTAG
that seem to be available for this purpose.


So, my questions are:

- Is it necessary/helpful to increase the n-fixnum-tag-bits via
   src/compiler/generic/early-vm.lisp for this purpose, or are lowtags
   already available?

- Is there already some (hidden, reserved, internal) infrastructure
   that allows to define a new (semi-numeric) type that is different
   from FIXNUM etc.?

- If now, is there some code that could be moved into a contrib?
   Yeah, I'm aware that this would be a highly restricted usecase --
   only one user per image to avoid conflicting tag bits and so on.


Thanks for any hints!