#15 persistent object storage

open
None
5
2004-08-18
2004-07-29
No

if I want to save a data structure,
I have two options now:

1. WRITE to a file
2. SAVEINITMEM

both have drawbacks:

1. WRITE uses ascii text which may be slow to
read and write, and it may fail for large
highly circular data because of STACK.

2. MEM cannot be loaded at will: if I want to run
something on 3 datasets, I must start 3 processes.

I suggest that we add object save/retrieve facility
with a berklely-db backend -
probably in the BDB module.

HASH Berkely-DB tables appear to be the right choice.
pointer objects are to be written as records keyed on
as_oint(obj) and with value being the memory to which
the object pointer refers, with other pointer objects
replaced
with as_oint(obj).
immediate slot values are to be written as is.

challenges:
1. thus must be platform-independent
(including 64-bit vs 32-bit).
2. will this require me to write a separate routine
for every single CLISP built-in type?

Comments?
Suggestions?

Discussion

  • Sam Steingold

    Sam Steingold - 2004-08-02

    Logged In: YES
    user_id=5735

    note also that MEM files are extremely platform-dependent,
    so they are next to useless for data storage.

    WRITE is good because it lets one edit data with a text editor.
    unfortunately, to get linebreaks there requires *print-pretty*
    set to T, which makes output VERY slot (remember, we
    are talking about HUGE data piles) and without any linebreaks
    even emacs edits huge files VERY slowly.

    a DB-based storage will let one edit the data "programmatically"
    before loading the whole thing.

     
  • Sam Steingold

    Sam Steingold - 2004-08-06

    Logged In: YES
    user_id=5735

    <http://article.gmane.org/gmane.lisp.clisp.devel:12214>
    <http://article.gmane.org/gmane.lisp.clisp.devel:12225>
    <http://article.gmane.org/gmane.lisp.clisp.devel:12226>
    <http://sourceforge.net/mailarchive/message.php?msg_id=9137008>
    <http://sourceforge.net/mailarchive/message.php?msg_id=9139407>

     
  • Sam Steingold

    Sam Steingold - 2004-08-06

    Logged In: YES
    user_id=5735

    <http://www.common-lisp.net/project/elephant/>
    an object database for common lisp

    no code released yet.

     
  • Sam Steingold

    Sam Steingold - 2004-08-06

    Logged In: YES
    user_id=5735

    <http://common-lisp.net/project/cl-store/>
    A Common Lisp Serialization Package

    writes binary to streams, can probably be adapted
    for database purposes.

     
  • Sam Steingold

    Sam Steingold - 2004-08-18

    Logged In: YES
    user_id=5735

    <http://lecture.pentaside.org/paper/persistence-lemmens.txt>

     
  • Sam Steingold

    Sam Steingold - 2004-08-18

    Logged In: YES
    user_id=5735

    different backends should be permitted - streams as well as BDB.

     
  • Sam Steingold

    Sam Steingold - 2004-08-18
    • summary: persistent object storage using berkeley-db --> persistent object storage
     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

JavaScript is required for this form.





No, thanks