Sounds quite practical. -bryan

 


From: jdbm-developer-admin@lists.sourceforge.net [mailto:jdbm-developer-admin@lists.sourceforge.net] On Behalf Of Kevin Day
Sent: Tuesday, April 11, 2006 5:28 PM
To: JDBM Developer listserv
Subject: re[6]: [Jdbm-developer] record managment

 

Bryan-

 

OK - as long as there is no significant performance hit - and this will let us reduce the size of the distro by ditching the (large) long->object map classes.

 

I see where you are headed with the OID object (and I agree that using flyweight is probably not necessary).

 

How about we say that we will design with the concept of an object as the record id (instead of the primitive long that we are currently using).  It will be handy to have the bit arithmetic encapsulated, and that will provide a *future* avenue for deferred allocation.  The baseline implementation would just return a simple hash of the underlying long recid.

 

Sound good?

 

- K

 

 

> Kevin,

I was not really thinking of OID as a reference object but as an
encapsulation of the 64-bit long identifier which provided some handy OID
math and gives us an opportunity to return an OID that is not yet locked to
a specific 64-bit long value.

I measured the performance benefit from the native long to object map in the
cache - less than 1% on a cache intensive application.  It was a great idea,
but it just did not pan out.

I see this as very similar to returning a Long from insert(byte[]), but with
the twist that (a) the assignment of the long value could be deferred and
(b) we can bundle some OID "math" operations on the interface.  There would
not be any OID#fetch() method or the like, so it is not like a Java
Reference object.  It is really just one level of indirection over the long
integer value.

In order to use a floating OID as a "Reference" object you would need have a
Reference field initialized with a private OID value.  As long as the OID is
floating you have to use a hard reference to the OID object so that you can
lookup the persistent object in the cache by the OID instance.  As soon as
the OID gets assigned its long value you can replace that Reference with a
WeakReference or with just the 64-bit integer value and materialize an OID
on an as needed basis.

We would have to do something a little sneaky to make hash maps with OID
keys work.  Before the long value is assigned, the lookup will only succeed
if you have that specific OID instance.  After the long value is assigned,
any OID instance with the same long value would match.

If I read you correctly, you are suggesting that there could be an OID
instance where the application could set the 64-bit value and do whatever it
needs to with the OID and then reuse it for another purpose -- the goal
being to reduce the need for GC of OIDs?  I could see that, but we are
creating Longs all the time right now.  While I was worried about a
performance hit from that it has clearly proven to be a non-issue.  I think
that using OIDs will likewise be a non-issue.

-bryan

________________________________________
From: jdbm-developer-admin@lists.sourceforge.net
[mailto:jdbm-developer-admin@lists.sourceforge.net] On Behalf Of Kevin Day
Sent: Tuesday, April 11, 2006 2:48 PM
To: JDBM Developer listserv
Subject: re[4]: [Jdbm-developer] record managment

Bryan-
 
The issue that I have with enforcing a reference object like this
(especially at the core record manager level) is that it pushes the
developer away from a natural way of working.  The database code winds up in
the business logic code, and all of what that entails.
 
If we were using default serialization, then we could do some things with
serialization proxies (I did this succesfully in a project a few years ago),
but this kind of thing probably belongs at a higher level in the system.
 
 
 
Now, back to the idea of using an object for OID...  we went through a lot
of steps last year to introduce a long->object map to jdbm - this would
pretty much invalidate that.  What are the implications of creating lots of
tiny little OID objects like this?  Generational GC is pretty good at
handling this kind of thing, but I think it's worth discussing in light of
the long->object map work that we did.
 
What do you think about using the Flyweight GOF pattern instead.  A single
OID object can be configured with the byte representation (actually a long
value), then the page and slot can be retrieved in an object oriented manner
(instead of explicitly doing bit operations in the consumers of the recids).
 
- K
 
 
 

>
I am thinking of
 
OID insert( byte[] data ).
 
I think that the policy could be configurable.  For an application which
could support floating OIDs, the default would be to defer the assignment of
the 64-bit identifier until the record was serialized and allocated on a
page image.  The big win here is in clustering of OIDs and/or records based
on their mature state.
 
For an application which can not support floating OIDs, the OID would be
eagerly assigned  exactly as it is today.
 
I think that we might benefit from an OID object for several reasons, not
the least is that it will end our constant creation of Longs.
 
I would see at least the following methods on OID:
 
long getPageId(); // just the page identifier.
int getSlot(); // the slot on the page.
long longValue(); // the entire 64-bit long value.
 
Supporting floating OIDs would require the application to abstract the
reference to the persistent object.  What I had in mind for the GOM
framework was supporting persistence by reachability  an object which is not
reachable from a named (root) object in the store is not persistent and will
never be assigned a fixed OID.  This gives you the same functionality for
transient and persistent data, which is nice.
 
-bryan
 


From: jdbm-developer-admin@lists.sourceforge.net
[mailto:jdbm-developer-admin@lists.sourceforge.net] On Behalf Of Kevin Day
Sent: Tuesday, April 11, 2006 12:00 PM
To: JDBM Developer listserv
Subject: re[2]: [Jdbm-developer] record managment
 

Bryan-

 

Given that the insert() operation returns a recid that is then used
extensively by other objects in the store, how do you see this working?  Are
you talking about not having a long recid that is returned for object
references, but instead having some sort of 'reference' object?  I do this
in my applications, but you have to handle serialization specially for the
reference objects...

 

- K

  

 
> I am wondering if we could define the record management API in terms of an
OID interface and then provide for delayed conversion of OIDs to 64-bit
integers.  After insert but before conversion the OID would "float".  The
conversion would happen when the object was evicted from cache or the
transaction commits.  At this point the OID would be assigned and locked for
the object.  I think that this would make it possible to cluster OIDs based
on the "mature" state of the object.

Internally, the OID object would maintain a long integer.  We need to
differentiate an OID which is NULL (0L) from one which is floating (-1L?).
All caching operations for records would be defined in terms of OIDs.

I would suggest a factory for OIDs.  Different stores might choose different
pages sizes, which would mean a different #of bits reserved for the slot# on
the page vs the pageId.  That context would have to come from a factory
associated with the store.

-bryan

-----Original Message-----
From: jdbm-developer-admin@lists.sourceforge.net
[mailto:jdbm-developer-admin@lists.sourceforge.net] On Behalf Of Thompson,
Bryan B.
Sent: Tuesday, April 11, 2006 6:32 AM
To: Kevin Day; JDBM Developer listserv
Subject: RE: [Jdbm-developer] record managment

Kevin,

Comments below.

-bryan

<
<



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd

<

 

------------------------------------------------------- This SF.Net email is sponsored by xPML, a groundbreaking scripting language that extends applications into web and mobile media. Attend the live webcast and join the prime developer group breaking into this new coding territory! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 _______________________________________________ Jdbm-developer mailing list Jdbm-developer@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/jdbm-developer