[Xorm-devel] Re: Distributed Cache Management

[Harry E. <ha...@gm...>]

Having spoken with Doug, I am looking at using the JGroups (or JavaGroups,
gotta love that Sun trademark enforcement) as the mechanism for transportin=
g
the notifications.

Thoughts:
1. Modify LRUCache to optionally notify a listener when a Row is added,
updated, or removed.
2. Subclass LRUCache to use (most likely) a JGroups NotificationBus for
cache events.
3. Setup JGroups to manage this notification, and provide properties the
subclass can read to allow for enhanced settings (UDP versus TCP
notifications, etc).
4. I walked through some xorm code, and it doesn't appear that relationship
proxy information is ever directly conveyed to the cache level, which means
that there is no way, at that level, to invalidate the Collection case I
listed previously. I could really use some insight into an approach for thi=
s
area (*cough* Wes *cough*) as this seems like a kinda important part. I
might have this wrong, but it appears xorm manages all collection level
information (12M and M2M) at the ObjectProxy level, and not at all in the
Row DataCache level. Is that correct?
5. I am unclear whether InterfaceInvocationHandlers need to have their Row
cleared in this scheme, or if they generally pass out of scope frequently
enough that the Row is the main thing to focus on. This has to do with the
concern that they hold a direct hard reference to a Row object, so throwing
that out of cache isn't going to accomplish much if they are still pointing
to it.

I am really interested in the JGroups DistributedLockManager class, but it
would have to be combined / enhanced with cache management to use it, and i=
t
seems like there should be an option, regardless, to only do cache
management without the overhead of distributed locks. The JBossCache stuff
also might be more appropriate in the long term, as it uses JGroups stuff,
but seems like it can handle both locks and caching. I will have to do more
research.

Input very much appreciated.

Harry

On 10/4/05, Harry Evans <ha...@gm...> wrote:
>
> Optimistic Distributed Cache Management
>
> This is a proposal for a simple distributed cache invalidation
> strategy for xorm. It might be implementable at only the cache
> manager layer, or might require integration at a higher layer. It
> does not do coordinated lock management, but would do reasonable cache
> invalidation in the cases where the same enitity was not modified in
> multiple instances in rapid succession. This is seen as the first
> phase in a more pessimistic strategy that would do actual lock
> management.
>
> It uses the multicast transmission facilities in Java to set up a peer
> to peer local network on which cache invalidation messages are sent to
> other servers using the same datastore. Detected events are broadcast
> to other servers, allowing them to either hollow the referenced object
> or remove them completely from cache. Below are the major pieces of
> the implementation as I see them.
>
> I am looking at using JRMS (Java Reliable Multicast Service) jar
> (http://www.experimentalstuff.com/Technologies/JRMS/index.html) to
> reasonably guarantee the receipt of packets to each server
> (specifically the LRMP variety, with late-join catchup disabled).
> Basic desription of major areas below, followed by questions.
>
> I would really really like to find a way to do this at the cache
> manager level, as it would be a simple properties change to enable it,
> by using a version of the LRU cache that implemented the additional
> functionality, but I am not sure this is possible. Feedback
> appreciated.
>
> Harry
>
> Detection:
> "Broadcast" means notification sent out over multicast
> Objects that are deleted must be broadcast to be removed
> Objects that have simple attributes or one way references updated must
> be broadcast to be hollowed or removed
> Changes to collections must be broadcast:
> ObjParent has a collection that contains objects, including ObjChild
> ObjChild has a direct reference to ObjParent
> ObjChild has its reference to ObjParent removed (or reassigned to
> ObjNewParent)
> ObjChild must be broadcast to be hollowed or removed
> ObjParent must either be broadcast to be hollowed or removed or must
> be broadcast have its collection set to unresolved
> ObjNewParent must either be broadcast to be hollowed or removed or
> must be broadcast to have its collection set to unresolved
>
> Transmission:
> Initial transmission is seen as non locking multicast packets
> All cache instances are both transmitters and receivers of packets
> (there is no central server)
> When a cache instance detects a broadcast event:
> It formulates a broadcast packet, containing the table name and
> primary key of the Object
> (Should this be class instead of table?)
> It transmits the packet on the multicast address for the group
> Each cache instance (except the sender) receives the packet
> Each cache instance purges (or makes hollow?) the referenced object
> (Does this need to distinguish between hollow vs removal for change
> vs delete, respectively?)
> (How are collections handled?)
>
> Packet format:
> Packets are datagrams, pssoibly of fixed length:
> Format: (Action ( Remove vs hollow?) + ) Primary key + Hash of
> (Class | table name?)
> (Can we use fixed length for packet? ie can we reliably hash class
> or table name?)
> Packet is fixed length to minimize message size. Variable length is
> possible, but seems bad.
> If using a hash, each cache manager calculates hash value upon entry
> of (class | table) into cache space. Unknown hashes are ignored (as
> they aren't in cache).
>
> Questions:
> How do we detect the appropriate broadcast events at the cache manager
> level?
> Can multiple VMs on the same host bind to the same multicast group
> address? If not, do we need some form of local relay? (Tested to be
> okay on windows)
> Do we use table name, class name, or something else for entity
> representation?
> Can we use a hash of the entity representation (class or table name)
> to allow for fixed length packets?
> Which hash formula do we use?
> Is there too much multicast traffic on a network of 60 machines? (we
> can use separate addresses for separate clusters)
> Are networks (or switches) commonly configured to support local
> multicast packets?
>