Re: [jgroups-users] Implementing Multiple ReplicatedHashMap Instances on a Cluster

[Jim T. <jim...@no...>]

Thanks, Bela.  I created an issue on JIRA for the bug.

I am moving down the path of a single channel with muxed RPC.  I'm making a
MuxReplicatedHashMap which requires that the initial state be passed to it
in the constructor (the alternate constructor, start(), group monitoring,
and state transfer are removed).  I'm planning to implement the state
transfer in the main channel as a list of maps which will be used when
creating the replicated maps.   In my program the state transfer will only
be requested when the channel first connects.   I'm a bit concerned that
there will be a delay between the time the state is transferred and the
MuxRpcDispatcher in each map are created.  Will any map updates from other
members that occur during the gap be lost or will they be re-transmitted
until the new maps are fully up and running? I guess I don't exactly know
what happens when one member of the cluster does not have an implementation
of the RPC (yet).

I have an unrelated question. A subset of my data (but the vast majority of
my message count) is well suited to send as unreliable unicast UDP (right
now I have non-jgroups sockets for that).  I'm contemplating sending them
using the Unreliable flag through my main channel or creating a
'unreliable' fork channel -- either of which would eliminate the need for
networking code and having to keep track of / pass around net addresses.
 Can I place Fork right after UDP and get the equivalent of a UDP socket
(and maybe add back some protocols if they make sense UFC/FRAG?)?  Would
that eliminate a bunch of overhead vs the Unreliable flag?  On Android I'm
very sensitive to transient/copied objects since they cause more frequent
GC and apparently method calls are more expensive than normal Java.

Thanks,

JT


On Mon, Apr 14, 2014 at 3:25 AM, Bela Ban <be...@ya...> wrote:

> Hi Jim
>
> sorry for the delay, I've been on PTO. Comments below.
>
> On 10/04/14 03:28, Jim Thomas wrote:
> > I need multiple ReplicatedHashMaps in what would otherwise be a single
> > channel cluster (all nodes need the exact same interface and have a
> > fixed configuration for the duration of connection to the cluster).
> > Thus far I have used multiple channels on a shared transport to
> > accomplish this successfully.  It is not clear to me how much extra
> > overhead I'm suffering due to having multiple channels for this vs a
> > single channel (discovery, membership, etc. are repeated right?), but
> > the allure of an even lighter weight implementation using forked
> > channels led me to investigate that possibility.
>
>
> You're right: while a shared transport reuses thread pools and sockets
> provided by the transport, the stacks above the transport are not shared
> and therefore protocols like FD_SOCK / FD_ALL etc all do their own thing
> and have duplicate resources.
>
>
> >  Very quickly I discovered that forked channels don't implement getState
>  (but if I
> > understand the reason behind that right that is partially why a forked
> > channel is attractive for my topology) but ReplicatedHashMap uses that
> > to synchronize with the cluster when it first joins.
>
> Right. The nature of fork channels doesn't render them useful for
> maintaining separate state (from the state of the main channel).
> Depending on what requirements you have wrt state transfer and messages
> delivery, you could implement state yourself. E.g. if your ops are
> idempotent and duplicates are not an issue, you could implement state
> transfer via RPCs invoked over the fork channel.
>
> Another option is to look into using Infinispan [1]. It allows you to
> maintain a number of caches over the same shared channel. State transfer
> is done per cache, so this is exactly what you want.
>
> > My application runs over wifi so I'm very sensitive to extra network
> > traffic.  So I've been contemplating modifying ReplicatedHashMap to
> > either work with multiple on a single channel
>
> +1, this would be more or less what Infinispan does. However, you lose
> some guarantees JGroups state transfer makes; namely the ordering
> between state transfer and modifications.
>
>
> > or alternatively work with
> > forked channels.  For a single channel I think I can make it work by
> > switching to MuxRpcDispatcher and assigning a map number to each
> > ReplicatedHashMap but I have not figured out how to fix getState /
> > setState (can those be muxed?).  For forked channels I'd have to add
> > getState / setState to the RPC and call that from start() instead of
> > using state for synchronization.  Do you have any opinions on what way
> > makes more sense?  Or is the multiple channels on a shared transport not
> > as bad as I'm thinking?
>
> I'd recommend picking the *multiple caches on a shared channel*
> paradigm, see above. You can ping me on IRC (#jgroups) if you want to
> discuss this interactively.
>
>
> > I think I've also found a bug.
>
>
> Can you create a JIRA issue, so I can look into this ? Please don't
> forget to attach your sanitized XML config file(s).
>
>
> > When setup a fork channel I get an error
> > if I don't have udp.xml but when I don't have a fork channel things work
> > fine with my main-stack.xml protocol file.  Literally if I just copy my
> > main-stack.xml to udp.xml and change nothing else it will not crash.
> >   Here is the exception:
> > 04-09 17:33:38.270: W/System.err(5478): java.io.FileNotFoundException:
> > JGRP000003: file "udp.xml" not found
> > 04-09 17:33:38.270: W/System.err(5478): at
> >
> org.jgroups.conf.ConfiguratorFactory.getXmlConfigurator(ConfiguratorFactory.java:211)
> > 04-09 17:33:38.270: W/System.err(5478): at
> >
> org.jgroups.conf.ConfiguratorFactory.getStackConfigurator(ConfiguratorFactory.java:102)
> > 04-09 17:33:38.270: W/System.err(5478): at
> > org.jgroups.JChannel.<init>(JChannel.java:172)
> > 04-09 17:33:38.270: W/System.err(5478): at
> > org.jgroups.JChannel.<init>(JChannel.java:123)
> > 04-09 17:33:38.270: W/System.err(5478): at
> > org.jgroups.fork.ForkChannel.<init>(ForkChannel.java:75)
> > 04-09 17:33:38.270: W/System.err(5478): at
> > org.jgroups.fork.ForkChannel.<init>(ForkChannel.java:118)
> > 04-09 17:33:38.270: W/System.err(5478): at
> >
> com.novawurks.jgroupstest.JGroupsTestActivity$JGroupsSetupThread.run(JGroupsTestActivity.java:119)
> >
> > Obviously I can just use udp.xml for my file name but I thought I'd pass
> > this along.
> >
> > I'm running 3.4.1 ported to Android.
>
> OK. This excludes the Infinispan solution, as Infinispan hasn't been
> ported to Android...
>
> [1] www.infinispan.org
>
> --
> Bela Ban, JGroups lead (http://www.jgroups.org)
>
>
> ------------------------------------------------------------------------------
> Learn Graph Databases - Download FREE O'Reilly Book
> "Graph Databases" is the definitive new guide to graph databases and their
> applications. Written by three acclaimed leaders in the field,
> this first edition is now available. Download your free book today!
> http://p.sf.net/sfu/NeoTech
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>