javagroups-users — Questions and problems related to using JavaGroups

Re: [jgroups-users] Reducing bandwidth usage

[Questions/problems r. to u. J. <jav...@li...>]

I think I've answered this one, haven't I?

Increase FD_ALL/STABLE timeouts. Try to batch as much as possible etc

On 01.04.20 19:17, Questions/problems related to using JGroups wrote:
> Also is there a way to limit packets and/or bandwidth used for udp 
> transmissions?
> 
> On Wed, Apr 1, 2020 at 12:17 PM Questions/problems related to using 
> JGroups <jav...@li... 
> <mailto:jav...@li...>> wrote:
> 
>     Thanks!
> 
>     What's the differences between FD_ALL, FD_ALL2 and FD_ALL3? It's not
>     clear from the javadocs
> 
>     On Wed, Apr 1, 2020 at 10:01 AM Questions/problems related to using
>     JGroups <jav...@li...
>     <mailto:jav...@li...>> wrote:
> 
> 
> 
>         On 01.04.20 15:53, Questions/problems related to using JGroups
>         wrote:
>          > I'm using a slightly modified version of "udp.xml" that
>         basically
>          > increases some of the timeouts. I've been wiresharking some
>         of the
>          > traffic to get an idea of bandwidth usage and consumption.
>          >
>          > a) is there a supported wireshark plugin to help decode the
>         jgroups
>          > messages?
> 
>         There used to be one (written in C), but because the wire format
>         kept
>         changing, we decided it was better to use the actual parsing
>         code to
>         implement a PCAP/NG parser. See [1] for details.
> 
>          > b) I'm noticing about 3-4 packets send per second to/from 2
>         nodes with
>          > some bursts to 7 packets. Is there a way to tell what these
>         messages are?
> 
>         Yes, use [1] to display the messages.
> 
>         I suspect most of the traffic is from a heartbeat protocol
>         (FD_ALL) or
>         stability (STABLE)
> 
>          > c) regarding (b), is there a way to reduce the chatter?
> 
>         Depending on what the messages are, yes, by increasing timeouts
>         (e.g.)
>         in FD_ALL.
> 
> 
>         [1] http://belaban.blogspot.com/2019/06/network-sniffing.html
> 
> 
>          > javagroups-users mailing list
>          > jav...@li...
>         <mailto:jav...@li...>
>          > https://lists.sourceforge.net/lists/listinfo/javagroups-users
>          >
> 
>         -- 
>         Bela Ban | http://www.jgroups.org
> 
> 
> 
>         _______________________________________________
>         javagroups-users mailing list
>         jav...@li...
>         <mailto:jav...@li...>
>         https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 
>     _______________________________________________
>     javagroups-users mailing list
>     jav...@li...
>     <mailto:jav...@li...>
>     https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 
> 
> 
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org

Re: [jgroups-users] Reducing bandwidth usage

[Questions/problems r. to u. J. <jav...@li...>]

On 01.04.20 18:16, Questions/problems related to using JGroups wrote:
> Thanks!
> 
> What's the differences between FD_ALL, FD_ALL2 and FD_ALL3? It's not 
> clear from the javadocs

Same concept, but with increasing version numbers, the protocols become 
more efficient. Take a look at [1] for details

[1] https://issues.redhat.com/browse/JGRP-2451

-- 
Bela Ban | http://www.jgroups.org

Re: [jgroups-users] Reducing bandwidth usage

[Questions/problems r. to u. J. <jav...@li...>]

Regarding packet decoding, i followed the guide from the blog but got an
exception from jgroups.
ArrayIndexOutOfBoundsException -30598 called from
conf.ClassConfigurator.create like 129
Message.readHeader(Message 836)
Message.readFrom(Message:709)
....

Since I'm using a slightly different udp config file, do i need to pass
that in as an argument to the ParseMessages's class?




On Wed, Apr 1, 2020 at 10:01 AM Questions/problems related to using JGroups
<jav...@li...> wrote:

>
>
> On 01.04.20 15:53, Questions/problems related to using JGroups wrote:
> > I'm using a slightly modified version of "udp.xml" that basically
> > increases some of the timeouts. I've been wiresharking some of the
> > traffic to get an idea of bandwidth usage and consumption.
> >
> > a) is there a supported wireshark plugin to help decode the jgroups
> > messages?
>
> There used to be one (written in C), but because the wire format kept
> changing, we decided it was better to use the actual parsing code to
> implement a PCAP/NG parser. See [1] for details.
>
> > b) I'm noticing about 3-4 packets send per second to/from 2 nodes with
> > some bursts to 7 packets. Is there a way to tell what these messages are?
>
> Yes, use [1] to display the messages.
>
> I suspect most of the traffic is from a heartbeat protocol (FD_ALL) or
> stability (STABLE)
>
> > c) regarding (b), is there a way to reduce the chatter?
>
> Depending on what the messages are, yes, by increasing timeouts (e.g.)
> in FD_ALL.
>
>
> [1] http://belaban.blogspot.com/2019/06/network-sniffing.html
>
>
> > javagroups-users mailing list
> > jav...@li...
> > https://lists.sourceforge.net/lists/listinfo/javagroups-users
> >
>
> --
> Bela Ban | http://www.jgroups.org
>
>
>
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>

Re: [jgroups-users] Reducing bandwidth usage

[Questions/problems r. to u. J. <jav...@li...>]

Also is there a way to limit packets and/or bandwidth used for udp
transmissions?

On Wed, Apr 1, 2020 at 12:17 PM Questions/problems related to using JGroups
<jav...@li...> wrote:

> Thanks!
>
> What's the differences between FD_ALL, FD_ALL2 and FD_ALL3? It's not clear
> from the javadocs
>
> On Wed, Apr 1, 2020 at 10:01 AM Questions/problems related to using
> JGroups <jav...@li...> wrote:
>
>>
>>
>> On 01.04.20 15:53, Questions/problems related to using JGroups wrote:
>> > I'm using a slightly modified version of "udp.xml" that basically
>> > increases some of the timeouts. I've been wiresharking some of the
>> > traffic to get an idea of bandwidth usage and consumption.
>> >
>> > a) is there a supported wireshark plugin to help decode the jgroups
>> > messages?
>>
>> There used to be one (written in C), but because the wire format kept
>> changing, we decided it was better to use the actual parsing code to
>> implement a PCAP/NG parser. See [1] for details.
>>
>> > b) I'm noticing about 3-4 packets send per second to/from 2 nodes with
>> > some bursts to 7 packets. Is there a way to tell what these messages
>> are?
>>
>> Yes, use [1] to display the messages.
>>
>> I suspect most of the traffic is from a heartbeat protocol (FD_ALL) or
>> stability (STABLE)
>>
>> > c) regarding (b), is there a way to reduce the chatter?
>>
>> Depending on what the messages are, yes, by increasing timeouts (e.g.)
>> in FD_ALL.
>>
>>
>> [1] http://belaban.blogspot.com/2019/06/network-sniffing.html
>>
>>
>> > javagroups-users mailing list
>> > jav...@li...
>> > https://lists.sourceforge.net/lists/listinfo/javagroups-users
>> >
>>
>> --
>> Bela Ban | http://www.jgroups.org
>>
>>
>>
>> _______________________________________________
>> javagroups-users mailing list
>> jav...@li...
>> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>>
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>

Re: [jgroups-users] Reducing bandwidth usage

[Questions/problems r. to u. J. <jav...@li...>]

Thanks!

What's the differences between FD_ALL, FD_ALL2 and FD_ALL3? It's not clear
from the javadocs

On Wed, Apr 1, 2020 at 10:01 AM Questions/problems related to using JGroups
<jav...@li...> wrote:

>
>
> On 01.04.20 15:53, Questions/problems related to using JGroups wrote:
> > I'm using a slightly modified version of "udp.xml" that basically
> > increases some of the timeouts. I've been wiresharking some of the
> > traffic to get an idea of bandwidth usage and consumption.
> >
> > a) is there a supported wireshark plugin to help decode the jgroups
> > messages?
>
> There used to be one (written in C), but because the wire format kept
> changing, we decided it was better to use the actual parsing code to
> implement a PCAP/NG parser. See [1] for details.
>
> > b) I'm noticing about 3-4 packets send per second to/from 2 nodes with
> > some bursts to 7 packets. Is there a way to tell what these messages are?
>
> Yes, use [1] to display the messages.
>
> I suspect most of the traffic is from a heartbeat protocol (FD_ALL) or
> stability (STABLE)
>
> > c) regarding (b), is there a way to reduce the chatter?
>
> Depending on what the messages are, yes, by increasing timeouts (e.g.)
> in FD_ALL.
>
>
> [1] http://belaban.blogspot.com/2019/06/network-sniffing.html
>
>
> > javagroups-users mailing list
> > jav...@li...
> > https://lists.sourceforge.net/lists/listinfo/javagroups-users
> >
>
> --
> Bela Ban | http://www.jgroups.org
>
>
>
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>

Re: [jgroups-users] Reducing bandwidth usage

[Questions/problems r. to u. J. <jav...@li...>]

On 01.04.20 15:53, Questions/problems related to using JGroups wrote:
> I'm using a slightly modified version of "udp.xml" that basically 
> increases some of the timeouts. I've been wiresharking some of the 
> traffic to get an idea of bandwidth usage and consumption.
> 
> a) is there a supported wireshark plugin to help decode the jgroups 
> messages?

There used to be one (written in C), but because the wire format kept 
changing, we decided it was better to use the actual parsing code to 
implement a PCAP/NG parser. See [1] for details.

> b) I'm noticing about 3-4 packets send per second to/from 2 nodes with 
> some bursts to 7 packets. Is there a way to tell what these messages are?

Yes, use [1] to display the messages.

I suspect most of the traffic is from a heartbeat protocol (FD_ALL) or 
stability (STABLE)

> c) regarding (b), is there a way to reduce the chatter?

Depending on what the messages are, yes, by increasing timeouts (e.g.) 
in FD_ALL.


[1] http://belaban.blogspot.com/2019/06/network-sniffing.html


> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org

[jgroups-users] Reducing bandwidth usage

[Questions/problems r. to u. J. <jav...@li...>]

I'm using a slightly modified version of "udp.xml" that basically increases
some of the timeouts. I've been wiresharking some of the traffic to get an
idea of bandwidth usage and consumption.

a) is there a supported wireshark plugin to help decode the jgroups
messages?
b) I'm noticing about 3-4 packets send per second to/from 2 nodes with some
bursts to 7 packets. Is there a way to tell what these messages are?
c) regarding (b), is there a way to reduce the chatter?

Re: [jgroups-users] Callback for non delivered messages?

[Questions/problems r. to u. J. <jav...@li...>]

On 31.03.20 20:46, Questions/problems related to using JGroups wrote:
> Hang on, assuming NAKACK2...I send a unicast message to node P who drops 
> offline before the message arrives and it never comes back online, 

You mean P crashes or is killed?

> jgroups will continue to retransmit to that node forever?

No, retransmission will stop when the new view excluding P has been 
installed.
Note that for NAKACK2, we're using negative acks anyway, so P would ask 
for retransmission from the sender, so when it dies, retransmission stops.

This is different from UNICAST3, where the sender re-sends messages 
until they're acked.

> There's no limits or anything like that?

Yes, there is: failure detection timeouts

> On Tue, Mar 31, 2020 at 3:30 AM Questions/problems related to using 
> JGroups <jav...@li... 
> <mailto:jav...@li...>> wrote:
> 
> 
> 
>     On 30.03.20 21:42, Questions/problems related to using JGroups wrote:
>      > Thanks Bela
>      >
>      > I would define "delivery failure" as in, JGroups attempted to
>     deliver a
>      > message to Node X but never received an ACK or NACK for this
>     specific
>      > message.
> 
>     This is not possible, as JGroups continues to try to deliver a
>     message M
>     to any member P until the message has been delivered, or P died (1) or
>     got partitioned away (2).
> 
>     In (1), in case P is restarted it will acquire the state (in case of a
>     stateful application). This will include M (if any other member
>     received
>     it) and its effect on the state. If M was a unicast message, it won't
>     get re-delivered.
> 
>     In (2), a merge will merge the subgroups into one, including the state.
>     This doesn't apply to unicast messages such as M, either.
> 
>      > In my use case, i want to be notified of this failure to
>      > deliver so that i can persist the message and reattempt delivery
>     at a
>      > later date or when Node X comes back online.
> 
>     Something like persistent JMS messages? Doesn't exist in JGroups. It
>     could be implemented though in a separate protocol:
>     * The sender of a message persists it in a shared DB
>     * Every message carries a unique ID, every receiver sends an ACK for
>     this ID back to the sender
>     * When an ACK from all receivers (or a given receiver in the unicast
>     case) has been received, the message will be deleted in the DB
>     * On restart/merge, the coordinator sends the unacked messages to a
>     joiner
> 
> 
> 
>      > On Mon, Mar 30, 2020 at 7:44 AM Questions/problems related to using
>      > JGroups <jav...@li...
>     <mailto:jav...@li...>
>      > <mailto:jav...@li...
>     <mailto:jav...@li...>>> wrote:
>      >
>      >
>      >
>      >     On 27.03.20 19:22, Questions/problems related to using
>     JGroups wrote:
>      >      > Is there an easy way I can get notification of delivery
>     failure of a
>      >      > give message?
>      >
>      >     What's a delivery failure? At the application level? You
>     could use
>      >     RpcDispatcher and check RspList for failures (exceptions). Or
>     use the
>      >     JChannel directly and send back ack messages which are
>     collected and
>      >     checked by the sender.
>      >
>      >      > Also related, Is Channel.send a blocking method?
>      >
>      >     Not per se. However, send() can block in the following cases:
>      >     * The transport is TCP and the send blocks because of a full TCP
>      >     send-window
>      >     * Flow control (MFC/UFC) blocks because the receiver(s) don't
>     send
>      >     enough credits back
>      >
>      >      > _______________________________________________
>      >      > javagroups-users mailing list
>      >      > jav...@li...
>     <mailto:jav...@li...>
>      >     <mailto:jav...@li...
>     <mailto:jav...@li...>>
>      >      > https://lists.sourceforge.net/lists/listinfo/javagroups-users
>      >      >
>      >
>      >     --
>      >     Bela Ban | http://www.jgroups.org
>      >
>      >
>      >
>      >     _______________________________________________
>      >     javagroups-users mailing list
>      > jav...@li...
>     <mailto:jav...@li...>
>      >     <mailto:jav...@li...
>     <mailto:jav...@li...>>
>      > https://lists.sourceforge.net/lists/listinfo/javagroups-users
>      >
>      >
>      >
>      > _______________________________________________
>      > javagroups-users mailing list
>      > jav...@li...
>     <mailto:jav...@li...>
>      > https://lists.sourceforge.net/lists/listinfo/javagroups-users
>      >
> 
>     -- 
>     Bela Ban | http://www.jgroups.org
> 
> 
> 
>     _______________________________________________
>     javagroups-users mailing list
>     jav...@li...
>     <mailto:jav...@li...>
>     https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 
> 
> 
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org

Re: [jgroups-users] Callback for non delivered messages?

[Questions/problems r. to u. J. <jav...@li...>]

Hang on, assuming NAKACK2...I send a unicast message to node P who drops
offline before the message arrives and it never comes back online, jgroups
will continue to retransmit to that node forever? There's no retransmit
limits or anything like that?

On Tue, Mar 31, 2020 at 3:30 AM Questions/problems related to using JGroups
<jav...@li...> wrote:

>
>
> On 30.03.20 21:42, Questions/problems related to using JGroups wrote:
> > Thanks Bela
> >
> > I would define "delivery failure" as in, JGroups attempted to deliver a
> > message to Node X but never received an ACK or NACK for this specific
> > message.
>
> This is not possible, as JGroups continues to try to deliver a message M
> to any member P until the message has been delivered, or P died (1) or
> got partitioned away (2).
>
> In (1), in case P is restarted it will acquire the state (in case of a
> stateful application). This will include M (if any other member received
> it) and its effect on the state. If M was a unicast message, it won't
> get re-delivered.
>
> In (2), a merge will merge the subgroups into one, including the state.
> This doesn't apply to unicast messages such as M, either.
>
> > In my use case, i want to be notified of this failure to
> > deliver so that i can persist the message and reattempt delivery at a
> > later date or when Node X comes back online.
>
> Something like persistent JMS messages? Doesn't exist in JGroups. It
> could be implemented though in a separate protocol:
> * The sender of a message persists it in a shared DB
> * Every message carries a unique ID, every receiver sends an ACK for
> this ID back to the sender
> * When an ACK from all receivers (or a given receiver in the unicast
> case) has been received, the message will be deleted in the DB
> * On restart/merge, the coordinator sends the unacked messages to a joiner
>
>
>
> > On Mon, Mar 30, 2020 at 7:44 AM Questions/problems related to using
> > JGroups <jav...@li...
> > <mailto:jav...@li...>> wrote:
> >
> >
> >
> >     On 27.03.20 19:22, Questions/problems related to using JGroups wrote:
> >      > Is there an easy way I can get notification of delivery failure
> of a
> >      > give message?
> >
> >     What's a delivery failure? At the application level? You could use
> >     RpcDispatcher and check RspList for failures (exceptions). Or use the
> >     JChannel directly and send back ack messages which are collected and
> >     checked by the sender.
> >
> >      > Also related, Is Channel.send a blocking method?
> >
> >     Not per se. However, send() can block in the following cases:
> >     * The transport is TCP and the send blocks because of a full TCP
> >     send-window
> >     * Flow control (MFC/UFC) blocks because the receiver(s) don't send
> >     enough credits back
> >
> >      > _______________________________________________
> >      > javagroups-users mailing list
> >      > jav...@li...
> >     <mailto:jav...@li...>
> >      > https://lists.sourceforge.net/lists/listinfo/javagroups-users
> >      >
> >
> >     --
> >     Bela Ban | http://www.jgroups.org
> >
> >
> >
> >     _______________________________________________
> >     javagroups-users mailing list
> >     jav...@li...
> >     <mailto:jav...@li...>
> >     https://lists.sourceforge.net/lists/listinfo/javagroups-users
> >
> >
> >
> > _______________________________________________
> > javagroups-users mailing list
> > jav...@li...
> > https://lists.sourceforge.net/lists/listinfo/javagroups-users
> >
>
> --
> Bela Ban | http://www.jgroups.org
>
>
>
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>

Re: [jgroups-users] Callback for non delivered messages?

[Questions/problems r. to u. J. <jav...@li...>]

On 30.03.20 21:42, Questions/problems related to using JGroups wrote:
> Thanks Bela
> 
> I would define "delivery failure" as in, JGroups attempted to deliver a 
> message to Node X but never received an ACK or NACK for this specific 
> message.

This is not possible, as JGroups continues to try to deliver a message M 
to any member P until the message has been delivered, or P died (1) or 
got partitioned away (2).

In (1), in case P is restarted it will acquire the state (in case of a 
stateful application). This will include M (if any other member received 
it) and its effect on the state. If M was a unicast message, it won't 
get re-delivered.

In (2), a merge will merge the subgroups into one, including the state. 
This doesn't apply to unicast messages such as M, either.

> In my use case, i want to be notified of this failure to 
> deliver so that i can persist the message and reattempt delivery at a 
> later date or when Node X comes back online.

Something like persistent JMS messages? Doesn't exist in JGroups. It 
could be implemented though in a separate protocol:
* The sender of a message persists it in a shared DB
* Every message carries a unique ID, every receiver sends an ACK for 
this ID back to the sender
* When an ACK from all receivers (or a given receiver in the unicast 
case) has been received, the message will be deleted in the DB
* On restart/merge, the coordinator sends the unacked messages to a joiner



> On Mon, Mar 30, 2020 at 7:44 AM Questions/problems related to using 
> JGroups <jav...@li... 
> <mailto:jav...@li...>> wrote:
> 
> 
> 
>     On 27.03.20 19:22, Questions/problems related to using JGroups wrote:
>      > Is there an easy way I can get notification of delivery failure of a
>      > give message?
> 
>     What's a delivery failure? At the application level? You could use
>     RpcDispatcher and check RspList for failures (exceptions). Or use the
>     JChannel directly and send back ack messages which are collected and
>     checked by the sender.
> 
>      > Also related, Is Channel.send a blocking method?
> 
>     Not per se. However, send() can block in the following cases:
>     * The transport is TCP and the send blocks because of a full TCP
>     send-window
>     * Flow control (MFC/UFC) blocks because the receiver(s) don't send
>     enough credits back
> 
>      > _______________________________________________
>      > javagroups-users mailing list
>      > jav...@li...
>     <mailto:jav...@li...>
>      > https://lists.sourceforge.net/lists/listinfo/javagroups-users
>      >
> 
>     -- 
>     Bela Ban | http://www.jgroups.org
> 
> 
> 
>     _______________________________________________
>     javagroups-users mailing list
>     jav...@li...
>     <mailto:jav...@li...>
>     https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 
> 
> 
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org

Re: [jgroups-users] Callback for non delivered messages?

[Questions/problems r. to u. J. <jav...@li...>]

Thanks Bela

I would define "delivery failure" as in, jGroups attempted to deliver a
message to Node X but never received an ACK or NACK for this specific
message. In my use case, i want to be notified of this failure to deliver
so that i can persist the message and reattempt delivery at a later date or
when Node X comes back online.


On Mon, Mar 30, 2020 at 7:44 AM Questions/problems related to using JGroups
<jav...@li...> wrote:

>
>
> On 27.03.20 19:22, Questions/problems related to using JGroups wrote:
> > Is there an easy way I can get notification of delivery failure of a
> > give message?
>
> What's a delivery failure? At the application level? You could use
> RpcDispatcher and check RspList for failures (exceptions). Or use the
> JChannel directly and send back ack messages which are collected and
> checked by the sender.
>
> > Also related, Is Channel.send a blocking method?
>
> Not per se. However, send() can block in the following cases:
> * The transport is TCP and the send blocks because of a full TCP
> send-window
> * Flow control (MFC/UFC) blocks because the receiver(s) don't send
> enough credits back
>
> > _______________________________________________
> > javagroups-users mailing list
> > jav...@li...
> > https://lists.sourceforge.net/lists/listinfo/javagroups-users
> >
>
> --
> Bela Ban | http://www.jgroups.org
>
>
>
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>

Re: [jgroups-users] Callback for non delivered messages?

[Questions/problems r. to u. J. <jav...@li...>]

On 27.03.20 19:22, Questions/problems related to using JGroups wrote:
> Is there an easy way I can get notification of delivery failure of a 
> give message?

What's a delivery failure? At the application level? You could use 
RpcDispatcher and check RspList for failures (exceptions). Or use the 
JChannel directly and send back ack messages which are collected and 
checked by the sender.

> Also related, Is Channel.send a blocking method?

Not per se. However, send() can block in the following cases:
* The transport is TCP and the send blocks because of a full TCP send-window
* Flow control (MFC/UFC) blocks because the receiver(s) don't send 
enough credits back

> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org

[jgroups-users] Callback for non delivered messages?

[Questions/problems r. to u. J. <jav...@li...>]

Is there an easy way I can get notification of delivery failure of a give
message?

Also related, Is Channel.send a blocking method?

Re: [jgroups-users] using jgroups on mixed architectures?

[Questions/problems r. to u. J. <jav...@li...>]

On Thu, Feb 20, 2020 at 1:24 AM Questions/problems related to using JGroups
<jav...@li...> wrote:

> >
> > We're using JGroups 4.1.8, channel based on TCP stack. Some customers
> > want to use our product where nodes are on different architectures (e.g.
> > IBM Power 9 and x86). I don't see any reason that wouldn't work since we
> > just need TCP to be available on the machines, but I can only do a
> > limited amount of testing.
>
> Perhaps you can use AWS or Google cloud to get hosts with different
> architectures for testing?
>

Yep, but I don't want to test every combination of machines. If I were paid
by the hour maybe.... :)


>
> > So am asking a hopefully dumb question: assuming the same version of
> > everything is used on each node, do you know of any limitations to
> > mixing up the node types?
>
> I haven't tried this, but I don't think this should be a problem


Cool, thanks!

Cheers,
Bobby

Re: [jgroups-users] using jgroups on mixed architectures?

[Questions/problems r. to u. J. <jav...@li...>]

On 19.02.20 19:29, Questions/problems related to using JGroups wrote:
> Hi,
> 
> We're using JGroups 4.1.8, channel based on TCP stack. Some customers 
> want to use our product where nodes are on different architectures (e.g. 
> IBM Power 9 and x86). I don't see any reason that wouldn't work since we 
> just need TCP to be available on the machines, but I can only do a 
> limited amount of testing.

Perhaps you can use AWS or Google cloud to get hosts with different 
architectures for testing?

> So am asking a hopefully dumb question: assuming the same version of 
> everything is used on each node, do you know of any limitations to 
> mixing up the node types?

I haven't tried this, but I don't think this should be a problem

> Thanks,
> Bobby
> 
> 
> 
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org

[jgroups-users] using jgroups on mixed architectures?

[Questions/problems r. to u. J. <jav...@li...>]

Hi,

We're using JGroups 4.1.8, channel based on TCP stack. Some customers want
to use our product where nodes are on different architectures (e.g. IBM
Power 9 and x86). I don't see any reason that wouldn't work since we just
need TCP to be available on the machines, but I can only do a limited
amount of testing.

So am asking a hopefully dumb question: assuming the same version of
everything is used on each node, do you know of any limitations to mixing
up the node types?

Thanks,
Bobby

[jgroups-users] Anyone using ExecutionService?

[Questions/problems r. to u. J. <jav...@li...>]

Is anyone using ExecutionService [1]?

I've never seen any questions about it, so I assume nobody uses it. In 
that case, I'd like to pull it?

Any disagreement?

[1] http://www.jgroups.org/manual5/index.html#ExecutionService

-- 
Bela Ban | http://www.jgroups.org

Re: [jgroups-users] Porting-Guide/Release-Notes between 3.6.14 and 4.0.15

[Questions/problems r. to u. J. <jav...@li...>]

>
>
> On 27.01.20 15:49, Questions/problems related to using JGroups wrote:
> > Hi:
> >   Can someone point me to a porting guide - trying to port code between
> > 3.6.14 and 4.0.15.  (e.g. things that will break and must be chagned
> between 3 and 4?)
>
> There is not porting guide. I suggest recompile your app against 4.x and
> fix the compilation errors - it's as simple as that.
>

We ran into some issues, but I don't remember the details now *except* our
main class, defined with:

extends ReceiverAdapter implements RequestHandler

...used to be able to implement receive() (for simpler messages) and
handle() (for things that needed to return info). With 4.x none of the
receive calls worked any more. It had something to do with us using a
MessageDispatcher, but I don't recall the details. We had to pull out
receive/cast message and move all those things. The questions I had are in
the users archive back around Nov 2017 and have "v3->v4" in the subject if
it helps.

There was also a change that our objects needed to move to streamable
instead of serializable, which wasn't hard, but I don't remember it being
obvious.

I can go back and try to find all my commits to see if there's anything
useful there, but anything I couldn't figure out easily is in the users
list already with that string in the subject.

Cheers,
Bobby

Re: [jgroups-users] Porting-Guide/Release-Notes between 3.6.14 and 4.0.15

[Questions/problems r. to u. J. <jav...@li...>]

On 27.01.20 15:49, Questions/problems related to using JGroups wrote:
> Hi:
>   Can someone point me to a porting guide - trying to port code between 
> 3.6.14 and 4.0.15.  (e.g. things that will break and must be chagned between 3 and 4?)

There is not porting guide. I suggest recompile your app against 4.x and 
fix the compilation errors - it's as simple as that.

>   We use the jgroups library in a "deployed" application on the JBoss 
> EAP7 platform, currently trying to
>   upgrade between EAP7.1.x and 7.2.X, the JBoss upgrade changes jgroups 
> between 3.6.14 and 4.0.15.
> 
>   Some of the inital issues was are facing include, any 
> hints/suggestions would be greatly appreciated.
> 
>   a) Issue related to:
> https://issues.redhat.com/browse/JGRP-1860
> 
>   Bela already provided an inital answer.  See [1] below for more details.
> 
>   b)  org.jgroups.util.UUID is NOT backwards compatibile
>   See [2] for more details.
> 
>   c) Channel deprecated.  Can we just change all uses of "Channel" to 
> "JChannel" and all will be OK?


Yes. Note that I'll probably re-activate Channel in 5.0: 
https://issues.redhat.com/browse/JGRP-2405


>   d) org.jgroups.timeoutexception was removed.  Can I simply import 
> java.util.concurrent.timeoutexception in
>   place of org.jgroups.timeoutexception?

Yes.


>   [1] In jgroups-3.6.14.Final-redhat-1.jar (EAP7.1.4) 
> org.jgroups.blocks.RpcDispatcher had a static interface defined.
> 
>      public static interface Marshaller {
>          public Buffer objectToBuffer(Object var1) throws Exception;
> 
>          public Object objectFromBuffer(byte[] var1, int var2, int var3) 
> throws Exception;
>      }
> 
> In 4.0.15 there is now a Interface Marshaller with completely different 
> methods/parameters.
> Any suggestions?

objectToBuffer(obj) used to return a byte array (Buffer), now it needs 
to write obj to an output stream. The reason this was done is that we 
can eliminate one memory allocation. You could use Util.objectToStream() 
to implement this but since you know your application, perhaps the 
(de-)serialization can be optimized.

objectFromStream can also used Util.objectFromStream().


> [2] With jgroups jgroups-3.6.14.Final-redhat-1.jar (EAP7.1.4) we 
> extended org.jgroups.util.UUID to
> create our ToddsPayloadUUID and add
> String(s) ipAddress, port and hostname (which are used by our application).
> 
> Unfortunately in 4.0.15 org.jgroups.util.UUID is missing some methods we 
> used, including:
> size(), readExternal(), writeExternal(), writeTo(), readFrom()

Not true!

UUID extends Address implements SizeStreamable extends Streamable.
* size() is now serializedSize()
* writeTo() and readFrom() are also defined in UUID
* readExternal() and writeExternal() have been removed as UUID does not 
implement Serializable. But why do you need them? Use readFrom() and 
writeTo(); these are more efficient anyway!


-- 
Bela Ban | http://www.jgroups.org

[jgroups-users] Airhacks podcast with Adam Bien

[Questions/problems r. to u. J. <jav...@li...>]

FYI: airhacks.fm/#episode_72

-- 
Bela Ban | http://www.jgroups.org

[jgroups-users] First release of 5.0.0.Alpha1

[Questions/problems r. to u. J. <jav...@li...>]

FYI:

[1] http://belaban.blogspot.com/2020/01/first-alpha-of-jgroups-50.html

-- 
Bela Ban | http://www.jgroups.org

[jgroups-users] Porting-Guide/Release-Notes between 3.6.14 and 4.0.15

[Questions/problems r. to u. J. <jav...@li...>]

Hi:
 Can someone point me to a porting guide - trying to port code between
3.6.14 and 4.0.15.
 (e.g. things that will break and must be chagned between 3 and 4?)

 We use the jgroups library in a "deployed" application on the JBoss EAP7
platform, currently trying to
 upgrade between EAP7.1.x and 7.2.X, the JBoss upgrade changes jgroups
between 3.6.14 and 4.0.15.

 Some of the inital issues was are facing include, any hints/suggestions
would be greatly appreciated.

 a) Issue related to:
 https://issues.redhat.com/browse/JGRP-1860

 Bela already provided an inital answer.  See [1] below for more details.

 b)  org.jgroups.util.UUID is NOT backwards compatibile
 See [2] for more details.

 c) Channel deprecated.  Can we just change all uses of "Channel" to
"JChannel" and all will be OK?

 d) org.jgroups.timeoutexception was removed.  Can I simply import
java.util.concurrent.timeoutexception in
 place of org.jgroups.timeoutexception?

 e) ... tbd

 Cheers


 [1] In jgroups-3.6.14.Final-redhat-1.jar (EAP7.1.4)
org.jgroups.blocks.RpcDispatcher had a static interface defined.

    public static interface Marshaller {
        public Buffer objectToBuffer(Object var1) throws Exception;

        public Object objectFromBuffer(byte[] var1, int var2, int var3)
throws Exception;
    }

In 4.0.15 there is now a Interface Marshaller with completely different
methods/parameters.
Any suggestions?

[2] With jgroups jgroups-3.6.14.Final-redhat-1.jar (EAP7.1.4) we extended
org.jgroups.util.UUID to
create our ToddsPayloadUUID and add
String(s) ipAddress, port and hostname (which are used by our application).

Unfortunately in 4.0.15 org.jgroups.util.UUID is missing some methods we
used, including:
size(), readExternal(), writeExternal(), writeTo(), readFrom()

When we create a JChannel, we add the class to the ClassConfigurator with:

 org.jgroups.conf.ClassConfigurator.add(1025, ToddsPayloadUUID.class);

We then call the create/add and AddressGenerator to the channel - code
snippet:


    private static Channel createSamChannel (String aInProps)
    {
...

 try
        {
            myC = new JChannel(aInProps);
            createSamChannelLogger.logActivity("Create JChannel with
config=" + aInProps);
            System.out.println("Create JChannel with config=" + aInProps);
            myC.addAddressGenerator(new AddressGenerator()
            {
                // Set Address Generator to SamPayloadUUID and set
IpAddress and port to be what is
                //configured in the jgroup configuration
                public Address generateAddress()
                {
                    Map
lTcpMap=com.timetra.nms.server.multiserver.core.common.Util.getJGroupElement("TCP");
                    String ipAddress =(String) lTcpMap.get("bind_addr");
                    String port = (String) lTcpMap.get("bind_port");
                    String hostname;
                    try
                    {
                        hostname =
InetAddress.getByName(ipAddress).getHostName();
                    }
                    catch (UnknownHostException e)
                    {
                        hostname = ipAddress;
                    }
                    return (Address)SamPayloadUUID.randomUUID(ipAddress,
port, hostname);
                }
            });
        }

Re: [jgroups-users] suggestions for working with OpenStack (NAT question)

[Questions/problems r. to u. J. <jav...@li...>]

On 21.01.20 15:49, Questions/problems related to using JGroups wrote:
> Thanks for this.
> 
> I didn't know about the external_addr field (I need to take some time 
> off and go through all the jgroups manuals again....). I think that 
> could solve the problem for us very easily.

Keeping my fingers crossed.

Note that 5.0.0.Alpha1 will be released very soon, as soon as I have the 
manual finished...

> Will hopefully get a chance to try this soon so we at least have a plan 
> for the customer asking about it.

Let me know whether/how this works,
Cheers,

> Thanks!
> Bobby
> 
> 
> On Tue, Jan 21, 2020 at 7:21 AM Questions/problems related to using 
> JGroups <jav...@li... 
> <mailto:jav...@li...>> wrote:
> 
> 
> 
>     On 20.01.20 18:55, Questions/problems related to using JGroups wrote:
>      > Hi,
>      >
>      > A customer wants to use our product on OpenStack, which (I assume
>     you know)
> 
>     I know it by name, but I've never used it
> 
>      > has private addresses for every node and then a floating IP
>      > address which can be reached everywhere. So they have nodes "not
>     on the
>      > same network, on different cloud providers / zones with incompatible
>      > addresses scheme."
>      >
>      > We're using a tcp stack, jgroups 4.1.8, and TCP#bind_addr has to
>     be set
>      > to the local, private, address or else jgroups can't bind to it. But
>      > then other nodes can't reach that node. Do you have suggestions
>     about
>      > how to make this work?
> 
>     Yes, you could bind JGroups to the private IP address and set
>     external_addr to the public IP address. This means JGroups will bind to
>     bind_addr:bind_port, but expose its address (in the discovery process)
>     as external_addr:(external_port). Note that if the latter is not
>     set, it
>     will be the same value as bind_port (even if it is 0; then the
>     ephemeral
>     port will be used).
> 
>     In TCPPING.initial_hosts, you'd list the *public* (external_addr) IP
>     addresses:ports
> 
>      > I don't have access to an OpenStack setup like this to test; I
>     can only
>      > test in one region, so I don't know if using floating IPs for
>      > TCPPING#initial_hosts should work or not for the nodes to find each
>      > other. (If so, that's great, but would mean a lot of work for us
>     to have
>      > the nodes carry around a 2nd IP address to pass to each other for
>     other
>      > reasons.)
> 
>     That should work, and I don't think you need to pass around a second IP
>     address, as all nodes are only known by their public IPs.
> 
>     As an alternative, which works across different cloud regions and cloud
>     providers, you could use the TUNNEL/GossipRouter combo, see [1] fpr
>     details.
>     Cheers,
> 
>     [1]
>     http://belaban.blogspot.com/2019/12/spanning-jgroups-kubernetes-based.html
> 
>      > Thanks,
>      > Bobby
>      >
>      >
>      >
>      > _______________________________________________
>      > javagroups-users mailing list
>      > jav...@li...
>     <mailto:jav...@li...>
>      > https://lists.sourceforge.net/lists/listinfo/javagroups-users
>      >
> 
>     -- 
>     Bela Ban | http://www.jgroups.org
> 
> 
> 
>     _______________________________________________
>     javagroups-users mailing list
>     jav...@li...
>     <mailto:jav...@li...>
>     https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 
> 
> 
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org

Re: [jgroups-users] suggestions for working with OpenStack (NAT question)

[Questions/problems r. to u. J. <jav...@li...>]

Thanks for this.

I didn't know about the external_addr field (I need to take some time off
and go through all the jgroups manuals again....). I think that could solve
the problem for us very easily.

Will hopefully get a chance to try this soon so we at least have a plan for
the customer asking about it.

Thanks!
Bobby


On Tue, Jan 21, 2020 at 7:21 AM Questions/problems related to using JGroups
<jav...@li...> wrote:

>
>
> On 20.01.20 18:55, Questions/problems related to using JGroups wrote:
> > Hi,
> >
> > A customer wants to use our product on OpenStack, which (I assume you
> know)
>
> I know it by name, but I've never used it
>
> > has private addresses for every node and then a floating IP
> > address which can be reached everywhere. So they have nodes "not on the
> > same network, on different cloud providers / zones with incompatible
> > addresses scheme."
> >
> > We're using a tcp stack, jgroups 4.1.8, and TCP#bind_addr has to be set
> > to the local, private, address or else jgroups can't bind to it. But
> > then other nodes can't reach that node. Do you have suggestions about
> > how to make this work?
>
> Yes, you could bind JGroups to the private IP address and set
> external_addr to the public IP address. This means JGroups will bind to
> bind_addr:bind_port, but expose its address (in the discovery process)
> as external_addr:(external_port). Note that if the latter is not set, it
> will be the same value as bind_port (even if it is 0; then the ephemeral
> port will be used).
>
> In TCPPING.initial_hosts, you'd list the *public* (external_addr) IP
> addresses:ports
>
> > I don't have access to an OpenStack setup like this to test; I can only
> > test in one region, so I don't know if using floating IPs for
> > TCPPING#initial_hosts should work or not for the nodes to find each
> > other. (If so, that's great, but would mean a lot of work for us to have
> > the nodes carry around a 2nd IP address to pass to each other for other
> > reasons.)
>
> That should work, and I don't think you need to pass around a second IP
> address, as all nodes are only known by their public IPs.
>
> As an alternative, which works across different cloud regions and cloud
> providers, you could use the TUNNEL/GossipRouter combo, see [1] fpr
> details.
> Cheers,
>
> [1]
> http://belaban.blogspot.com/2019/12/spanning-jgroups-kubernetes-based.html
>
> > Thanks,
> > Bobby
> >
> >
> >
> > _______________________________________________
> > javagroups-users mailing list
> > jav...@li...
> > https://lists.sourceforge.net/lists/listinfo/javagroups-users
> >
>
> --
> Bela Ban | http://www.jgroups.org
>
>
>
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
>

Re: [jgroups-users] suggestions for working with OpenStack (NAT question)

[Questions/problems r. to u. J. <jav...@li...>]

On 20.01.20 18:55, Questions/problems related to using JGroups wrote:
> Hi,
> 
> A customer wants to use our product on OpenStack, which (I assume you know)

I know it by name, but I've never used it

> has private addresses for every node and then a floating IP 
> address which can be reached everywhere. So they have nodes "not on the 
> same network, on different cloud providers / zones with incompatible 
> addresses scheme."
> 
> We're using a tcp stack, jgroups 4.1.8, and TCP#bind_addr has to be set 
> to the local, private, address or else jgroups can't bind to it. But 
> then other nodes can't reach that node. Do you have suggestions about 
> how to make this work?

Yes, you could bind JGroups to the private IP address and set 
external_addr to the public IP address. This means JGroups will bind to 
bind_addr:bind_port, but expose its address (in the discovery process) 
as external_addr:(external_port). Note that if the latter is not set, it 
will be the same value as bind_port (even if it is 0; then the ephemeral 
port will be used).

In TCPPING.initial_hosts, you'd list the *public* (external_addr) IP 
addresses:ports

> I don't have access to an OpenStack setup like this to test; I can only 
> test in one region, so I don't know if using floating IPs for 
> TCPPING#initial_hosts should work or not for the nodes to find each 
> other. (If so, that's great, but would mean a lot of work for us to have 
> the nodes carry around a 2nd IP address to pass to each other for other 
> reasons.)

That should work, and I don't think you need to pass around a second IP 
address, as all nodes are only known by their public IPs.

As an alternative, which works across different cloud regions and cloud 
providers, you could use the TUNNEL/GossipRouter combo, see [1] fpr details.
Cheers,

[1] 
http://belaban.blogspot.com/2019/12/spanning-jgroups-kubernetes-based.html

> Thanks,
> Bobby
> 
> 
> 
> _______________________________________________
> javagroups-users mailing list
> jav...@li...
> https://lists.sourceforge.net/lists/listinfo/javagroups-users
> 

-- 
Bela Ban | http://www.jgroups.org