Re: [jgroups-users] JGroups 3.5.0.Beta7

[Bela B. <be...@ya...>]

On 05/06/14 15:12, Bram Klein Gunnewiek wrote:
> Hm, using getBuffer(); works.

If you use getRawBuffer(), you also have to take getOffset() and 
getLength() into account. You might have started reading from the wrong 
offset (0) or you could have read beyond the end of the buffer (length).


> Everything in the application now works
> but we don't use the specific discovery protocols you mentioned so I
> can't comment on that. Things we noticed:
>
>   * org.jgroups.util.Util.readString(DataInput); and
>     writeString(DataOutput) where removed (no idea why)

They were moved to Bits. Util has started to become to large, so I 
wanted to move at least some methods to different classes.

>   * we previously used getRawBuffer() to get the byte buffer. We had to
>     change that to getBuffer();

See above. I'm sure you didn't use offset or length. Prior to 3.5, 
offset was always 0 but this has change now as the message passing got 
rid of message copying. This is more efficient.

>   * Field PING.timeout is deprecated: GMS.join_timeout should be used
>     instead
>   * Field PING.num_initial_members is deprecated: will be ignored
>   * Field UDP.loopback is deprecated: enabled by default


Yes, these fields have been deprecated, what about it ?


> What we also noticed during this upgrade whas that our nodes could not
> find eachother anymore when forcing IPv4 like this:
>
> System.setProperty("java.net.preferIPv4Stack", "true");


I don't think this works; you have to pass the system props to the JVM. 
Once the JVM has been started, this has no effect.


> We tried that because, although we bind on an IPv4 address, JGroups
> builds an IPv6 stack:
>
> 13:13:26.767 [main] DEBUG org.jgroups.stack.Configurator - set property
> UDP.mcast_group_addr to default value /ff0e:0:0:0:0:8:8:8
> 13:13:26.768 [main] DEBUG org.jgroups.stack.Configurator - set property
> UDP.diagnostics_addr to default value /ff0e:0:0:0:0:0:75:75


By default the JVM tries to use an IPv6 stack. You can force IPv4 by 
using system
java -Djava.net.preferIPv4Stack=true classname


> 13:13:26.789 [main] DEBUG org.jgroups.protocols.UDP - sockets will use
> interface 192.168.8.139


Aha, setting the system prop programmatically apparently works in your 
JVM (perhaps they changed this between JDK versions) ! You probably set 
this halfway through startup... ?

> E.G. /*without*/ setting the prefer IPv4Stack to true it works, if we
> set it to true nodes can't find each other.

Both IPv4 and IPv6 have been tested to work, so perhaps you have IPv4 
addrs in your XML file but set some addrs via system props ?


> On 06/05/2014 02:44 PM, Bram Klein Gunnewiek wrote:
>> We wrote our own Address implementation and converted our old code to be
>> compatible with 3.5.X. We are having errors deserializing our received
>> messages. Our implementation comes down to this:
>>
>>       public void receive(Message msg) {
>>           Object data = null;
>>           try {
>>               data = serializer.deserialize(msg.getRawBuffer());
>>               if (null == data) {
>>                   logger.warn(String.format("Serializer returned null on
>> deserialisation for received message %s. Not forwarding message", msg));
>>               }
>>           } catch (Exception e) {
>>               e.printStackTrace();
>>               logger.trace(String.format("Deserializing JGroups message
>> %s failed (not serialized with our serializer?)", msg), e);
>>           }
>>       ....
>>
>> However, on the receiving end the byte buffer we get from
>> msg.getRawBuffer() results in data thats not de-serializable:
>>
>> java.io.StreamCorruptedException: invalid stream header: 19400206
>>
>> We serialize/deserialize with our own serializer that uses regular java
>> serialisation/deserialisation. Deserialise goes like this:
>>
>>               ObjectInputStream ois = new ObjectInputStream(new
>> ByteArrayInputStream(serialized));
>>               @SuppressWarnings("unchecked")
>>               T ret = (T) ois.readObject();
>>               ois.close();
>>
>> If we broadcast a message the message can be succesfully deserialised by
>> the instance thats actually sending the broadcast. Has something changed
>> in 3.5 that could be causing this problem?
>>
>>
>> On 05/28/2014 04:38 PM, Bela Ban wrote:
>>> Great ! Make sure to test discovery if you happen to use some of the
>>> more exotic discovery protocols such as RACKSPACE_PING, SWIFT_PING etc...
>>>
>>> On 28/05/14 16:21, Bram Klein Gunnewiek wrote:
>>>> 3.4.4.Final. Our software is not in production yet and we try to update
>>>> as much as possbile to ship with the latest version once ready.
>>>>
>>>> On 05/28/2014 04:17 PM, Bela Ban wrote:
>>>>> Excellent ! Which version are you upgrading from ?
>>>>>
>>>>>
>>>>> On 28/05/14 14:29, Bram Klein Gunnewiek wrote:
>>>>>> On 05/28/2014 02:04 PM, Bela Ban wrote:
>>>>>>> On 28/05/14 13:53, Bram Klein Gunnewiek wrote:
>>>>>>>> Is the PayloadUUID already removed/replaced in this version?
>>>>>>> No, only deprecated. But its implementation was changed to extend
>>>>>>> ExtendedUUID. I'll remove it in 4.0.
>>>>>>>
>>>>>>>> There was a thread about that on this newsgroup that it would be removed in 3.5 but
>>>>>>>> I can't find it. If the PayloadUUID is replaced (and the replacement is
>>>>>>>> final) we will make our application compatible with 3.5 and start using it
>>>>>>> You have a choice of continuing to use PayloadUUID or switch to
>>>>>>> ExtendedUUID in 3.5...
>>>>>> Thanks, we will change our implementation to ExtendedUUID and start
>>>>>> using 3.5 somewhere next week.
>>>>>>>> Met vriendelijke groet / Kind regards,
>>>>>>>> Bram Klein Gunnewiek | Shock Media B.V.
>>>>>>>>
>>>>>>>> Tel: +31 (0)546 - 714360
>>>>>>>> Fax: +31 (0)546 - 714361
>>>>>>>> Web:https://www.shockmedia.nl/
>>>>>>>>
>>>>>>>> On 05/28/2014 01:47 PM, Bela Ban wrote:
>>>>>>>>> FYI,
>>>>>>>>>
>>>>>>>>> I just released Beta7; it contains Tristan's SASL related changed and my
>>>>>>>>> monster commit for JGRP-1826 [1].
>>>>>>>>>
>>>>>>>>> JGRP-1826 contains a lot of changes to the discovery protocols, and
>>>>>>>>> forms the basis to make discovery of file based (FILE_PING) or cloud
>>>>>>>>> based stores (S3_PING, GOOGLE_PING) much faster, especially for large
>>>>>>>>> clusters.
>>>>>>>>>
>>>>>>>>> However, it won't be until JGRP-1841 [1] for these optimizations to come
>>>>>>>>> into play.
>>>>>>>>>
>>>>>>>>> Nevertheless I wanted to merge this monster commit and tackle the
>>>>>>>>> remaining work in new branches.
>>>>>>>>>
>>>>>>>>> It also removes dependencies between merge and discovery protocols; the
>>>>>>>>> code is much cleaner now.
>>>>>>>>>
>>>>>>>>> I tested the following discovery protocols with UDP(ip_mcast=false) and
>>>>>>>>> TCP for discovery and merging (MERGE3):
>>>>>>>>> - PING
>>>>>>>>> - MPING
>>>>>>>>> - TCPPING
>>>>>>>>> - TCPGOSSIP
>>>>>>>>> - FILE_PING
>>>>>>>>> - GOOGLE_PING
>>>>>>>>> - JDBC_PING
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> *Not* tested were S3_PING, SWIFT_PING and RACKSPACE_PING.
>>>>>>>>>
>>>>>>>>> S3_PING *should* work, as it contains almost all functionality of
>>>>>>>>> GOOGLE_PING (which extends it), but it would nevertheless be good if we
>>>>>>>>> could test this.
>>>>>>>>>
>>>>>>>>> Volunteers ?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> [1]https://issues.jboss.org/browse/JGRP-1826
>>>>>>>>> [2]https://issues.jboss.org/browse/JGRP-1841



-- 
Bela Ban, JGroups lead (http://www.jgroups.org)