Re: [javagroups-users] Using Jgroups in a large scale environment

[Bela B. <be...@ya...>]

Benoît wrote:
> Hi,
>
> We have been using JGroups for 3 years: several dozens of hosts are
> connected to and communicate with each other through an IP multicast
> transport based channel. It works fine.
> One of next step is to increase the number of members, even up to hundreds
> of hosts (and more !). I know you are working on many enhancements to be
> suited to large scale solutions.

Yes, the latest issue was created just today [1]. The overall JIRA issue 
is [2].

This is an ongoing process. JGroups was originally designed for 10s of 
nodes, not hundreds. But, over time, requirements have changed, and 
folks are starting to use JGroups in large(r) scale scenarios.

> So the following questions:
> - Have you had the opportunity of performing any test in a large scale 
> environment ?

No, unfortunately I don't have access to a lab with a large number of 
hosts... :-(

> if not, do you know if somebody has already done it ?

I've gotten emails from people who have successfully used JGroups in a 
cluster of several hundred nodes (300-400). Mike Jensen's work on 
treemesh is going to several 1000s of nodes, but at the expense of 
reliability (more peer-to-peer semantics, for dissemination of 
video/audio to large number of nodes).

> - Do you think JGroups can be used to connect several hundreds of 
> hosts within the same cluster ?

Yes, but definitely with configuration changes (the configs shipped with 
JGroups assume clusters of 4-20 nodes) and most probably with protocol 
changes as well...

Thye introduction of ergonomics [3] is also important, as it will 
dynamically change properties in several protocol, e.g. the thread pool 
size will be adapted as a function of the cluster size. Other properties 
to be dynamically changed ar flow control (number of credits), stability 
messages, failure detection etc

> What do you think about it ?

For different reasons (outlined in [4]), I don't think a large flat 
cluster consisting of hundreds of nodes make sense. Upgrading the HW/SW 
will lead to downtime, so in [4] I suggested to divide large clusters 
into mod-cluster domains.

> Thanks for your responses.
> Regards,

[1] https://jira.jboss.org/browse/JGRP-1241
[2] https://jira.jboss.org/browse/JGRP-100
[3] https://jira.jboss.org/browse/JGRP-1052
[4] 
http://www.redhat.com/promo/summit/2010/presentations/jbossworld/optimize-secure-manage/wed/bban-1130-dynamic/talk.pdf


-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss