Re: [javagroups-users] JGroups performance related question

[Bela B. <be...@ya...>]

Hi Bulat,

first off, could you please send questions to the JGroups mailing list
(copied), and not to me directly, so we have a searchable archive of Q&As ?

Comments inline

Bulat Nigmatullin wrote:
> Hi, Bela!
>
> I've tested different conrigurations of cluster.
> 2 node cluster, 5K messages/sec load on cluster -> 100% CPU load on 
> every node

OK, using the perf test (http://www.jboss.org/community/docs/DOC-11594),
I configured 2 nodes on the same box, each sender sends 1 million 5K
messages, for a total of 2 mill (10GBytes total). The network is 1GB
*without* jumbo frames.

I got 30'700 messages /sec / node, that's 153 MB / sec / node. This is
per *node* !

The CPU was between 60% and 70% per core.

When I run tests like these on a real cluster, results usually get
better, because then the same CPU doesn't have to deal with N senders,
but only with 1.


> 3 node cluster, 9K messages/sec load - > 150% CPU load on every node

I got 14'000 msgs/sec/node, throughput 129 MByte/sec/node for 3 nodes,
each sending 1 million 9K messages. CPU was 80% per core.



> Every message ~ 100 bytes.

Oops ! I thought you're sending bigger messages ! Let me try again, this
time with 4 nodes and sending 1 million 500b messages:

I got 50'200 messages/sec/node and 25 MBytes/sec/node throughput.


> I'm looking for the way to improve cluster performance at least 2 
> times, cause i think 4 node cluster will run out of CPU. Does move to 
> UDP give it to me?

Oh, you tested with TCP ? I used UDP. UDP is usually better with larger
clusters, but TCP should be about the same for small clusters, see
http://www.jgroups.org/performance.html for details on previous
performance benchmarks.

> The project target is a cluster able to hold 30K messages/sec load.

That's easily achievable: I did this even with sending 5K messages
around the 2 node cluster. With 500 byte messages (5 times more than
what you send), I got 50'000/sec/node in a 4 node cluster !

-- 
Bela Ban
Lead JGroups / Clustering Team
JBoss - a division of Red Hat