[openMosix-general] Help required on MRTG/clustering

[Amol S. <amo...@co...>]

Dear all,
I am not sure if this question needs to appear here.

I am setting up a MRTG server which polls close to 100 nodes. A lot of perl scripts process the 
data collected by this server. This processing is heavy and involves database (mysql) and flat mrtg 
log files. There are other perl scripts which are used as part of a mailing engine (uses sendmail).
MRTG is called by cron every 5 minutes and the 100 nodes are divided such that there are 4 simple 
shell scripts which contain the MRTG execution lines. These 4 scripts are called in cron every 5 
minutes. There are other scheduled jobs which run only few (1-4) times a day.

Few days back, when I increased my nodes polled by 20 (from 80 to 100) and introduced a perl 
script, I could see a lot of processing happening on the box. 'top' showed that most of it was 
being crunched up my mysql (20-40%), mrtg, snmpwalks. So, I have got 3 servers now to implement 
clustering and make sure that:
1.access times are fast.
2.MRTG is able to complete polling of all the devices within 5 mins.

There are not too many hits to the site in a day (but backend processing is high). Also, certain 
pages call for database access initiaiting a lot of load on the system once called. So, assuming 
that even if there are 4-5 parallel hits to these database intensive pages, it can pull down my box 
(i may be wrong here).

All machines will be internal to my network (only one server has a domain name entry on a DNS 
*internal* to the organisation).
To improve the overall performance, I want to put my databases on one particular server and mrtg 
stuff on other server. My database is being populated by data from the mrtg flat files (this will 
need the perl scripts to access files on other machine, not local). I am not sure how to implement 
this. The third machine will be the one which has a DNS entry. Is it possible to have such a setup 
where:

1.MRTG flat files are being updated on one server (server 1, only this server has a snmp read 
access)
2.Database (which uses above flat files) on other server (server 2)
3.A server with DNS entry which will be interface to the users (server 3, will contain pages which 
will need to access data from above 2 servers)
4.Cron runs on server 1 and 2 (or even 3).
5.All servers form a cluster for load balancing/process migration and fail over (fail over will be 
for server 3. maybe server1 or 2 can act as backup)

Please help me on above, I have been working on openmosix initially, but now I think openmosix may 
not be an answer to above design. As suggested by Marcus earlier on the list, I am readinh above 
LVS and Ultra Monkey too. Is this design to do anything with nfs/samba (I need to access files over 
a lan, server 1-server 2). What technology best fits for above design? Will openmosix/lvs/ultra 
monkey help?


Many Thanks,
Amol Sapkal

"Everything that goes up must come down"
Cheers,

Amol Sapkal
--------------------------
Temperament is not destiny