Menu
▾
▴
ci-linux-devel — Discussion about Cluster Infrastructure for Linux
You can subscribe to this list here.
| 2001 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(25) |
Jul
(13) |
Aug
(11) |
Sep
(14) |
Oct
(5) |
Nov
(7) |
Dec
(4) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2002 |
Jan
(14) |
Feb
(10) |
Mar
(30) |
Apr
(9) |
May
(20) |
Jun
(12) |
Jul
(7) |
Aug
(6) |
Sep
(6) |
Oct
(34) |
Nov
(14) |
Dec
(9) |
| 2003 |
Jan
|
Feb
(9) |
Mar
(2) |
Apr
(2) |
May
(5) |
Jun
(14) |
Jul
(1) |
Aug
(7) |
Sep
(6) |
Oct
(5) |
Nov
|
Dec
|
| 2004 |
Jan
|
Feb
(1) |
Mar
(1) |
Apr
|
May
|
Jun
(1) |
Jul
(2) |
Aug
(2) |
Sep
(4) |
Oct
|
Nov
(7) |
Dec
(1) |
| 2005 |
Jan
(1) |
Feb
(2) |
Mar
|
Apr
(4) |
May
(5) |
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
| 2006 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(11) |
Jun
(2) |
Jul
|
Aug
(5) |
Sep
(5) |
Oct
(1) |
Nov
(1) |
Dec
|
| 2007 |
Jan
|
Feb
(1) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2008 |
Jan
(2) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
| 2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
(1) |
Jun
|
Jul
|
Aug
|
Sep
|
Oct
(2) |
Nov
|
Dec
|
| 2011 |
Jan
(1) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
|
From: David B. Z. <dav...@hp...> - 2002-06-10 17:59:58
|
Below is something I wrote up last week, but was waiting for Bruce to comment on it before sending it out. Now that I see what you've done with /etc/init.d/clusterinit, I thought I'd send this out. I will examine what you've done today. Last week I did something similiar. I wasn't as concerned with the dependent node networking, but I wanted to replace rc.sysinit for dependent nodes only. I copied the redhat rc.sysinit to rc.sysinit.nodeup and removed all the things which the dependent nodes should not be duplicating. I also removed the execution of rc for run level 3 from rc.nodeup. Keep in mind that only the first booting node runs rc.sysinit just like base linux. Since only dependent nodes run rc.nodeup, only the dependent nodes run rc.sysinit.nodeup. --------- You've brought up an important architectural issue. Once there is a single root it requires clusterization to have duplicate services running. One way to clusterize things is adding context dependent links (i.e. /var/run as you proposed for the *.pid files). The current set-up of having rc.nodeup call rc.sysinit then running complete rc 3 runlevel processing was fine when we had a non-shared root. Now with CFS and GFS we really need to NOT do this. Looking at rc.sysinit on a redhat install, I see that it does all sorts of stuff which should NOT be done again on a joining node in the shared-root case. In a cluster there would generally be two kinds of services. The first kind is a single instance of the service (single process or set of processes on one node) running with keepalive to restart it on node failures. The second kind is the service that is cluster aware, so that processes could exist on multiple nodes, but they cooperate with each other. In non-stop clusters we parallelized inetd, for example. It maintained processes on all nodes, and kept a list of pids which it updated as nodes came and went. The whole /var/run/service_name.pid mechanism I would propose is only used for non-cluster aware serives which are restricted to running on the root node, but may be restarted on node failure. It is assumed that to restart the service we might have to remove the .pid file and then on (re)start the service would create the file again with the new pid. Aneesh Kumar K.V wrote: > Hi, > > I guess we need to have node specific /var/run directory also. > > Otherwise on debian some sevices may not come up on node2. They check > /var/run/service_name.pid file to see whether the service is already > running or not. > > That make it for debian /etc/init.d/rcS add these lines before doing > the for loop show below > > # > # Cluster specific remounts. > # > # > mount --bind /etc/network-`/usr/sbin/clusternode_num` /etc/network > mount --bind /run-`/usr/sbin/clusternode_num` /var/run > > # > # Call all parts in order. > # > for i in /etc/rcS.d/S??* > > > -aneesh > > > > > > _______________________________________________________________ > > Don't miss the 2002 Sprint PCS Application Developer's Conference > August 25-28 in Las Vegas -- http://devcon.sprintpcs.com/adp/index.cfm > > _______________________________________________ > ssic-linux-devel mailing list > ssi...@li... > https://lists.sourceforge.net/lists/listinfo/ssic-linux-devel > > -- David B. Zafman | Hewlett-Packard Company Linux Kernel Developer | Open SSI Clustering Project mailto:dav...@hp... | http://www.hp.com "Thus spake the master programmer: When you have learned to snatch the error code from the trap frame, it will be time for you to leave." |
|
From: Aneesh K. K.V <ane...@di...> - 2002-06-09 06:55:52
|
Hi, Instead of going and modifying /etc/rc.d/rc.sysinit for all the mount. I guess it is better to execute a script there. ie /etc/init.d/clusterinit. and /etc/init.d/clusterinit as of now on the Redhat Machine i have here looks like. #! /bin/bash NETWORK_SCRIPTS_DIR=/etc/sysconfig/network-scripts VAR_RUN_DIR=/var/run VAR_LOCK_DIR=/var/lock /usr/sbin/clusternode_num 2>/dev/null if [ $? -ne 0 ]; then # Not inside a NSC cluster. # Cluster cannot have a node number 0 echo "Configuring in a Non SSI Cluster" mount --bind $NETWORK_SCRIPTS_DIR-0 $NETWORK_SCRIPTS_DIR mount --bind $VAR_RUN_DIR-0 $VAR_RUN_DIR mount --bind $VAR_LOCK_DIR-0 $VAR_LOCK_DIR else #Inside a NSC cluster. echo "Configuring in a SSI Cluster" mount --bind $NETWORK_SCRIPTS_DIR-`/usr/sbin/clusternode_num` $NETWORK_SCRIPTS_DIR mount --bind $VAR_RUN_DIR-`/usr/sbin/clusternode_num` $VAR_RUN_DIR mount --bind $VAR_LOCK_DIR-`/usr/sbin/clusternode_num` $VAR_LOCK_DIR fi this file will grow over a period of time. Changes needed in cluster installation on node 1 cd /etc/sysconfig mkdir network-scripts-1 cd network-scripts-1 cp ../network-script/* . mv network-scripts network-scripts-0 mkdir network-scripts Why network-scritps-0 ? Well Most of the people will be trying SSI with a general machine that is already running linux, and they want to boot a normal kernel and a SSI kernel out of the same machine. We don't want to create problems just because he did a SSI kernel installation.( I will tell more about that later ).In the methods described in the web page once you modify the networking scripts there is no way you can boot a normal kernel because we already screwed the configuration file. So for NON SSI we here uses dirname-0 and then mount it as dirname . That seems to be a safe game for me. This need to be done only in master node. Over a period of time we will identify more and more directories that are node specific :)( already we have three ) How to distribute SSI ? rpm and bebian packages. I guess if we ask guys to download the kernel patch it run cluster-tools compile and install it, it is going to be a tough job and most of them stop after downloading the kernel itself. A solution is to provide an rpm or debian package that will install the kernel with the respective modules and then ask the user to run the configure_cluster script. This script will configure the cluster for him. I guess it is fine to say that CFS is the default configuration and each node should have a boot disk( including normal nodes ) . So that we can remove dhcp and network booting. It scares people. :) May be we can have dhcp and network booting configuration in a production environment. Any volunteers for preparing RPM and debian packages ? -aneesh |
|
From: Aneesh K. K.V <ane...@di...> - 2002-05-31 06:57:33
|
Hi, Deleted the previous patches and posted one single patch. http://ssic-linux.sourceforge.net/contrib/ssic-linux-2.4.16-ipvs-1.0.2-hidden.patch Howto build (1) Prepare the SSI kernel source as per the documentations in the SSI website. (2) Apply the above patch. the above patch contain all the modifications that i posted in my previous mail. -aneesh On Fri, 2002-05-31 at 04:00, Aneesh Kumar K.V wrote: > Hi, > > I have posted the patches at the following URL. > > http://ssic-linux.sourceforge.net/contrib/Configure.help.lvs.ci > http://ssic-linux.sourceforge.net/contrib/Configure.help.lvs.ssi > http://ssic-linux.sourceforge.net/contrib/ksyms.c.lvs.ssi > http://ssic-linux.sourceforge.net/contrib/linux-2.4.16-ipvs-1.0.2.patch > > the ipvs patch contain the ipvs-1.0.2 patch and the > hidden-forward_shared-noarp-2.4.16-1.diff > > > HOWTO build. > > download the linux kernel 2.4.16 from kernel.org > Apply the linux-2.4.16-ipvs-1.0.2.patch to the kernel tree. > ( From here onwards follow the cluster build procedure ) > Apply the UML patch ( neglect the error ) > cp the ci and ssi sources from the sand box. > > cp -alf <path-ci>/ci-kernel/. . > cp -alf <path-ssic-linux>/ssi-kernel/. . > > download Configure.help.lvs.* file and ksyms.c.lvs.ssi > If you are building only CI replace the Documentation/Configure.help > in the linux kernel source with Configure.help.lvs.ci > cp Configure.help.lvs.ci <path-linux-sour>/Documentation/Configure.help > > If building SSI kernel > cp Configure.help.lvs.ssi <path-linux-sour>/Documentation/Configure.help > cp ksyms.c.lvs.ssi <path-linux-sour>/kernel/ksyms.c > > > After that Configure the kernel looking into information from SSI and > LVS website. > > Once the cluster boots, you can configure the LVS by following the > steps specified at the > http://www.linuxvirtualserver.org/Joseph.Mack/mini-HOWTO/LVS-mini-HOWTO.html > > > Status > I haven't tried building a CI LVS setup. Hopefully it should work. > Successfully configured SSI with LVS using VS-DR forwarding. > > > If you run into problems Please let me know > > -aneesh > > > > > > _______________________________________________________________ > > Don't miss the 2002 Sprint PCS Application Developer's Conference > August 25-28 in Las Vegas -- http://devcon.sprintpcs.com/adp/index.cfm > > _______________________________________________ > ssic-linux-devel mailing list > ssi...@li... > https://lists.sourceforge.net/lists/listinfo/ssic-linux-devel |
|
From: Brian J. W. <Bri...@hp...> - 2002-05-23 03:11:53
|
It includes Aneesh's fixes for some Makefile bugs, as well as Laura's
fix for a bug in the onall command.
Sorry I didn't release it sooner. I wanted to get a better handle on
Richard Xieq's problem to make sure there wasn't another bug that needed
fixing.
--
Brian Watson | "Now I don't know, but I been told it's
Software Developer | hard to run with the weight of gold,
Open SSI Clustering Project | Other hand I heard it said, it's
Hewlett-Packard Company | just as hard with the weight of lead."
| -Robert Hunter, 1970
mailto:Bri...@hp...
http://opensource.compaq.com/
|
|
From: Aneesh K. K.V <ane...@di...> - 2002-05-19 08:44:27
|
Hi, http://www.dell.com/us/en/esg/topics/power_ps1q01-linux.htm -aneesh On Sat, 2002-05-18 at 23:05, Anuj Garg wrote: > In a clustered environment typically hosts are interconnected > using network connection, and maintains record of each other > health using heartbeat mechanism. > > If network link between hosts is down then it splits the network > in two halfs with each host assuming that it is the master of > the network. Since there is no connection between the hosts they > can't differentiate between the scenario where other host is > down OR they have just lost their connectivity. > > Any idea how to solve this issue or any books, references etc ? > Your responses will be deeply appreciated. > > Regards, > Anuj Garg > > __________________________________________________ > Do You Yahoo!? > LAUNCH - Your Yahoo! Music Experience > http://launch.yahoo.com > > _______________________________________________________________ > Hundreds of nodes, one monster rendering program. > Now that's a super model! Visit http://clustering.foundries.sf.net/ > > _______________________________________________ > ci-linux-devel mailing list > ci-...@li... > https://lists.sourceforge.net/lists/listinfo/ci-linux-devel -- http://photos.yahoo.com/kvaneesh |
|
From: Anuj G. <anu...@ya...> - 2002-05-18 17:35:31
|
In a clustered environment typically hosts are interconnected using network connection, and maintains record of each other health using heartbeat mechanism. If network link between hosts is down then it splits the network in two halfs with each host assuming that it is the master of the network. Since there is no connection between the hosts they can't differentiate between the scenario where other host is down OR they have just lost their connectivity. Any idea how to solve this issue or any books, references etc ? Your responses will be deeply appreciated. Regards, Anuj Garg __________________________________________________ Do You Yahoo!? LAUNCH - Your Yahoo! Music Experience http://launch.yahoo.com |
|
From: Brian J. W. <Bri...@hp...> - 2002-05-17 01:34:55
|
"Aneesh Kumar K.V" wrote:
>
> Hi,
> The attached patch correct some of the build issues that I faced on a
> redhat machine. It takes the backup of all the files before it replaces
> it with cluster files.
> [snip]
The Cluster Tools build system is such a mess!! Unfortunately I had a
part to play in creating that mess. Your patch helps resolve some of the
problems, but the build system could certainly stand to be redesigned.
I have a few changes to your patch:
- the make line can be removed from the uninstall_ssi rule, since
you moved it to the uninstall rule
- rather than spell out $(UML_ROOT)/etc/sysconfig/network-scripts
in several places, I set $(NETSCRIPTS) equal to it
- I cleaned up my messy logic for calling configure_cluster and
deconfigure_cluster
- there should now be no need to save /etc/fstab unless UML_ROOT
is set
- similarly there's no need to restore /etc/fstab in
deconfigure_cluster
My updated patch is attached. Note that it has _not_ been tested, since
I don't have convenient access to a hardware-based cluster right now.
The permissions problem with configure_cluster and deconfigure_cluster
leads me to believe that the 0.6.5 release doesn't work for hardware
installations. That's what I get for only testing on UML!! :(
In two or three days I'll roll a new release of Cluster Tools and SSI
with these changes. If anyone's found other problems with the 0.6.5
release, please speak up now.
--
Brian Watson | "Now I don't know, but I been told it's
Software Developer | hard to run with the weight of gold,
Open SSI Clustering Project | Other hand I heard it said, it's
Hewlett-Packard Company | just as hard with the weight of lead."
| -Robert Hunter, 1970
mailto:Bri...@hp...
http://opensource.compaq.com/ |
|
From: Aneesh K. K.V <ane...@di...> - 2002-05-16 09:52:59
|
Hi, The attached patch correct some of the build issues that I faced on a redhat machine. It takes the backup of all the files before it replaces it with cluster files. Why ? configure_cluster didn't have execute permission and as a result on my system( which doesn't have a shared storage ) it went and replaced the fstab with the one came with cluster tools without leaving me a backup . Similarly the network configuration files. ( while rebuilding the fstab file i made a mistake. This caused the machine to crash during the next boot. and I had to create a custom initial ram disk to get the machine back. It was actually pain. Having a backup is always a good idea. ). If the .old file is already there, no backup is taken .This prevent rewriting already taken backup when i run make install again and again . running uninstall in ssi dir ( deconfigure_cluster script is here ).This brings the system back to the old state. If there is nothing wrong with the patch. I can do a checking. -aneesh Index: Makefile =================================================================== RCS file: /cvsroot/ci-linux/cluster-tools/Makefile,v retrieving revision 1.2 diff -r1.2 Makefile 62a63 > make -C ssi uninstall Index: ssi/Makefile =================================================================== RCS file: /cvsroot/ci-linux/cluster-tools/ssi/Makefile,v retrieving revision 1.6 diff -r1.6 Makefile 33a34,44 > @if [ ! -f $(UML_ROOT)/etc/sysconfig/network-scripts/ifcfg-eth0.old ]; then \ > f=$(UML_ROOT)/etc/sysconfig/network-scripts/ifcfg-eth0; \ > echo "Saving existing $$f as $$f.old"; \ > cp -f $$f $$f.old; \ > fi > @if [ ! -f $(UML_ROOT)/etc/sysconfig/network-scripts/network-functions.old ]; then \ > f=$(UML_ROOT)/etc/sysconfig/network-scripts/network-functions; \ > echo "Saving existing $$f as $$f.old"; \ > cp -f $$f $$f.old; \ > fi > 62a74,79 > @if [ ! -f $(UML_ROOT)/etc/fstab.old ]; then \ > f=$(UML_ROOT)/etc/fstab; \ > echo "Saving existing $$f as $$f.old"; \ > cp -f $$f $$f.old; \ > fi > chmod u+x ./configure_cluster 77a95 > chmod u+x ./deconfigure_cluster Index: ssi/deconfigure_cluster =================================================================== RCS file: /cvsroot/ci-linux/cluster-tools/ssi/deconfigure_cluster,v retrieving revision 1.1 diff -r1.1 deconfigure_cluster 7a8,13 > [ -f /etc/sysconfig/network-scripts/ifcfg-eth0.old ] && > mv -f /etc/sysconfig/network-scripts/ifcfg-eth0.old /etc/sysconfig/network-scripts/ifcfg-eth0 > [ -f /etc/sysconfig/network-scripts/network-functions.old ] && > mv -f /etc/sysconfig/network-scripts/network-functions.old /etc/sysconfig/network-scripts/network-functions > [ -f /etc/fstab.old ] && > mv -f /etc/fstab.old /etc/fstab |
|
From: Brian J. W. <Bri...@hp...> - 2002-05-15 20:15:16
|
"Aneesh Kumar K.V" wrote:
> Well In the case of already existing connection which will be doing read
> and write( If i understand correctly ) they will be carried out as
> remote operation on the socket,which means even though the application
> migrates the socket still remain at the first node(node1).( Brian
> should be able to tell more about that. Brian ? ).
Yes, processes and their sockets can be on different nodes and still
work. Processes can migrate whereas sockets cannot yet, so it's
important that they be able to work together remotely.
In the currently released SSI code, processes can do file operations
(read, write, poll, ioctl, etc.) on their remote sockets. I'm working on
allowing them to also do remote socket operations (sendmsg, recvmsg,
accept, etc.). This should be available in the next release after 0.6.5.
Hope this clarifies things a bit.
--
Brian Watson | "Now I don't know, but I been told it's
Software Developer | hard to run with the weight of gold,
Open SSI Clustering Project | Other hand I heard it said, it's
Hewlett-Packard Company | just as hard with the weight of lead."
| -Robert Hunter, 1970
mailto:Bri...@hp...
http://opensource.compaq.com/
|
|
From: Brian J. W. <Bri...@hp...> - 2002-05-15 20:07:53
|
Alexandre CASSEN wrote:
> Just let me sketch a simple env :
>
> example specs: Two VIPs exposed to the world, loadbalancing using LVS on a
> realserver pool.
> Sketch:
>
> +------------------+
> | WAN area |
> +------------------+
>
> .............................................[CI/SSI cluster].....
> . .
> . .
> . +----[VIP1]---+ +----[VIP2]---+ .
> . | | | | .
> . | LVS node1 | | LVS node2 | .
> . +-------------+ +-------------+ .
> . .
> . +----[RIP1]---+ +----[RIP2]---+ +----[RIP3]---+ .
> . | | | | | | .
> . | app node1 | | app node2 | | app node3 | .
> . +-------------+ +-------------+ +-------------+ .
> ..................................................................
>
There's not much point in running two LVS director nodes on the same
network. Barring any failures, the CVIP and LVS director would be
"stuck" to node1 and only node1. If node1 fails, then the CVIP would
failover to node2. As Bruce described, node2 would poll the other
servers to find out their connection states, it would build its own
director table with this information, and then it would begin acting as
the one and only CVIP/director node. Until this happens node2 cannot be
considered a CVIP/director node any more than node3.
--
Brian Watson | "Now I don't know, but I been told it's
Software Developer | hard to run with the weight of gold,
Open SSI Clustering Project | Other hand I heard it said, it's
Hewlett-Packard Company | just as hard with the weight of lead."
| -Robert Hunter, 1970
mailto:Bri...@hp...
http://opensource.compaq.com/
|
|
From: Brian J. W. <Bri...@hp...> - 2002-05-15 04:50:55
|
"Aneesh Kumar K.V" wrote:
>
> Hi,
>
> The DLM changes are actually patches to DLM rather than to CI/SSI. I
> have asked Brian to put the url in the CHANGES file may be in the next
> release of CI so that people can easily locate the DLM patches.
Aneesh-
Can you check your patch and the DLM patch into the 3rd-party/
subdirectory of the ci-linux repository? Also can you write up a short
description of what you did and how to use it in a file such as
README.dlm and check that in? Finally, feel free to add a line to the
top of the CHANGES file. It'll go out with the next release of CI.
--
Brian Watson | "Now I don't know, but I been told it's
Software Developer | hard to run with the weight of gold,
Open SSI Clustering Project | Other hand I heard it said, it's
Hewlett-Packard Company | just as hard with the weight of lead."
| -Robert Hunter, 1970
mailto:Bri...@hp...
http://opensource.compaq.com/
|
|
From: Aneesh K. K.V <ane...@di...> - 2002-05-15 03:52:54
|
Hi, The DLM changes are actually patches to DLM rather than to CI/SSI. I have asked Brian to put the url in the CHANGES file may be in the next release of CI so that people can easily locate the DLM patches. -aneesh On Wed, 2002-05-15 at 02:01, Greg Freemyer wrote: > Out of curiosity, are Aneesh's DLM changes part of this? > > If not, is that because DLM is not yet a part of SSI? > > Greg Freemyer > Internet Engineer > Deployment and Integration Specialist > Compaq ASE - Tru64 > Compaq Master ASE - SAN Architect > The Norcross Group > www.NorcrossGroup.com > > >> It's now available on > >> http://ci-linux.sf.net/ > > >> It features a new where_pid command, a version of init.ssi that can > >> handle nodes booting in parallel, and stuff to support UML. > > >> Here's the changelog: > > >> - added where_pid command to query which node a process is running on > >> (Laura Ramirez) > >> - fixed init so that nodes can boot in parallel (Laura Ramirez) > >> - added a UML_ROOT option to install Cluster Tools on a loopback mounted > >> root image (Brian Watson) > >> - added a --uml option to cluster_mkinitrd for building a UML initrd > >> (Brian Watson) > >> - other changes made to support UML (Brian Watson) > >> - made Cluster Tools build properly on Alpha (Aneesh Kumar) > >> - miscellaneous bug fixes > > >> -- > >> Brian Watson | "Now I don't know, but I been told it's > >> Software Developer | hard to run with the weight of gold, > >> Open SSI Clustering Project | Other hand I heard it said, it's > >> Hewlett-Packard Company | just as hard with the weight of lead." > >> | -Robert Hunter, 1970 > > >> mailto:Bri...@co... > >> http://opensource.compaq.com/ > > >> _______________________________________________________________ > > >> Have big pipes? SourceForge.net is looking for download mirrors. We supply > >> the hardware. You get the recognition. Email Us: ban...@so... > >> _______________________________________________ > >> ci-linux-devel mailing list > >> ci-...@li... > >> https://lists.sourceforge.net/lists/listinfo/ci-linux-devel > > > > > > > _______________________________________________________________ > > Have big pipes? SourceForge.net is looking for download mirrors. We supply > the hardware. You get the recognition. Email Us: ban...@so... > _______________________________________________ > ci-linux-devel mailing list > ci-...@li... > https://lists.sourceforge.net/lists/listinfo/ci-linux-devel |
|
From: Alexandre C. <Ale...@wa...> - 2002-05-14 22:26:51
|
Bruce, > I think there may be some confusion. Hopefully I can >clarify. There was a technology on Unixware called >NonStop Clusters for Unixware (NSC for short) that was SSI. >The networking piece of NSC had two parts - a CVIP (Cluster >Virtual IP) capability and multi-homed host across the cluster >capability. The CVIP capability was quite similar to LVS. I >won't go into all the detail of what the multi-homed capability >was, although I would be happy to another time. > >As we are moving various NSC functionality to Linux, we decided to >leverage the LVS activity rather than porting the CVIP >capability we had. The intent is not to have LVS and >CVIP but just LVS+. aah....!!!! OK, limpid ! this is what was suspecting. LVS+ you mean LVS for CI/SSI (just kidding) >The goal of integrating CI/SSI with LVS was to enhance the CI/SSI >environment, not necessarily enhance all LVS environments. As you >point out, the plan is that the director(s) would be inside the CI/SSI >cluster and would load level connections only to nodes also in the >cluster. Integration involves reconciling the nodenum with IP >addresses and leveraging the nodedown capability of CI/SSI when >servers go down and facilitating a failover of the director >when that node goes down. OK. here the VRRP and (keepalived) healthcheck derivation to CI/SSI code with the patch from Aneesh. Derivating Healthceck to the inside CI/SSI check framework will limit keepalived CI/SSI integration to the internal CI/SSI available checkers ? This can be a starting point, I agree. >There is also interest in extending LVS in this environment in a couple of >ways. First, we may be able to automatically detect when processes bind() >to a registered VIP port, and thus avoid having to declare where the >servers for a port are. More significantly, however, we are considering >allowing the VIP to be used for outgoing connections. Ok, I see. Interresting point... >Below I tried to clear up some of the possible confusion. Ok, thanks for your clarifications, I am sync with your CI/SSI conceptual design, I was confusing with the CVIP since I have red some infos on this in pdf on the site. Best regards and thanks for your time, Alexandre |
|
From: Bruce W. <br...@ka...> - 2002-05-14 21:19:56
|
Alexandre,
I think there may be some confusion. Hopefully I can
clarify. There was a technology on Unixware called
NonStop Clusters for Unixware (NSC for short) that was SSI.
The networking piece of NSC had two parts - a CVIP (Cluster
Virtual IP) capability and multi-homed host across the cluster
capability. The CVIP capability was quite similar to LVS. I
won't go into all the detail of what the multi-homed capability
was, although I would be happy to another time.
As we are moving various NSC functionality to Linux, we decided to
leverage the LVS activity rather than porting the CVIP
capability we had. The intent is not to have LVS and
CVIP but just LVS+.
Doing this has two components - integration with the membership/
nodeup, nodedown capability of CI/CLMS; and extending some of the
LVS concepts (which may or may not be applicable to other LVS
configurations).
The goal of integrating CI/SSI with LVS was to enhance the CI/SSI
environment, not necessarily enhance all LVS environments. As you
point out, the plan is that the director(s) would be inside the CI/SSI
cluster and would load level connections only to nodes also in the
cluster. Integration involves reconciling the nodenum with IP
addresses and leveraging the nodedown capability of CI/SSI when
servers go down and facilitating a failover of the director
when that node goes down.
There is also interest in extending LVS in this environment in a couple of
ways. First, we may be able to automatically detect when processes bind()
to a registered VIP port, and thus avoid having to declare where the
servers for a port are. More significantly, however, we are considering
allowing the VIP to be used for outgoing connections. This of course
requires extensions on all nodes in the CI/SSI cluster. If I understand
it properly, such a capability is closer to what exists in TruClusters.
Below I tried to clear up some of the possible confusion.
>
> Hi Aneesh,
>
> > Attaching below is the patch that add Cluster Infrastructure (
> > http://ci-linux.sourceforge.net ) as the health check mechanism for the
> > keepalive daemons. Please comment on the patch so that i can continue
> > working on it
> > --[..snip patch..]--
>
> I have spend a little time reading the CI/SSI docs available on the
> projects sites and I have some questions :
>
> If I make some mistake please correct me : According to what I have red on
> CI/SSI :
>
> * CI provide low level primitives like ICS & CLMS that are used in SSI
> code. CI/SSI hookes/patches are installed on all the cluster node. A node
> is represented by its NODENUM that is passed to linux kernel during linux
> kernel bootstrap. So CI kernel hooks (ICS & CLMS) permit to localize a
> specific node according to its NODENUM.
Yes, but nodes also have IP addresses and there will be interfaces to
go from nodenum to IP address and back.
>
> * SSI use CI primitives (libcluster) to present the collection of cluster
> nodes as single system image. This is the real cluster definition, cluster
> (I agree) is not only limited to a single virtualization but multiple
> system subsystems like FS, network, IPC, process migration, ...
This could be a bit misleading. The SSI is not presented by libcluster
but instead by clusterizing all the base system calls via hooks and extensions
in the Linux kernel. libcluster actually provides the interfaces to
"look-under-the-covers" to see the individual components of the cluster.
>
> => This mean that all the cluster nodes must have the same kernel/userspace
> enabled functionnalities. This is the point that make me perplexed with the
> CI/SSI LVS integration. If I fully understand, using LVS with CI/SSI
> introduce the LVS director virtualization based on CI CLUSTER_NODENUM
> identification.
> So we run a LVS director pool. A LVS director submit
> schedulers decisions to loadbalance incoming traffic to a specific
> realserver. So the CI/SSI provide the CVIP to expose LVS cluster. This is
> my first question : Using this CVIP mean that all node consider this IP as
> a local address, so what about LVS connection table ? if traffic is
> oriented to a specific LVS director the first time and then jump to another
> the second time, what about connections that require persistents
> connections ? we will probably introduce a loadbalancing breaking point ?
I'm not sure if the CVIP vs. LVS discussion above helps with this. Note that
my understanding is that incoming connections will work very much the standard
LVS way, including the special cases for persistent connections. Outgoing
connections using the VIP will have to be registered with the director
so responses can be routed to the correct node/server. The director may
also have be extended to provide a clusterwide port space as well.
>
> => LVS code is a routing software, so if we run LVS in a director CI/SSI
> virtualized environnement, the LVS connection table must be synced probably
> using a hook into the ICS (a ICS channel) to garanty loadbalancing decision
> integrity (consistence). Another point is the CVIP takeover, does the CVIP
> is a roaming VIP on the cluster ? if not what makes this CVIP takeover on
> another cluster node ?
While it may be desireable at some point to integrate LVS with ICS, it is
not necessary to get started. As far as facilitating VIP takeover, I think
we should leverage the existing LVS syncd code, although untimately I would
prefer to rebuild the director on failover (by polling the servers) rather
than paying the overhead of replicating on every connect and disconnect.
That is the strategy we used in NSC for failover.
>
> => Reading the patch, it figures out that CI nodes availability is checked
> with "clusternode_info" over a cluster NODENUM. So node availibility is
> returned according to your "keepalive" tool that inform and update the
> kernel CI cluster node map. This imply that CI/SSI must be installed on all
> CI/SSI cluster nodes and LVS too. This introduce the fact that a CI/SSI
> cluster is some kind of global virtual cluster including (with the
> loadbalancing eyes) LVS directors and realservers... This is my other
> perplexed point. To use CI healcheck framework, we need to hook kernel with
> CI patch, but we need to patch kernel with LVS code to provide
> loadbalancing part. So realservers and LVS director are global part of the
> same cluster exposed and a server can be LVS director and realserver
> (webserver for example) at a time. This limit the realserver pool to be
> linux ? I doesn t fully understand that point ... :/
Yes. All the code (subject to dynamic loading) will be the same on all
nodes and any node could be director or server or both and only nodes
that are part of the CI/SSI cluster can participate. That being said,
I don't think it is out of the question to consider other configuration,
like having the director outside the CI/SSI cluster while the various
members of the cluster are each standard LVS servers; or possibly
just have the director in the cluster with servers external.
bruce
bru...@hp..., formerly of compaq.com and tandem.com and orginally locus.com
|
|
From: Greg F. <fre...@No...> - 2002-05-14 20:35:57
|
Out of curiosity, are Aneesh's DLM changes part of this? If not, is that because DLM is not yet a part of SSI? Greg Freemyer Internet Engineer Deployment and Integration Specialist Compaq ASE - Tru64 Compaq Master ASE - SAN Architect The Norcross Group www.NorcrossGroup.com >> It's now available on >> http://ci-linux.sf.net/ >> It features a new where_pid command, a version of init.ssi that can >> handle nodes booting in parallel, and stuff to support UML. >> Here's the changelog: >> - added where_pid command to query which node a process is running on >> (Laura Ramirez) >> - fixed init so that nodes can boot in parallel (Laura Ramirez) >> - added a UML_ROOT option to install Cluster Tools on a loopback mounted >> root image (Brian Watson) >> - added a --uml option to cluster_mkinitrd for building a UML initrd=20 >> (Brian Watson) >> - other changes made to support UML (Brian Watson) >> - made Cluster Tools build properly on Alpha (Aneesh Kumar) >> - miscellaneous bug fixes >> --=20 >> Brian Watson | "Now I don't know, but I been told it's >> Software Developer | hard to run with the weight of gold, >> Open SSI Clustering Project | Other hand I heard it said, it's >> Hewlett-Packard Company | just as hard with the weight of lead." >> | -Robert Hunter, 1970 >> mailto:Bri...@co... >> http://opensource.compaq.com/ >> _______________________________________________________________ >> Have big pipes? SourceForge.net is looking for download mirrors. We supply >> the hardware. You get the recognition. Email Us: ban...@so... >> _______________________________________________ >> ci-linux-devel mailing list >> ci-...@li... >> https://lists.sourceforge.net/lists/listinfo/ci-linux-devel |
|
From: Alexandre C. <ale...@ca...> - 2002-05-14 14:00:45
|
Hi Aneesh,
> SSI is also a kernel patch. libcluster is the interface for the user
> application in accessing many of the cluster information.
Ok.
> You mean what happens when the application migrates to other node ?
Yes exactly. According to your CI terminology what is the "application" ?
LVS code, WEB application, fs or all together ?
IMHO, if all together, this can result of some kind framework code
repetition. What I mean :
Introducing network loadbalancing in a logic network topology must
introduce some network level abstraction. This mean that we clearly locate
the virtualservers (VIP) & realservers. Loadbalancing framework will
schedule inbound connections on VIP to the elected realserver.
This loadbalancing framework can be incorporated into a global cluster
infrastructure product (like CI/SSI) only if it respect the LVS inside
framework design. Otherwise some big part of the code from LVS (for
instance scheduling decisions) will be backported to the global cluster
infrastructure framework (to deal with the CI/SSI CVIP). This backporting
process can be very time consuming.
If LVS is a part of a CI/SSI cluster, more than one node will have the LVS
code. Since CI/SSI design use the "CVIP" that is a roaming VIP on all LVS
node, the LVS connection table will be altered. A solution can be to
introduce somme kind of CI/SSI CVIP stickyness on a specific LVS node. This
mean that all traffic will be directed to a specific LVS director. That way
we can solve this problem since using this stickyness all connection will
be accepted by the same LVS node so scheduling decisions will be consistent
en LVS inside design will be respected.
This is a requierment since LVS is able to deal with LVS-NAT|TUN|DR, even
more we can use fwmark to mark inbound traffic before processing the
scheduling decision (I don t really know the impact of using fwmark in the
CI code).
Another point, if for CI/SSI you want a failover protection for the CVIP,
the CVIP takeover must (IMHO) be handled by a specific routing hot standby
protocol (VRRP), this really introduce the question : What is the meaning
of CVIP for use with LVS since CVIP must sticked to a specific LVS node ?
> If i understand your point here. I have the server/daemon running at
> node1 and when it migrates to node2( as per the decision taken by load
> leveler ) and if the server requires persistents connection what will
> happen ?
Yes exactly, what is the CI/SSI meaning of the "node migration process" ?
This sound like a node election/scheduling process isn t it ? if yes, is it
possible to stick the CVIP to a specific LVS node ?
LVS cluster topology, which mean in short of "LVS loadbalancing" is a
multiple active/active LVS topology. This active/active functionnality must
be (IMHO) by a specific dedicated hotstandby protocol. This is why, we (LVS
team) have started develop a robust VRRP framework embended into
keepalived.
From my eyes, LVS director clusterisation must handle the following topics
:
* VIP takeover
* LVS director connections table synchronization.
The current devel status on that 2 points are :
* VIP takeover : Done using VRRP keepalived framework. Currently IPSEC-AH
need to be tuned, especially the IPSEC seqnum synchronization in fallback
scenario.
* LVS director connections table sync : Currently this part is handled by a
LVS part called syncd, that subscribe to a multicast group sending
connection table. This kernel space process can be drived by the VRRP
framework (according to a specific VRRP instance state). Currently the LVS
syncd can not be used in an active/active env since it can only exist one
MASTER syncd sending connections info to the multicast group. This will be
enhanced I think (our time is too short :) )
> Well In the case of already existing connection which will be doing read
> and write( If i understand correctly ) they will be carried out as
> remote operation on the socket,which means even though the application
> migrates the socket still remain at the first node(node1).( Brian
> should be able to tell more about that. Brian ? ).
>
> We haven't decided about what should happen to the main socket (
> socket() ) that is bound to the address.
Ok. for me the point that I don t understand is the CVIP point :/
Just let me sketch a simple env :
example specs: Two VIPs exposed to the world, loadbalancing using LVS on a
realserver pool.
Sketch:
+------------------+
| WAN area |
+------------------+
.............................................[CI/SSI cluster].....
. .
. .
. +----[VIP1]---+ +----[VIP2]---+ .
. | | | | .
. | LVS node1 | | LVS node2 | .
. +-------------+ +-------------+ .
. .
. +----[RIP1]---+ +----[RIP2]---+ +----[RIP3]---+ .
. | | | | | | .
. | app node1 | | app node2 | | app node3 | .
. +-------------+ +-------------+ +-------------+ .
..................................................................
WAN client will access VIP1 and VIP2 exposing a virtual service to the
world (http for example). Then LVS node1 will schedule traffic to app
node1,2,3 (the same for LVS node2).
What is the meaning of CVIP regarding VIP1,2 ? if I understand, CVIP will
transit periodically to LVS node1 and node2, this will break the LVS inside
scheduling coherence because connection table will have no longer
pertinence.
This finaly mean that the CVIP node affectation CI/SSI design is something
like scheduling decision. This scheduling decision will perturbe the LVS
scheduling decision since the LVS code scheduling decision requierd VIP
owned by the director to garanty scheduling performance. This 2 scheculing
levels introduce a concept problem, this problem can be highlighted for
persistents connections.
For example if LVS (node1,2) loadbalance SSL servers (node1,2,3), one
connection should be sticked to a specific app node otherwise ssl protocol
will be corrupted. This stickyness is done if the app node connection
between client & server is present in the LVS conn table. If LVS VIP change
from node from LVS node1 to LVS node2, then LVS node2 will not have the
entry in its connections table so LVS scheduler will be played and another
app node can be elected which will broke this persistence need.
So I can see 2 solutions :
* Replicating the whole director connection table in realtime
* Stick CVIP to a specific LVS node.
=> Then the LVS VIP takeover will be handled by VRRP (for example).
> right now I am thinking of using a script that will select potential LVS
> directors manually and syn the table between them using --sync option
> and fialover using VRRP
Oh OK. But what will be the script key decision for selecting a specific
LVS director ?
this will introduce some scheduling decisions.
This is the part a little obscur for me : What is the LVS key decision
selection ?
> I was trying to bring a Cluster Wide IP functionality to the SSI cluster
> using LVS and VRRP( keepalive ). Also I will be looking into the initial
> work done by Kai. That means nothing is decided. We could either use LVS
> and VRRP or the code from HP's NSC for SCO unixware.
ah OK... This cluster Wide IP must depend on the takeover protocol you will
choose (VRRP, ...).
> The patch was intented to make people able to run LVS with keepalive on
> a CI cluster.Whether it form a base of Cluster Wide IP for SSI is yet
> to be decided.
Yes, yes no problem for me, I will add your patch into keepalived, really
not a problem. I just want to know a little bit more on CI/SSI :)
> I was actually pointing at the initialize_nodemap function. If it fails
> how am i going to inform the main line code. Does a exit() will do it.
This Keepalived code doesn t use pthread (for porting reasons), the thread
functionnality is provided by a central I/O MUX. So to end a thread, just
use return() at the end of your thread functions.
> If my initialize_node_map fails then VRRP( I use the term VRRP because
> there is another keep alive with SSI. To differentiate i guess for the
> time being VRRP is ok ? ) cannot run with CI .
(VRRP name is ok for me, some people mistake VRRP and vrrpd this is why I
use keepalived).
Ok for init_node_map.
> I am not aware of one. But it will be good to have a mapping of node
> number with node IP address. For example in a configuration. with node1
> having two IP address one for cluster interconnect and other for
> external network, it will be good to have an interface like
>
> clusternode_t cluster_node_num( uint32_t addr_ip).
>
> On both the IP it will return me node1.
Hmm, yes, I agree. as a part of your libcluster.
> The CI/SSI also allow an application to know about the node up/down
> events by registering for SIGCLUSTER signal. But using SIGCLUSTER
> doesn't well fit into the existing VRRP( keepalive) model with IO
> multiplexer. Infact first i tried to do it with signal . But later
> scraped it because it doesn't fit well.
Oh OK.
I was thinking of a AF_CLUSTER family. Create a socket with this family
will create a kernel channel to the CI/SSI core code. Then the CI/SSI code
can broadcast kernel message on node status that will be received userspace
to update node status => no polling so overhead will be optimized.
Best regards,
Alexandre
|
|
From: Brian J. W. <Bri...@co...> - 2002-05-14 09:04:17
|
It's now available on http://ci-linux.sf.net/ It features a new where_pid command, a version of init.ssi that can handle nodes booting in parallel, and stuff to support UML. Here's the changelog: - added where_pid command to query which node a process is running on (Laura Ramirez) - fixed init so that nodes can boot in parallel (Laura Ramirez) - added a UML_ROOT option to install Cluster Tools on a loopback mounted root image (Brian Watson) - added a --uml option to cluster_mkinitrd for building a UML initrd (Brian Watson) - other changes made to support UML (Brian Watson) - made Cluster Tools build properly on Alpha (Aneesh Kumar) - miscellaneous bug fixes -- Brian Watson | "Now I don't know, but I been told it's Software Developer | hard to run with the weight of gold, Open SSI Clustering Project | Other hand I heard it said, it's Hewlett-Packard Company | just as hard with the weight of lead." | -Robert Hunter, 1970 mailto:Bri...@co... http://opensource.compaq.com/ |
|
From: Aneesh K. K.V <ane...@di...> - 2002-05-14 05:26:59
|
Hi, On Mon, 2002-05-13 at 20:08, Alexandre CASSEN wrote: > > > Hi Aneesh, > > > Attaching below is the patch that add Cluster Infrastructure ( > > http://ci-linux.sourceforge.net ) as the health check mechanism for the > > keepalive daemons. Please comment on the patch so that i can continue > > working on it > > --[..snip patch..]-- > > I have spend a little time reading the CI/SSI docs available on the > projects sites and I have some questions : > > If I make some mistake please correct me : According to what I have red on > CI/SSI : > > * CI provide low level primitives like ICS & CLMS that are used in SSI > code. CI/SSI hookes/patches are installed on all the cluster node. A node > is represented by its NODENUM that is passed to linux kernel during linux > kernel bootstrap. So CI kernel hooks (ICS & CLMS) permit to localize a > specific node according to its NODENUM. > > * SSI use CI primitives (libcluster) to present the collection of cluster > nodes as single system image. This is the real cluster definition, cluster > (I agree) is not only limited to a single virtualization but multiple > system subsystems like FS, network, IPC, process migration, ... SSI is also a kernel patch. libcluster is the interface for the user application in accessing many of the cluster information. > > => This mean that all the cluster nodes must have the same kernel/userspace > enabled functionnalities. This is the point that make me perplexed with the > CI/SSI LVS integration. If I fully understand, using LVS with CI/SSI > introduce the LVS director virtualization based on CI CLUSTER_NODENUM > identification. So we run a LVS director pool. A LVS director submit > schedulers decisions to loadbalance incoming traffic to a specific > realserver. So the CI/SSI provide the CVIP to expose LVS cluster. This is > my first question : Using this CVIP mean that all node consider this IP as > a local address, so what about LVS connection table ? if traffic is > oriented to a specific LVS director the first time and then jump to another > the second time, what about connections that require persistents > connections ? we will probably introduce a loadbalancing breaking point ? You mean what happens when the application migrates to other node ?.If i understand your point here. I have the server/daemon running at node1 and when it migrates to node2( as per the decision taken by load leveler ) and if the server requires persistents connection what will happen ? Well In the case of already existing connection which will be doing read and write( If i understand correctly ) they will be carried out as remote operation on the socket,which means even though the application migrates the socket still remain at the first node(node1).( Brian should be able to tell more about that. Brian ? ). We haven't decided about what should happen to the main socket ( socket() ) that is bound to the address. > > => LVS code is a routing software, so if we run LVS in a director CI/SSI > virtualized environnement, the LVS connection table must be synced probably > using a hook into the ICS (a ICS channel) to garanty loadbalancing decision > integrity (consistence). Another point is the CVIP takeover, does the CVIP > is a roaming VIP on the cluster ? if not what makes this CVIP takeover on > another cluster node ? right now I am thinking of using a script that will select potential LVS directors manually and syn the table between them using --sync option and fialover using VRRP > > => Reading the patch, it figures out that CI nodes availability is checked > with "clusternode_info" over a cluster NODENUM. So node availibility is > returned according to your "keepalive" tool that inform and update the > kernel CI cluster node map. This imply that CI/SSI must be installed on all > CI/SSI cluster nodes and LVS too. This introduce the fact that a CI/SSI > cluster is some kind of global virtual cluster including (with the > loadbalancing eyes) LVS directors and realservers... This is my other > perplexed point. To use CI healcheck framework, we need to hook kernel with > CI patch, but we need to patch kernel with LVS code to provide > loadbalancing part. So realservers and LVS director are global part of the > same cluster exposed and a server can be LVS director and realserver > (webserver for example) at a time. This limit the realserver pool to be > linux ? I doesn t fully understand that point ... :/ I was trying to bring a Cluster Wide IP functionality to the SSI cluster using LVS and VRRP( keepalive ). Also I will be looking into the initial work done by Kai. That means nothing is decided. We could either use LVS and VRRP or the code from HP's NSC for SCO unixware. The patch was intented to make people able to run LVS with keepalive on a CI cluster.Whether it form a base of Cluster Wide IP for SSI is yet to be decided. > > => Conserning your patch, you have done all in the right place. My only > inputs on your contributed code are : > > 1. > +void ci_get_handler(vector strvec) > +{ > + nodemap = (nodenum_ip_map_t *)malloc(sizeof(nodenum_ip_map_t) > *cluster_maxnodes()+1); /* zero is not considered as a valid node */ > + if( initialize_nodemap(nodemap) < 0 ); > + > + /* How do i inform the main line code ? */ > + queue_checker(NULL,dump_ci_check,ci_check_thread,NULL); > +} > > the main line code is informed by queue_checker. During bottstrap > Keepalived parse the configuration file and then queue a checker on each > keyword known. After the keepalived.conf has been successfully parsed, > keepalived de-queue each checker running the specific checker registration > function. => this is the code in check_api.c the function > register_checkers_thread(). > I was actually pointing at the initialize_nodemap function. If it fails how am i going to inform the main line code. Does a exit() will do it. Or is there any other particular way i should exit from the thread. If my initialize_node_map fails then VRRP( I use the term VRRP because there is another keep alive with SSI. To differentiate i guess for the time being VRRP is ok ? ) cannot run with CI . > 2. > +int node_to_address(clusternode_t node, real_server *rs) > +{ > + > + if( node > cluster_maxnodes() ) { > + syslog(LOG_ERR,"Node number greater than Max node num \n"); > + return -1; > + } > + > + rs->addr_ip = nodemap[node].addr_ip; > + /* do I need to fill the rest of the values here ? */ > + > + return 1; > +} > + > > real_server structure is filled during keepalived.conf parsing. In our case > the configuration file looks like : > > virtual_server 10.10.10.2 80 { > delay_loop 30 > lb_algo rr > lb_kind NAT > protocol TCP > > real_server 192.168.200.6 80 { > weight 1 > CI-LINUX > } > > } > > so the node_to_address() function should be suppress since realserver > structure is filled into parser.c. > > Is there another way to get the node num from the IP address ? I am not aware of one. But it will be good to have a mapping of node number with node IP address. For example in a configuration. with node1 having two IP address one for cluster interconnect and other for external network, it will be good to have an interface like clusternode_t cluster_node_num( uint32_t addr_ip). On both the IP it will return me node1. some kind of > procfs entry so we can reduce again the code removing initialize_nodemap > function ? Much more, if we can get this info through procfs, is it > possible to create some kind of kernel reflection function that reflect > nodemap status to an internal keepalived daemon nodestatus representation ? > > the code I am addressing is : > > +int nodestatus(real_server real) > +{ > + > + int node_num; > + clusternode_info_t ni; > + > + if((node_num = address_to_nodenum(real.addr_ip) ) == 0 ) > + return UNKNOWN_NODE; > + if(clusternode_info(node_num, sizeof(ni), &ni) >= 0) { > + if (ni.node_state == CLUSTERNODE_UP ) > + return UP; > + else > + /* I am insterested only in two state either fully up or > down */ > + return DOWN; > + } > + else { > + syslog(LOG_ERR,"Error in getting the cluster information > \n"); > + } > + return UNKNOWN_NODE; > + > +} > > If we can use some kind of kernel reflection this can be done with > something like : > > +int nodestatus(real_server rs) > +{ > + > + int status = 0; > + > + status = CI_NODE_STATE(rs); > + if (status < 0) > + return UNKNOWN_NODE; > + return (status == CLUSTERNODE_UP)?UP:DOWN; > +} > > CI_NODE_STATE() a simple C macro returning a daemon ci structure status > flags that is updated by kernel broadcast through a kernel socket to the CI > kernel hooks ? this the design keepalived run with netlink. > The CI/SSI also allow an application to know about the node up/down events by registering for SIGCLUSTER signal. But using SIGCLUSTER doesn't well fit into the existing VRRP( keepalive) model with IO multiplexer. Infact first i tried to do it with signal . But later scraped it because it doesn't fit well. > Best regards, and thanks for your inputs, > Alexandre > > -aneesh |
|
From: Alexandre C. <ale...@ca...> - 2002-05-13 14:38:59
|
Hi Aneesh, > Attaching below is the patch that add Cluster Infrastructure ( > http://ci-linux.sourceforge.net ) as the health check mechanism for the > keepalive daemons. Please comment on the patch so that i can continue > working on it > --[..snip patch..]-- I have spend a little time reading the CI/SSI docs available on the projects sites and I have some questions : If I make some mistake please correct me : According to what I have red on CI/SSI : * CI provide low level primitives like ICS & CLMS that are used in SSI code. CI/SSI hookes/patches are installed on all the cluster node. A node is represented by its NODENUM that is passed to linux kernel during linux kernel bootstrap. So CI kernel hooks (ICS & CLMS) permit to localize a specific node according to its NODENUM. * SSI use CI primitives (libcluster) to present the collection of cluster nodes as single system image. This is the real cluster definition, cluster (I agree) is not only limited to a single virtualization but multiple system subsystems like FS, network, IPC, process migration, ... => This mean that all the cluster nodes must have the same kernel/userspace enabled functionnalities. This is the point that make me perplexed with the CI/SSI LVS integration. If I fully understand, using LVS with CI/SSI introduce the LVS director virtualization based on CI CLUSTER_NODENUM identification. So we run a LVS director pool. A LVS director submit schedulers decisions to loadbalance incoming traffic to a specific realserver. So the CI/SSI provide the CVIP to expose LVS cluster. This is my first question : Using this CVIP mean that all node consider this IP as a local address, so what about LVS connection table ? if traffic is oriented to a specific LVS director the first time and then jump to another the second time, what about connections that require persistents connections ? we will probably introduce a loadbalancing breaking point ? => LVS code is a routing software, so if we run LVS in a director CI/SSI virtualized environnement, the LVS connection table must be synced probably using a hook into the ICS (a ICS channel) to garanty loadbalancing decision integrity (consistence). Another point is the CVIP takeover, does the CVIP is a roaming VIP on the cluster ? if not what makes this CVIP takeover on another cluster node ? => Reading the patch, it figures out that CI nodes availability is checked with "clusternode_info" over a cluster NODENUM. So node availibility is returned according to your "keepalive" tool that inform and update the kernel CI cluster node map. This imply that CI/SSI must be installed on all CI/SSI cluster nodes and LVS too. This introduce the fact that a CI/SSI cluster is some kind of global virtual cluster including (with the loadbalancing eyes) LVS directors and realservers... This is my other perplexed point. To use CI healcheck framework, we need to hook kernel with CI patch, but we need to patch kernel with LVS code to provide loadbalancing part. So realservers and LVS director are global part of the same cluster exposed and a server can be LVS director and realserver (webserver for example) at a time. This limit the realserver pool to be linux ? I doesn t fully understand that point ... :/ => Conserning your patch, you have done all in the right place. My only inputs on your contributed code are : 1. +void ci_get_handler(vector strvec) +{ + nodemap = (nodenum_ip_map_t *)malloc(sizeof(nodenum_ip_map_t) *cluster_maxnodes()+1); /* zero is not considered as a valid node */ + if( initialize_nodemap(nodemap) < 0 ); + + /* How do i inform the main line code ? */ + queue_checker(NULL,dump_ci_check,ci_check_thread,NULL); +} the main line code is informed by queue_checker. During bottstrap Keepalived parse the configuration file and then queue a checker on each keyword known. After the keepalived.conf has been successfully parsed, keepalived de-queue each checker running the specific checker registration function. => this is the code in check_api.c the function register_checkers_thread(). 2. +int node_to_address(clusternode_t node, real_server *rs) +{ + + if( node > cluster_maxnodes() ) { + syslog(LOG_ERR,"Node number greater than Max node num \n"); + return -1; + } + + rs->addr_ip = nodemap[node].addr_ip; + /* do I need to fill the rest of the values here ? */ + + return 1; +} + real_server structure is filled during keepalived.conf parsing. In our case the configuration file looks like : virtual_server 10.10.10.2 80 { delay_loop 30 lb_algo rr lb_kind NAT protocol TCP real_server 192.168.200.6 80 { weight 1 CI-LINUX } } so the node_to_address() function should be suppress since realserver structure is filled into parser.c. Is there another way to get the node num from the IP address ? some kind of procfs entry so we can reduce again the code removing initialize_nodemap function ? Much more, if we can get this info through procfs, is it possible to create some kind of kernel reflection function that reflect nodemap status to an internal keepalived daemon nodestatus representation ? the code I am addressing is : +int nodestatus(real_server real) +{ + + int node_num; + clusternode_info_t ni; + + if((node_num = address_to_nodenum(real.addr_ip) ) == 0 ) + return UNKNOWN_NODE; + if(clusternode_info(node_num, sizeof(ni), &ni) >= 0) { + if (ni.node_state == CLUSTERNODE_UP ) + return UP; + else + /* I am insterested only in two state either fully up or down */ + return DOWN; + } + else { + syslog(LOG_ERR,"Error in getting the cluster information \n"); + } + return UNKNOWN_NODE; + +} If we can use some kind of kernel reflection this can be done with something like : +int nodestatus(real_server rs) +{ + + int status = 0; + + status = CI_NODE_STATE(rs); + if (status < 0) + return UNKNOWN_NODE; + return (status == CLUSTERNODE_UP)?UP:DOWN; +} CI_NODE_STATE() a simple C macro returning a daemon ci structure status flags that is updated by kernel broadcast through a kernel socket to the CI kernel hooks ? this the design keepalived run with netlink. Best regards, and thanks for your inputs, Alexandre |
|
From: Greg F. <fre...@No...> - 2002-05-08 19:07:41
|
Annesh, Note: I intentionally removed the keepalive list from the cc-list I just looked over your patch and it looks like you did a very good job of = keeping the changes isolated from the core keepalived code. I hope they accept your patch. Have you or anyone else explained the purpose of the patch and that the = SSI-Linux project is considering leveraging their project as a sub-component? I know the developer of Mosix was not at all happy that the SSI-Linux project = leveraged the load-leveler he developed and OpenSourced. =20 It might be a good idea get conceptual buy-in from the keepalive daemon = developers before incorporating their project into SSI. Maybe one of the Compaq (I mean HP) guys can talk further about that. BTW: Will you still be Digital India? Greg Freemyer Internet Engineer Deployment and Integration Specialist Compaq ASE - Tru64 Compaq Master ASE - SAN Architect The Norcross Group www.NorcrossGroup.com >> Hi,=20 >> Attaching below is the patch that add Cluster Infrastructure ( >> http://ci-linux.sourceforge.net ) as the health check mechanism for the >> keepalive daemons. Please comment on the patch so that i can continue >> working on it=20 >> -aneesh=20 >> diff -Nru --exclude-from=3D/tmp/ex.file keepalived-0.5.7/Makefile.in >> keepalived-0.5.7.new/Makefile.in >> --- keepalived-0.5.7/Makefile.in Fri May 3 00:33:26 2002 >> +++ keepalived-0.5.7.new/Makefile.in Wed May 8 21:22:03 2002 >> @@ -8,6 +8,7 @@ >> KERNEL :=3D _KRNL_2_$(shell uname -r | cut -d'.' -f2)_ >> IPVS_FLAG :=3D @IPVS_SUPPORT@ >> VRRP_FLAG :=3D @VRRP_SUPPORT@ >> +CI_LINUX :=3D @CI_LINUX@ >> =20 >> prefix =3D @prefix@ >> exec_prefix =3D @exec_prefix@ >> @@ -62,8 +63,12 @@ >> vrrp_ipsecah.o >> endif >> =20 >> +ifeq ($(CI_LINUX),_WITH_CI_LINUX_) >> +CI_LINUX_OBJ=3Dcheck_ci.c >> +endif=20 >> + >> INCLUDE =3D -I/usr/src/linux/include >> -OBJS =3D $(CORE_OBJS) $(IPVS_OBJS) $(VRRP_OBJS) >> +OBJS =3D $(CORE_OBJS) $(IPVS_OBJS) $(VRRP_OBJS) $(CI_LINUX_OBJS) >> =20 >> .c.o: >> $(CC) -o $@ $(CFLAGS) $(IPVSFLAGS) $(INCLUDE) -c $*.c >> diff -Nru --exclude-from=3D/tmp/ex.file keepalived-0.5.7/check_ci.c >> keepalived-0.5.7.new/check_ci.c >> --- keepalived-0.5.7/check_ci.c Thu Jan 1 05:30:00 1970 >> +++ keepalived-0.5.7.new/check_ci.c Wed May 8 02:19:03 2002 >> @@ -0,0 +1,152 @@ >> +#include "check_ci.h" >> +#include "parser.h" >> +#include "check_api.h" >> +#include "smtp.h" >> +#include "ipwrapper.h" >> + >> + >> +static nodenum_ip_map_t *nodemap; >> + >> +void dump_ci_check(void *data) >> +{ >> + >> + syslog(LOG_INFO, " Keepalive method =3D CI_LINUX"); >> + syslog(LOG_INFO, " Cluster Infrastructure for Linux"); >> +} >> + >> + >> +int ci_check_thread(thread *thread) >> +{ >> + checker *checker; >> + int status ; >> + checker =3D THREAD_ARG(thread); >> + >> + status =3D nodestatus(*(checker->rs)); >> + =20 >> + switch(status) { >> + >> + case UP: >> + if (!ISALIVE(checker->rs)) { >> + smtp_alert(thread->master, checker->rs >> + , NULL >> + , "UP" >> + , "=3D> CI-Linux CHECK succeed on service >> <=3D\n\n"); >> + perform_svr_state(UP, checker->vs, checker->rs); >> + } >> + break; >> + case DOWN: >> + if (ISALIVE(checker->rs)) { >> + smtp_alert(thread->master, checker->rs >> + , NULL >> + , "DOWN" >> + , "=3D> TCP CHECK failed on service >> <=3D\n\n"); >> + perform_svr_state(DOWN, checker->vs, checker->rs); >> + } >> + break; >> + default: >> + syslog(LOG_ERR,"Unknown node status \n"); >> + } >> + >> + >> + /* Register the next check */ >> + thread_add_timer(thread->master, ci_check_thread >> + , checker >> + , checker->vs->delay_loop); >> + return 0; >> + >> +} >> + >> +void ci_get_handler(vector strvec) >> +{ >> + nodemap =3D (nodenum_ip_map_t >> *)malloc(sizeof(nodenum_ip_map_t)*cluster_maxnodes()+1); /* zero is not >> considered as a valid node */ >> + if( initialize_nodemap(nodemap) < 0 ); >> + >> + /* How do i inform the main line code ? */ >> + =20 >> + queue_checker(NULL,dump_ci_check,ci_check_thread,NULL); >> +} >> + >> +void install_ci_check_keyword(void) >> +{ >> + install_keyword("CI-LINUX", &ci_get_handler); >> +} >> +int initialize_nodemap(nodenum_ip_map_t *nodemap) >> +{ >> + FILE *fp; >> + char buf[BUFFSIZE]; >> + int node_number ; >> + >> + if( (fp =3D fopen(CLUSTERTAB,"r")) =3D=3D NULL ) >> + return -1; >> + while(fscanf(fp,"%s",buf) !=3D EOF ){ >> + if( buf[0] =3D=3D '#' ){ >> + if(fscanf (fp, "%[^\n]",buf) =3D=3D EOF ) >> + syslog(LOG_ERR," %s File Format Error \n", >> + CLUSTERTAB); >> + bzero(buf,BUFFSIZE); >> + continue; >> + } >> + node_number =3D atoi(buf); >> + if ( node_number > cluster_maxnodes()) { >> + syslog(LOG_ERR,"Node number greater than MAX node num\n"); >> + return -1; >> + } >> + if(fscanf (fp, "%s",buf) =3D=3D EOF ) >> + syslog(LOG_ERR," %s File Format Error \n",CLUSTERTAB); >> + nodemap[node_number].addr_ip =3D inet_addr(buf); >> + >> + if(fscanf (fp, "%[^\n]",buf) =3D=3D EOF ) >> + syslog(LOG_ERR," %s File Format Error \n",CLUSTERTAB); >> + bzero(buf,BUFFSIZE); >> + >> + } >> + >> + >> + return 1; >> +} >> + >> +clusternode_t address_to_nodenum(uint32_t addr_ip) >> +{ >> + int i ; >> + int max_nodes =3D cluster_maxnodes(); >> + for( i =3D 1 ; i<=3D max_nodes;i++) { >> + if( nodemap[i].addr_ip =3D=3D addr_ip ) >> + return i; >> + } >> + return 0; /* Not a valid node */ >> +} >> +int node_to_address(clusternode_t node, real_server *rs) >> +{ >> + >> + if( node > cluster_maxnodes() ) { >> + syslog(LOG_ERR,"Node number greater than Max node num \n"); >> + return -1; >> + } >> + >> + rs->addr_ip =3D nodemap[node].addr_ip; >> +/* do I need to fill the rest of the values here ? */ >> + =20 >> + return 1; >> +} >> + >> +int nodestatus(real_server real) >> +{ >> + >> + int node_num; >> + clusternode_info_t ni; >> + >> + if((node_num =3D address_to_nodenum(real.addr_ip) ) =3D=3D 0 ) >> + return UNKNOWN_NODE; >> + if(clusternode_info(node_num, sizeof(ni), &ni) >=3D 0) { >> + if (ni.node_state =3D=3D CLUSTERNODE_UP ) >> + return UP; >> + else=20 >> +/* I am insterested only in two state either fully up or down */ >> + return DOWN; =20 >> + } >> + else { >> + syslog(LOG_ERR,"Error in getting the cluster information >> \n"); >> + } >> + return UNKNOWN_NODE; >> + >> +} >> diff -Nru --exclude-from=3D/tmp/ex.file keepalived-0.5.7/check_ci.h >> keepalived-0.5.7.new/check_ci.h >> --- keepalived-0.5.7/check_ci.h Thu Jan 1 05:30:00 1970 >> +++ keepalived-0.5.7.new/check_ci.h Wed May 8 21:11:59 2002 >> @@ -0,0 +1,34 @@ >> +#ifndef _CI_H >> +#define _CI_H >> +#include <signal.h> >> +#include <pthread.h> >> +#include <linux/cluster.h> /* Should change this to cluster.h alone */ >> + >> +#include <syslog.h> >> +#include <sys/socket.h> >> +#include <netinet/in.h> >> +#include <arpa/inet.h> >> + >> +#include "data.h" >> + >> +#define SIGCLUSTER 2 >> +#define CLUSTERTAB "/etc/clustertab" >> +#define BUFFSIZE 100 >> +#define UP 1 >> +#define DOWN 2 >> +#define UNKNOWN_NODE 0 >> + >> + >> +typedef struct nodenum_ip_map{ >> + uint32_t addr_ip; >> +} nodenum_ip_map_t; >> + >> +extern int initialize_nodemap(nodenum_ip_map_t *nodemap); >> +extern int node_to_address(clusternode_t node, real_server *rs); >> +extern clusternode_t address_to_nodenum(uint32_t addr_ip); >> +extern int nodestatus(real_server real); >> + >> + >> + >> +#endif /* _CI_H */ >> + =20 >> diff -Nru --exclude-from=3D/tmp/ex.file keepalived-0.5.7/configure.in >> keepalived-0.5.7.new/configure.in >> --- keepalived-0.5.7/configure.in Fri May 3 01:05:03 2002 >> +++ keepalived-0.5.7.new/configure.in Wed May 8 23:03:38 2002 >> @@ -56,6 +56,9 @@ >> [ --disable-vrrp do not use the VRRP framework]) >> AC_ARG_ENABLE(debug, >> [ --enable-debug compile with debugging flags]) >> +AC_ARG_WITH(ci_linux, >> + [--with-ci_linux, Compile with Cluster Infrastructure support( defalut >> no)], >> + ci_linux=3D$withval,ci_linux=3D"no") >> =20 >> if test "${enable_lvs}" =3D "no"; then >> if test "${enable_vrrp}" =3D "no"; then >> @@ -77,6 +80,10 @@ >> DFLAGS=3D"-D_DEBUG_" >> AC_SUBST(DFLAGS) >> fi >> +if test "${ci_linux}" =3D "yes"; then >> +CI_LINUX=3D"_WITH_CI_LINUX_" >> +AC_SUBST(CI_LINUX) >> +fi >> =20 >> AC_SUBST(IPVS_SUPPORT) >> AC_SUBST(VRRP_SUPPORT) >> @@ -111,6 +118,9 @@ >> AC_CHECK_LIB(crypto, MD5_Init,,AC_MSG_ERROR([OpenSSL libraries are >> required])) >> AC_CHECK_LIB(ssl, SSL_CTX_new,,AC_MSG_ERROR([OpenSSL libraries are >> required])) >> AC_CHECK_LIB(popt, poptGetContext,,AC_MSG_ERROR([Popt libraries is >> required])) >> +if test "${ci_linux}" =3D "yes"; then >> +AC_CHECK_LIB(cluster,cluster_maxnodes,,AC_MSG_ERROR([libcluster libraries >> are required])) >> +fi >> =20 >> dnl ----[ Create object list ]---- >> if test "${KERNEL_CODE}" =3D "2"; then >> @@ -127,6 +137,9 @@ >> !!! OpenSSL is not properly installed on your system. !!! >> !!! Can not include OpenSSL headers files. !!!])) >> AC_CHECK_HEADERS(fcntl.h sys/ioctl.h sys/time.h syslog.h unistd.h) >> +if test "${ci_linux}" =3D "yes"; then >> +AC_CHECK_HEADERS(linux/cluster.h,,AC_MSG_ERROR([linux/cluster.h file not >> found])) >> +fi >> =20 >> dnl ----[ Checks for typedefs, structures, and compiler characteristics >> ]---- >> AC_C_CONST >> _______________________________________________________________ >> Have big pipes? SourceForge.net is looking for download mirrors. We supply >> the hardware. You get the recognition. Email Us: ban...@so... >> _______________________________________________ >> ssic-linux-devel mailing list >> ssi...@li... >> https://lists.sourceforge.net/lists/listinfo/ssic-linux-devel |
|
From: Aneesh K. K.V <ane...@di...> - 2002-05-08 11:49:24
|
Hi, Attaching below is the patch that add Cluster Infrastructure ( http://ci-linux.sourceforge.net ) as the health check mechanism for the keepalive daemons. Please comment on the patch so that i can continue working on it -aneesh diff -Nru --exclude-from=/tmp/ex.file keepalived-0.5.7/Makefile.in keepalived-0.5.7.new/Makefile.in --- keepalived-0.5.7/Makefile.in Fri May 3 00:33:26 2002 +++ keepalived-0.5.7.new/Makefile.in Wed May 8 21:22:03 2002 @@ -8,6 +8,7 @@ KERNEL := _KRNL_2_$(shell uname -r | cut -d'.' -f2)_ IPVS_FLAG := @IPVS_SUPPORT@ VRRP_FLAG := @VRRP_SUPPORT@ +CI_LINUX := @CI_LINUX@ prefix = @prefix@ exec_prefix = @exec_prefix@ @@ -62,8 +63,12 @@ vrrp_ipsecah.o endif +ifeq ($(CI_LINUX),_WITH_CI_LINUX_) +CI_LINUX_OBJ=check_ci.c +endif + INCLUDE = -I/usr/src/linux/include -OBJS = $(CORE_OBJS) $(IPVS_OBJS) $(VRRP_OBJS) +OBJS = $(CORE_OBJS) $(IPVS_OBJS) $(VRRP_OBJS) $(CI_LINUX_OBJS) .c.o: $(CC) -o $@ $(CFLAGS) $(IPVSFLAGS) $(INCLUDE) -c $*.c diff -Nru --exclude-from=/tmp/ex.file keepalived-0.5.7/check_ci.c keepalived-0.5.7.new/check_ci.c --- keepalived-0.5.7/check_ci.c Thu Jan 1 05:30:00 1970 +++ keepalived-0.5.7.new/check_ci.c Wed May 8 02:19:03 2002 @@ -0,0 +1,152 @@ +#include "check_ci.h" +#include "parser.h" +#include "check_api.h" +#include "smtp.h" +#include "ipwrapper.h" + + +static nodenum_ip_map_t *nodemap; + +void dump_ci_check(void *data) +{ + + syslog(LOG_INFO, " Keepalive method = CI_LINUX"); + syslog(LOG_INFO, " Cluster Infrastructure for Linux"); +} + + +int ci_check_thread(thread *thread) +{ + checker *checker; + int status ; + checker = THREAD_ARG(thread); + + status = nodestatus(*(checker->rs)); + + switch(status) { + + case UP: + if (!ISALIVE(checker->rs)) { + smtp_alert(thread->master, checker->rs + , NULL + , "UP" + , "=> CI-Linux CHECK succeed on service <=\n\n"); + perform_svr_state(UP, checker->vs, checker->rs); + } + break; + case DOWN: + if (ISALIVE(checker->rs)) { + smtp_alert(thread->master, checker->rs + , NULL + , "DOWN" + , "=> TCP CHECK failed on service <=\n\n"); + perform_svr_state(DOWN, checker->vs, checker->rs); + } + break; + default: + syslog(LOG_ERR,"Unknown node status \n"); + } + + + /* Register the next check */ + thread_add_timer(thread->master, ci_check_thread + , checker + , checker->vs->delay_loop); + return 0; + +} + +void ci_get_handler(vector strvec) +{ + nodemap = (nodenum_ip_map_t *)malloc(sizeof(nodenum_ip_map_t)*cluster_maxnodes()+1); /* zero is not considered as a valid node */ + if( initialize_nodemap(nodemap) < 0 ); + + /* How do i inform the main line code ? */ + + queue_checker(NULL,dump_ci_check,ci_check_thread,NULL); +} + +void install_ci_check_keyword(void) +{ + install_keyword("CI-LINUX", &ci_get_handler); +} +int initialize_nodemap(nodenum_ip_map_t *nodemap) +{ + FILE *fp; + char buf[BUFFSIZE]; + int node_number ; + + if( (fp = fopen(CLUSTERTAB,"r")) == NULL ) + return -1; + while(fscanf(fp,"%s",buf) != EOF ){ + if( buf[0] == '#' ){ + if(fscanf (fp, "%[^\n]",buf) == EOF ) + syslog(LOG_ERR," %s File Format Error \n", + CLUSTERTAB); + bzero(buf,BUFFSIZE); + continue; + } + node_number = atoi(buf); + if ( node_number > cluster_maxnodes()) { + syslog(LOG_ERR,"Node number greater than MAX node num\n"); + return -1; + } + if(fscanf (fp, "%s",buf) == EOF ) + syslog(LOG_ERR," %s File Format Error \n",CLUSTERTAB); + nodemap[node_number].addr_ip = inet_addr(buf); + + if(fscanf (fp, "%[^\n]",buf) == EOF ) + syslog(LOG_ERR," %s File Format Error \n",CLUSTERTAB); + bzero(buf,BUFFSIZE); + + } + + + return 1; +} + +clusternode_t address_to_nodenum(uint32_t addr_ip) +{ + int i ; + int max_nodes = cluster_maxnodes(); + for( i = 1 ; i<= max_nodes;i++) { + if( nodemap[i].addr_ip == addr_ip ) + return i; + } + return 0; /* Not a valid node */ +} +int node_to_address(clusternode_t node, real_server *rs) +{ + + if( node > cluster_maxnodes() ) { + syslog(LOG_ERR,"Node number greater than Max node num \n"); + return -1; + } + + rs->addr_ip = nodemap[node].addr_ip; +/* do I need to fill the rest of the values here ? */ + + return 1; +} + +int nodestatus(real_server real) +{ + + int node_num; + clusternode_info_t ni; + + if((node_num = address_to_nodenum(real.addr_ip) ) == 0 ) + return UNKNOWN_NODE; + if(clusternode_info(node_num, sizeof(ni), &ni) >= 0) { + if (ni.node_state == CLUSTERNODE_UP ) + return UP; + else +/* I am insterested only in two state either fully up or down */ + return DOWN; + } + else { + syslog(LOG_ERR,"Error in getting the cluster information \n"); + } + return UNKNOWN_NODE; + +} diff -Nru --exclude-from=/tmp/ex.file keepalived-0.5.7/check_ci.h keepalived-0.5.7.new/check_ci.h --- keepalived-0.5.7/check_ci.h Thu Jan 1 05:30:00 1970 +++ keepalived-0.5.7.new/check_ci.h Wed May 8 21:11:59 2002 @@ -0,0 +1,34 @@ +#ifndef _CI_H +#define _CI_H +#include <signal.h> +#include <pthread.h> +#include <linux/cluster.h> /* Should change this to cluster.h alone */ + +#include <syslog.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <arpa/inet.h> + +#include "data.h" + +#define SIGCLUSTER 2 +#define CLUSTERTAB "/etc/clustertab" +#define BUFFSIZE 100 +#define UP 1 +#define DOWN 2 +#define UNKNOWN_NODE 0 + + +typedef struct nodenum_ip_map{ + uint32_t addr_ip; +} nodenum_ip_map_t; + +extern int initialize_nodemap(nodenum_ip_map_t *nodemap); +extern int node_to_address(clusternode_t node, real_server *rs); +extern clusternode_t address_to_nodenum(uint32_t addr_ip); +extern int nodestatus(real_server real); + + + +#endif /* _CI_H */ + diff -Nru --exclude-from=/tmp/ex.file keepalived-0.5.7/configure.in keepalived-0.5.7.new/configure.in --- keepalived-0.5.7/configure.in Fri May 3 01:05:03 2002 +++ keepalived-0.5.7.new/configure.in Wed May 8 23:03:38 2002 @@ -56,6 +56,9 @@ [ --disable-vrrp do not use the VRRP framework]) AC_ARG_ENABLE(debug, [ --enable-debug compile with debugging flags]) +AC_ARG_WITH(ci_linux, + [--with-ci_linux, Compile with Cluster Infrastructure support( defalut no)], + ci_linux=$withval,ci_linux="no") if test "${enable_lvs}" = "no"; then if test "${enable_vrrp}" = "no"; then @@ -77,6 +80,10 @@ DFLAGS="-D_DEBUG_" AC_SUBST(DFLAGS) fi +if test "${ci_linux}" = "yes"; then +CI_LINUX="_WITH_CI_LINUX_" +AC_SUBST(CI_LINUX) +fi AC_SUBST(IPVS_SUPPORT) AC_SUBST(VRRP_SUPPORT) @@ -111,6 +118,9 @@ AC_CHECK_LIB(crypto, MD5_Init,,AC_MSG_ERROR([OpenSSL libraries are required])) AC_CHECK_LIB(ssl, SSL_CTX_new,,AC_MSG_ERROR([OpenSSL libraries are required])) AC_CHECK_LIB(popt, poptGetContext,,AC_MSG_ERROR([Popt libraries is required])) +if test "${ci_linux}" = "yes"; then +AC_CHECK_LIB(cluster,cluster_maxnodes,,AC_MSG_ERROR([libcluster libraries are required])) +fi dnl ----[ Create object list ]---- if test "${KERNEL_CODE}" = "2"; then @@ -127,6 +137,9 @@ !!! OpenSSL is not properly installed on your system. !!! !!! Can not include OpenSSL headers files. !!!])) AC_CHECK_HEADERS(fcntl.h sys/ioctl.h sys/time.h syslog.h unistd.h) +if test "${ci_linux}" = "yes"; then +AC_CHECK_HEADERS(linux/cluster.h,,AC_MSG_ERROR([linux/cluster.h file not found])) +fi dnl ----[ Checks for typedefs, structures, and compiler characteristics ]---- AC_C_CONST |
|
From: John B. <joh...@hp...> - 2002-05-08 01:22:49
|
Migrate and rfork now work under UML. As a part of that, I have changed the UML patch we use to 2.4.18-22. Apply this patch to a 2.4.16 kernel. (Makefile will reject; everything else applies with very little fuzz.) This change is incompatible with the previously released SSI UML rootfs. Brian is working on an updated rootfs and initrd. You will also need to update your UML tools to the latest version (or so) on your host. Brian is doing some final checks on the rootfs and should begin pushing it to sourcefource soon. John Byrne P.S. I still have some imports to do, but this shouldn't directly affect anything. |
|
From: Aneesh K. K.V <ane...@di...> - 2002-04-27 05:32:16
|
Hi Brian , The installer/manager I was discussing is run after a Linux system is fully installed. That is one install his favorite Linux distribution and after that he runs this utility cluster_manager. If the cluster manager finds that it is not running on SSI cluster it should first guide the user to install one. The method I discussed in my previous mail could help here. If found to be running on a SSI cluster it just behaves as a cluster manager. -aneesh |
|
From: Brian J. W. <Bri...@co...> - 2002-04-26 20:52:55
|
Sorry for the delay in responding. A couple of weeks ago, I was in a rush to finish the draft of the SSI/UML Howto before I went on vacation. > I was wondering why we don't have a GUI installer for SSI cluster. Basically, it was because of time constraints. I did most of the work of writing the installation instructions and developing the netboot tools to simplify dependent node installation. I would have loved to go on and implement a user-friendly GUI installer, but with the limited number of active SSI developers, I really need to work on other things. This seems like a good project for a new SSI developer to work on, especially one with GUI design experience. > (1). I will install a Debian ( I am a debian lover even though i use > redhat for my work :) ) Linux on my machine. Cool. The SSI installer should support as many as distributions as possible, limited only by the availibility of developers who work on a given distribution. I only work and test on Red Hat (more out of comfort than any deeply held religious beliefs), but I try to be good about making changes to the Debian and Caldera specific files in Cluster Tools whenever applicable. > (2) Get the latest kernel releases from kernel.org. Nah. There should be binary releases of the SSI and CI kernels. Building kernels is for developers and bleeding-edge users working from CVS. Maybe an interim step someone could work on is figuring out how to roll binary packages for SSI and CI. > (3) Get the SSI utilites ( which include cluster tools . Dlm daemon > (dlmdu ) and the GUI-cluster build utility from SSI Website AND DHCP > DAEMON ( why I will tell later in step 7 and 8 ). Could be a .deb or > .rpm This would be nice with the Debian package manager, since it can pull down all packages that a desired package depends on. It can also immediately launch the GUI installer (or a faux GUI installer if X is not running). > (7)He start the cluster gui. The cluster GUI detect that he is running > in a SSI cluster so he wont give the option of building a cluster. Now > there will be lot of tabbed window most of them for monitoring cluster > events resources etc. A good GUI designer is needed here :) Ahh... you're describing a GUI cluster manager, rather than just a GUI installer. That would be nice to have eventually. > Click on the > tabbed window ADD node. Here he will display all the hardware address > that is there in the DHCP log file. what i found is that most of the > distribution comes with a different log file for DHCP and we cannot > expect that we can get the MAC address from /var/log/messages some of > them doesn't enable logging. We should not ask the user to enable it and > put it in this file . I admit that I didn't put much thought into this at the time. For those distributions that have dhcpd logging somewhere other than /var/log/messages, someone could add a little more intelligence to the addnode command to deal with this. > So we make a slight modification to dhcp that > will put all the hardware address to log file that we name . That is > the reason we give a different dhcp server. We should be careful about modifying as few commands as possible. It seems better to request that a distribution maintainer turn on logging in dhcpd, rather than distribute our own slightly modified version of the command. > ( There should be a button > that clear the content of this file. Because if your subnet falls in the > same subnet as the organization then your dhcp server will have the > MAC address of all those machines in the subnet that is sending > messages. ) The MAC addresses are listed in order of most to least recent. Regardless of how long the list is, the desired MAC address should be near the top. -- Brian Watson | "Now I don't know, but I been told it's Software Developer | hard to run with the weight of gold, Open SSI Clustering Project | Other hand I heard it said, it's Compaq Computer Corp | just as hard with the weight of lead." | -Robert Hunter, 1970 mailto:Bri...@co... http://opensource.compaq.com/ |
|
From: David B. Z. <dav...@co...> - 2002-04-22 18:16:14
|
Resent from Fri April 22: Some minor changes have occured in the CI tree. Unfortunately, I didn't have time today to verify that a CI kernel builds and boots. I will verify that nothing is broken Monday morning. |
1 message has been excluded from this view by a project administrator.
