Re: [SSI-devel] new checkins wrt to ipvs load balancing

[Bruce W. <br...@ka...>]

Aneesh,
I'm concerned about the mgmt/admin overhead of this approach.
Could we just automatically do listens that come from specific port
bindings and not load balance ephemeral ports?  Below is
a writeup dealing with this w.r.t. portmapper and NFS.

bruce

Aneesh,
  David Zafman alerted me to the portmap problem, which you probably realized 
with respect to having all listens register with LVS.  There are some
implementation details to handle but I would suggest the following strategy:
  - listens done from binds to a specfic port are automatically registered;
  - listens done from binds which request a port (and get an ephemeral port)
       are not automatically registered with LVS.

Portmapper:
  - there is one per node and typically it has registered with it programs
      bound on ephemeral ports and thus not registered with LVS.
  - David Zafman is going to see how easily we can have a portmapper per node.
  - something extra is needed to deal with NFS server and perhaps other
       apps which want to listen on CVIP but are ephemeral (more on this later).

NFS Server:
  - the tentative plan is for their to be a NFS client per node (along with
       a client statd and client lockd) - David has already done this except
       perhaps for the portmapper issue.  The outgoing requests will use the
       IP address of each node (not the CVIP) and will use a hostname
       associated with that address (so the remote server can deal with
       your failure and reboot, etc.).
  - the NFS server will listen only on CVIP(s).  There will be another
       server-only lockd and server-only statd that will work only with respect
       to the CVIP(s) (note - not clear exactly how multiple CVIPs work yet
       but I think we might have a statd/lockd/nfsd per CVIP, to handle all
       the failure cases).  Question is, how to we handle these multiple
       statd's in portmapper?"

Plan:
       1. Somehow we get the info to the portmapper on the CVIP node.
       2. That portmapper now knows that if a request comes on CVIP it
	answers one way (with the server statd/lockd port) and otherwise
	it answers with the client statd/lockd port.
       3. On failover of the CVIP, a new statd/lockd are set up for the
	server and registed with the portmapper on that node;  the
	server statd that is started up will send the "I just rebooted" 
	message to all the remote clients who will resend their locks so
	we can rebuild.  This will not affect all the cluster-as-a-NFS-client
	locks.

So how to get the info to portmapper?  Dave is looking into how we do this
in a non-invasive sort of way.  What we need is to have two instances of the
same program on two different ports; one is for CVIP only and the other is
for any other address.


> Hi,
> 
> I did some checking wrt to ipvs
> 
> 1) Add file /proc/cluster/ip_vs_portweight ( This file is NOT cluster
> wide it shows the port range and the weight associated to ports on the
> particular node. That means on each node the content will be different )
> 
> 2) By default ports are not balanced/registered with ipvs. 
> 
> 3) IF you want to register ports automatically with ipvs first you have
> add it to the port balancing list. This can be done by using the new
> command /usr/sbin/setport_weight  ( Please build cluster tools after
> running ./configure  )
> 
> Example: 
> First you can register a port range 
> 
> /usr/sbin/setport_weight --start-port=0 --end-port=1024 --weight=1 
> 
> and then say
> 
> /etc/init.d/xinetd restart 
> 
> Good luck
> -aneesh 
> 
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: Etnus, makers of TotalView, The debugger 
> for complex code. Debugging C/C++ programs can leave you feeling lost and 
> disoriented. TotalView can help you find your way. Available on major UNIX 
> and Linux platforms. Try it free. www.etnus.com
> _______________________________________________
> ssic-linux-devel mailing list
> ssi...@li...
> https://lists.sourceforge.net/lists/listinfo/ssic-linux-devel