Re: [SSI-users] 2.6.14-smp kernel headers and sources

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi All,

I found the problem.

It turns out that the problem was in the compile source configs in that I had changed the include/cluster/config.h file to increase the max number of nodes from 15 to 125:

(ORIGINAL) ----------------------------------------
#ifndef _CLUSTER_CONFIG_H
#define _CLUSTER_CONFIG_H

#include <linux/param.h>
#include <linux/threads.h>

/*
 * Define the maximum clusternode_t value actually supported, i.e. each
 * subsystem must be able to manage/allocate for nodes 1 - NSC_MAX_NODE_VALUE.
 * Should be a runtime value, but that requires more support not yet
 * available to a) dynamically size all relevant data structures and b)
 * make sure root node's value is passed to joining nodes, and
 * supersedes their own.
 * When changing this value, please be aware of the limits specified below.
 * Setting this smaller saves quite a bit of memory.
 */
/* #define NSC_MAX_NODE_VALUE  125 */ /* default */
#define NSC_MAX_NODE_VALUE  15

/*
 * Define the part of the pid_t reserved for clusternode_t values.
 * Needed regardless of VPROC support in order to generate NSC_NODE_T_RANGE_MAX
 */
#define NODESHIFT       16      /* used by VPROC */
//#define NODESHIFT     23      /* reserve 4 million PIDs per node */

/*
 * Derive the maximum valid clusternode_t value, used ONLY for data-type
 * range checking
 * The PID_MAX defined in threads.h provides a limit to the
 * number of nodes that can be operating at one time.
 */
#if defined(VPROC) || defined(CONFIG_CPID)
#define NSC_NODE_T_RANGE_MAX    (PID_MAX_LIMIT >> NODESHIFT)

#if NSC_MAX_NODE_VALUE > NSC_NODE_T_RANGE_MAX
        #error "NSC_MAX_NODE_VALUE is too large to fit into pid number"
#endif
#endif

#define MAXLOCPID       ((1<<NODESHIFT)-1)  /* number at which pids recycle */
#define LAST_DAEMON_PID 300     /* RESERVED_PIDS */

#define CLUSTER_IFCONFIG_SZ     64
#define CLUSTER_CLMSINFO_SZ     256
#define CLUSTER_ICSROUTE_SZ     64

#endif /* !_CLUSTER_CONFIG_H */
--------------------------------------------------------

When I made the change, the system just crashed out the nodes while booting and I am not really sure why.

The fix, was to set the values back to the default settings like above.

I will try again to make a smaller adjustment to something like 30 nodes or something to see if that works as I had tried 125 and created the crash on the nodes.

Is there a dynamical way to change the max nodes without having to re-compile the kernel again?

Thanks and have a great day,
Lonnie Cumberland, Prof.
Physicist

> -----Original Message-----
> From: Cumberland, Lonnie
> Sent: Wednesday, August 18, 2010 12:57 PM
> To: Cumberland, Lonnie; John Hughes; Mulyadi Santosa
> Cc: Openssi users
> Subject: RE: [SSI-users] 2.6.14-smp kernel headers and sources
> 
> Hello All,
> 
> Well, after testing a number of different things, it turns out that the
> main node seems to be running great and even the fuse module is
> working, but that the other nodes are not coming online. They seem to
> get the kernel, boot the kernel and then crash out.
> 
> Thanks and have a great day,
> Lonnie Cumberland, Prof.
> Physicist
> 
> 
> > -----Original Message-----
> > From: Cumberland, Lonnie [mailto:lon...@ni...]
> > Sent: Wednesday, August 18, 2010 10:18 AM
> > To: John Hughes; Mulyadi Santosa
> > Cc: Openssi users
> > Subject: Re: [SSI-users] 2.6.14-smp kernel headers and sources
> >
> > Hello All,
> >
> > Well, there is good news and bad news.
> >
> > The good news is that the compile went well although I did not know
> how
> > to get the "make" commandline switches to SMP and i686 optizimations.
> >
> > The bad news is that even though the kernel seems to boot ok on the
> > main node, the other nodes do not seems to complete the booting
> > properly and crash out after tftp send over the new kernel which I
> put
> > into place via "ssi-ksync".
> >
> > There was a document on the OpenSSI website that said to do:
> >
> > # make oldconfig  		(by using the .config that was previously
> > set up)
> > # make dep        		(the compile reported that this was
> > unnecessary now)
> > # make bzImage    		(this went fine)
> > # make modules    		(this went fine)
> > # make modules_install    	(this went fine)
> >
> > # cp arch/i386/boot/bzImage /boot/vmlinuz-2.6.14_ssi  (this went
> well)
> >
> > # mkinitrd --cfs /boot/initrd-2.6.14_ssi.img  2.6.14__ssi   (PROBLEMS
> > HERE: /usr/sbin/mkinitrd: unrecognized option `--cfs')
> >
> > Instead, I did:
> >
> > # mkinitrd -o /boot/initrd-2.6.14_ssi.img  2.6.14__ssi  (since I did
> > this in the install and it went well)
> >
> > But still the other nodes do not seem to come online.
> >
> > Any ideas on how to fix this?
> >
> > Thanks and have a great day,
> > Lonnie Cumberland, Prof.
> > Physicist
> >
> > > -----Original Message-----
> > > From: John Hughes [mailto:jo...@Ca...]
> > > Sent: Wednesday, August 18, 2010 7:34 AM
> > > To: Mulyadi Santosa
> > > Cc: Cumberland, Lonnie; Openssi users
> > > Subject: Re: [SSI-users] 2.6.14-smp kernel headers and sources
> > >
> > > Mulyadi Santosa wrote:
> > > > On Wed, Aug 18, 2010 at 05:42, Cumberland, Lonnie
> > > > <lon...@ni...> wrote:
> > > >
> > > >> I'll look to see what version I have tomorrow morning.
> > > >>
> > > >> Do I have to remove the whole 4.0 version if that is the problem
> > and
> > > install either 3.4 or 3.5 as you mentioned?
> > > >>
> > > >
> > > > Nope, in Fedora/RHEL, this is usually called gcc-compat or
> compat-
> > gcc
> > > > to refer to the "old" gcc. So for example, if Fedora 5 provides
> gcc
> > > > 3.x and 4.x, the 3.x is installed by gcc-compat. Thus, what you
> > need
> > > > to remove the gcc package (but not the compat one).
> > > >
> > > On Debian just install the gcc-3.4 package.
> > >
> > > apt-get install gcc-3.4
> > >
> >
> >
> > ---------------------------------------------------------------------
> --
> > -------
> > This SF.net email is sponsored by
> >
> > Make an app they can't live without
> > Enter the BlackBerry Developer Challenge
> > http://p.sf.net/sfu/RIM-dev2dev
> > _______________________________________________
> > Ssic-linux-users mailing list
> > Ssi...@li...
> > https://lists.sourceforge.net/lists/listinfo/ssic-linux-users