|
From: Advice P. <LA...@gm...> - 2009-04-19 15:14:45
|
I read this in the Wikipedia: "OpenSSI is designed to be used for both high performance and high availability clusters, it is possible to create an OpenSSI cluster with no single point of failure, for example the file system can be mirrored between two nodes, so if one node crashes the process accessing the file will fail over to the other node. Alternatively the cluster can be designed in such a manner that every node has direct access to the file system." -- View this message in context: http://www.nabble.com/How-do-I-make-every-node-have-direct-access-to-the-file-system--tp23123954p23123954.html Sent from the ssic-linux-users mailing list archive at Nabble.com. |
|
From: John H. <jo...@Ca...> - 2009-04-20 08:52:43
|
Advice Pro wrote: > I read this in the Wikipedia: > > "OpenSSI is designed to be used for both high performance and high > availability clusters, it is possible to create an OpenSSI cluster with no > single point of failure, for example the file system can be mirrored between > two nodes, so if one node crashes the process accessing the file will fail > over to the other node. Alternatively the cluster can be designed in such a > manner that every node has direct access to the file system." > Use GFS or Lustre. I don't think anyone is working on this at the moment, so the code has probably bit-rotted to oblivion. |
|
From: <Co...@Vi...> - 2009-05-08 09:43:49
|
John Hughes wrote: > > Advice Pro wrote: >> I read this in the Wikipedia: >> >> "OpenSSI is designed to be used for both high performance and high >> availability clusters, it is possible to create an OpenSSI cluster with >> no >> single point of failure, for example the file system can be mirrored >> between >> two nodes, so if one node crashes the process accessing the file will >> fail >> over to the other node. Alternatively the cluster can be designed in such >> a >> manner that every node has direct access to the file system." >> > Use GFS or Lustre. I don't think anyone is working on this at the > moment, so the code has probably bit-rotted to oblivion. > > ------------------------------------------------------------------------------ > Stay on top of everything new and different, both inside and > around Java (TM) technology - register by April 22, and save > $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. > 300 plus technical and hands-on sessions. Register today. > Use priority code J9JMT32. http://p.sf.net/sfu/p > _______________________________________________ > Ssic-linux-users mailing list > Ssi...@li... > https://lists.sourceforge.net/lists/listinfo/ssic-linux-users > > i am trying following apporoach: the root file system resides on a nfs share. every node has a harddisk with only the boot partition on it. we want to achieve, that every slave node can become master node so that all nodes must die before cluster gets offline. any better ideas somebody? regards cc -- View this message in context: http://www.nabble.com/How-do-I-make-every-node-have-direct-access-to-the-file-system--tp23123954p23443046.html Sent from the ssic-linux-users mailing list archive at Nabble.com. |
|
From: <Co...@Vi...> - 2009-05-08 20:32:17
|
CooLCaT@Vienna wrote: > > > i am trying following apporoach: > > the root file system resides on a nfs share. every node has a harddisk > with only the boot partition on it. we want to achieve, that every slave > node can become master node so that all nodes must die before cluster gets > offline. > > any better ideas somebody? > > regards > > cc > > ok after reading this: *** NOTE: NFS client is not yet supported in the new OpenSSI 2.6 kernel *** i guess i have to do something else: actually we want that every slave node has the ability to get a master node. as far as i know drbd can only mirror two nodes. my approach was, that simply change the root file system to a nfs share and leave the boot partition on the local machine, but it seems that this isnt working as expected. i always thought that when i use drbd, that if the master node goes down, the next potential node mounts the filesystem and takes over. did i missunderstood? regards cc -- View this message in context: http://www.nabble.com/How-do-I-make-every-node-have-direct-access-to-the-file-system--tp23123954p23452013.html Sent from the ssic-linux-users mailing list archive at Nabble.com. |
|
From: Roger T. <rog...@gm...> - 2009-05-08 22:57:55
|
On Fri, May 8, 2009 at 4:32 PM, CooLCaT@Vienna <co...@co...> wrote: > > > > CooLCaT@Vienna wrote: > > > > > > i am trying following apporoach: > > > > the root file system resides on a nfs share. every node has a harddisk > > with only the boot partition on it. we want to achieve, that every slave > > node can become master node so that all nodes must die before cluster > gets > > offline. > > > > any better ideas somebody? > > > > regards > > > > cc > > > > > > ok > > after reading this: > > *** NOTE: NFS client is not yet supported in the new OpenSSI 2.6 kernel *** > > i guess i have to do something else: > > actually we want that every slave node has the ability to get a master > node. > as far as i know drbd can only mirror two nodes. > > my approach was, that simply change the root file system to a nfs share and > leave the boot > partition on the local machine, but it seems that this isnt working as > expected. You already tried? What problems did you encounter? It is not yet supported only because we haven't looked at fixing parallel NFS. There is only one bug report in our Sourceforge tracker and there doesn't seem to be much demand for the fix (until recently). i always thought that when i use drbd, that if the master node goes down, > the next potential node mounts the filesystem and takes over. > > did i missunderstood? You were talking about CFS? -Roger |
|
From: <Co...@Vi...> - 2009-05-09 10:49:27
|
Roger Tsang wrote: > > On Fri, May 8, 2009 at 4:32 PM, CooLCaT@Vienna <co...@co...> wrote: > >> >> >> >> CooLCaT@Vienna wrote: >> > >> > >> > i am trying following apporoach: >> > >> > the root file system resides on a nfs share. every node has a harddisk >> > with only the boot partition on it. we want to achieve, that every >> slave >> > node can become master node so that all nodes must die before cluster >> gets >> > offline. >> > >> > any better ideas somebody? >> > >> > regards >> > >> > cc >> > >> > >> >> ok >> >> after reading this: >> >> *** NOTE: NFS client is not yet supported in the new OpenSSI 2.6 kernel >> *** >> >> i guess i have to do something else: >> >> actually we want that every slave node has the ability to get a master >> node. >> as far as i know drbd can only mirror two nodes. >> >> my approach was, that simply change the root file system to a nfs share >> and >> leave the boot >> partition on the local machine, but it seems that this isnt working as >> expected. > > > You already tried? What problems did you encounter? It is not yet > supported only because we haven't looked at fixing parallel NFS. There is > only one bug report in our Sourceforge tracker and there doesn't seem to > be > much demand for the fix (until recently). > > i always thought that when i use drbd, that if the master node goes down, >> the next potential node mounts the filesystem and takes over. >> >> did i missunderstood? > > > You were talking about CFS? > > -Roger > > ------------------------------------------------------------------------------ > The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your > production scanning environment may not be a perfect world - but thanks to > Kodak, there's a perfect scanner to get the job done! With the NEW KODAK > i700 > Series Scanner you'll get full speed at 300 dpi even with all image > processing features enabled. http://p.sf.net/sfu/kodak-com > _______________________________________________ > Ssic-linux-users mailing list > Ssi...@li... > https://lists.sourceforge.net/lists/listinfo/ssic-linux-users > > hi all! what have i done: i added nfs to /etc/mkinitrd/modules rebuild initrd with: $ modprobe loop $ mkinitrd -o /boot/initrd.img-2.6.10-ssi-686-smp 2.6.10-ssi-686-sm $ mkinitrd -o /boot/initrd.img-2.6.10-ssi-686-smp /lib/modules/2.6.10-ssi-686-smp/ $ mkinitrd -k -o /boot/initrd.img-2.6.10-ssi-686-smp 2.6.10-ssi-686-smp $ ssi-ksync added to grub menu.lst: title Debian SSI mit NFS support root (hd0,0) kernel /vmlinuz-2.6.12-ssi-686-smp root=/dev/nfs rw nfsroot=10.0.0.254:/mnt/cluster/cluster/system ip=10.0.0.1 initrd /initrd.img-2.6.12-ssi-686-smp savedefault then i booted with knoopix and copied the whole /dev/sda5 partition which is regulary mounted on / to the nfs share when rebooting with nfs, the system stops with: mount: special device /initrd/proc does not exist mount: mount point sys does not exist Unmounting /sys /sbin/init: 655: /initrd/sbin/doumount: not found Unmounting /proc /sbin/init: 655: /initrd/sbin/doumount: not found Starting init exec: 655: /initrd/usr/sbin/chroot: not found i also dont see any mount request on the nfs server on the nfs share i changed in fstab 10.0.0.254:/mnt/cluster/cluster/system / nfs rsize=8192,wsize=8192,timeo=14,intr,node=* 0 0 one other problem i ran into when i try to mount the nfs share booted from the local harddisk in fstab is: 10.0.0.254:/mnt/cluster/cluster/system /mnt/nas nfs rsize=8192,wsize=8192,timeo=14,intr,node=1 0 0 node1:~# mount -a mount: wrong fs type, bad option, bad superblock on 10.0.0.254:/mnt/cluster/cluster/system, missing codepage or helper program, or other error (for several filesystems (e.g. nfs, cifs) you might need a /sbin/mount.<type> helper program) In some cases useful info is found in syslog - try dmesg | tail or so dmesg|tail brings nfs warning: mount version newer than kernel NFS: mount program didn't pass remote address! i read a while ago, that someday drbd will support more than two nodes. maybe i will have to wait until that day :( we just simply want, that e.g. /dev/sda2 which is the / filesystem resides on a nfs share. so that we have more than two nodes that can become master node. one other thing, which we havent tried yet. the master node and all slave nodes have two nics. one for external access and one for the cluster. what happens if the main node goes down? is the external interface on the new master then fired up with the same external address or is the cluster then offline for external access? regards cc -- View this message in context: http://www.nabble.com/How-do-I-make-every-node-have-direct-access-to-the-file-system--tp23123954p23459380.html Sent from the ssic-linux-users mailing list archive at Nabble.com. |
|
From: Walker, B. J (HP-Labs) <bru...@hp...> - 2009-05-11 17:23:06
|
Another way of putting it: In native OpenSSI (without integrating with GFS or OCFS or other "cluster" filesystems), all filesystems physically stored on all nodes are transparently visible to all process/users on all nodes (thus the name Single System Image). This is done under the covers using technology called CFS. There is no requirement for each node to be physically attached to each filesystem and in fact CFS is like NFS in that it goes over the interconnect to the single node that is physically attached to the disk with the data. If you have more than one node attached to the disk (shared disk), then CFS can do transparent failover if the node which initially served up the data (mounted the filesystem) fails. In fact I believe it works that you can transparently move the serving of the filesystem from one node to another while it is active. As John mentioned, there have been integratiion efforts with GFS and OCFS to allow direct physical access to the data from all nodes (of course assuming you have direct physical access from all nodes). Not sure if that code currently works. Bruce walker -----Original Message----- From: John Hughes [mailto:jo...@Ca...] Sent: Monday, May 11, 2009 12:50 AM To: CooLCaT@Vienna Cc: ssi...@li... Subject: Re: [SSI-users] How do I make every node have direct access to the file system? Could we go back to the beginning. What exactly are you trying to achieve? "every node has direct access to the filesystem"? Then you don't want NFS, that gives you no more "direct" access than OpenSSI CFS. It sounds like you want GFS, which OpenSSI supports (although I've never tried it). ------------------------------------------------------------------------------ The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com _______________________________________________ Ssic-linux-users mailing list Ssi...@li... https://lists.sourceforge.net/lists/listinfo/ssic-linux-users |
|
From: John H. <jo...@Ca...> - 2009-05-11 07:49:39
|
Could we go back to the beginning. What exactly are you trying to achieve? "every node has direct access to the filesystem"? Then you don't want NFS, that gives you no more "direct" access than OpenSSI CFS. It sounds like you want GFS, which OpenSSI supports (although I've never tried it). |
|
From: John H. <jo...@Ca...> - 2009-05-12 06:48:14
|
Walker, Bruce J (HP-Labs) wrote: > As John mentioned, there have been integratiion efforts with GFS and OCFS to allow direct physical access to the data from all nodes (of course assuming you have direct physical access from all nodes). Not sure if that code currently works. > And of course this is the downside of such a scenario: "assuming you have direct physical access from *all* nodes", which could be tricky if you're using something like parallel SCSI (in practice limited to two nodes as each node takes one of the 16 available SCSI ids, also horribly difficult to get termination right) or expensive if you're using FC (FC HBA's and switch ports cost a small fortune). Maybe iSCSI is the solution. Of course you have to watch out for the classic problem of setting up a redundant system and then connecting it to a non-redundant SAN! |
|
From: <Co...@Vi...> - 2009-05-12 13:18:32
|
hi! thirst of all i must thank you all for your postings. let's begin from the start: i have a running master node with 2 partitions /dev/sda1 mounted as /boot /dev/sda2 mounted as / let's forget about swap partition for now there are howtos for mirroring /dev/sda2 with the help of drbd to another node if the master node goes done then the other takes over, as far as i understood the whole thing. i havent had time yet to play with the drbd solution. but what happends if the second node goes also down? the cluster becomes unavailable. so we thought of using a nfs share instead of /dev/sda2 so the master node boots up and instead of using /dev/sda2 as / filesystem it uses a nfs share. the next node boots from it's hard disk, finds the masternode an continues as usual we tought of using openfiler as nfs server, because openfiler includes also failover to a second openfiler installation. the goal is, that every node can become a failover node, not even only two nodes which are mirrored via drbd. i think due to my bad english, that there were some missunderstandings regarding the access to the filesystem. i hope i cleared it out a little bit now. greets cc -- View this message in context: http://www.nabble.com/How-do-I-make-every-node-have-direct-access-to-the-file-system--tp23123954p23502430.html Sent from the ssic-linux-users mailing list archive at Nabble.com. |
|
From: John H. <jo...@Ca...> - 2009-05-12 13:56:20
|
CooLCaT@Vienna wrote: > let's forget about swap partition for now > (every node has it's own swap) > there are howtos for mirroring /dev/sda2 with the help of drbd to another > node > > if the master node goes done then the other takes over, as far as i > understood the whole thing. > i havent had time yet to play with the drbd solution. > > but what happends if the second node goes also down? > the cluster becomes unavailable. > True. > so we thought of using a nfs share instead of /dev/sda2 > > so the master node boots up and instead of using /dev/sda2 as / filesystem > it uses a > nfs share. the next node boots from it's hard disk, finds the masternode an > continues as usual > > we tought of using openfiler as nfs server, because openfiler includes also > failover to a second > openfiler installation. > And what happens if the second openfiler node goes down? > the goal is, that every node can become a failover node, not even only two > nodes which are mirrored via drbd. > Then you realy need every node to have physical access to the root filesystem. But make sure that you're not just replacing the two init nodes by two fileserver nodes - that doesn't gain you much. I'd think about FC or iSCSI for a setup like this. But how probable is it that both init nodes would go down? > i think due to my bad english, that there were some missunderstandings > regarding the access to the filesystem. i hope i cleared it out a little bit > now. > Your English is very good. |
|
From: Christopher G. S. I. <cg...@ld...> - 2009-05-12 17:15:56
|
----- "CooLCaT@Vienna" <co...@co...> wrote: > we tought of using openfiler as nfs server, because openfiler includes > also > failover to a second > openfiler installation. > > the goal is, that every node can become a failover node, not even only > two > nodes which are mirrored via drbd. It's a nice goal, but the costs far, far outweigh the benefits. If you had the time and experience on hand, even a simple redundant FC setup with two controllers and two switches is in the tens of thousands of dollars. You could go the cheap route with ATAoE, iSCSI, or GNBD and two block servers replicating with drbd in a criss-cross fashion to spread the load (or maybe OpenFiler will be sufficient for you), but it's going to take a while to set up and if you haven't done it before, it may make you go bald. Even though it's a simple redundant configuration, it will cover most failure scenarios. Right now, I think you're overengineering it. If you have concerns about both of your block servers dying at the same time, make them redundant internally and use separate redundant NICs for the storage network. Most of your failures are going to come from software, so get them set up, do all of your benchmarking, tuning, and testing, finalize the configuration, and then leave them alone! Don't network them except on the private SAN and a secure, isolated administrative network. That will give you some a little bit of wiggle room to test your security patches before applying them to your live servers. Use GFS for the shared root filesystem because it's the most mature, the most configurable, and the most like a natural POSIX compliant filesystem, even though it can be a bitch at first. For other shared filesystems in small clusters, OCFS2 will take care of most needs and it's really easy to set up. -- Christopher G. Stach II |
|
From: Robert W. <no...@ro...> - 2009-05-14 13:04:43
|
On Tue, May 12, 2009 at 06:18:25AM -0700, CooLCaT@Vienna wrote: > > hi! > > thirst of all i must thank you all for your postings. > > let's begin from the start: > > i have a running master node with 2 partitions > > /dev/sda1 mounted as /boot > /dev/sda2 mounted as / > > let's forget about swap partition for now > > there are howtos for mirroring /dev/sda2 with the help of drbd to another > node Right. > if the master node goes done then the other takes over, as far as i > understood the whole thing. > i havent had time yet to play with the drbd solution. Thats the scope of drbd. > but what happends if the second node goes also down? > the cluster becomes unavailable. Right. > so we thought of using a nfs share instead of /dev/sda2 > > so the master node boots up and instead of using /dev/sda2 as / filesystem > it uses a > nfs share. the next node boots from it's hard disk, finds the masternode an > continues as usual > > we tought of using openfiler as nfs server, because openfiler includes also > failover to a second > openfiler installation. Hm. Lets see: you do not trust your 2 openSSI-master nodes, so you want a solution, where every node of your cluster could take over the master role, and you rely therefore on a 2-node-fileserver for the root file system? Is your openfiler file server so much more reliable than your 2 openSSI master nodes? Maybe it could be cheeper to invest the money for the openfiler-system into some more reliable HW for your OpenSSI-nodes??? What are your requirements (in uptime, costs, etc) for your system? What is the probability for a failure of your 2 openSSi master nodes _at_the_same_time_? (Because I guess, if one server fails, then you will nearly immediately repair or substitute your failed server ...) > the goal is, that every node can become a failover node, not even only two > nodes which are mirrored via drbd. Relying on a (only) two node file server ;-)) Regards, Robert (PS: I am not really an expert, but if you like, then we could discuss this stuff also privately in german ...) |
