om-2-6-devel Mailing List for Simple Openmosix 2.6
Status: Beta
Brought to you by:
weswagner
You can subscribe to this list here.
2007 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
(6) |
Aug
(7) |
Sep
(1) |
Oct
|
Nov
(2) |
Dec
|
---|
From: Wes W. <wes...@gm...> - 2007-11-28 17:06:37
|
Oooooh very artful poke to encourage me to get it done. Your open source judo is strong, and I am no match. -Wes On Nov 28, 2007 9:00 AM, Niels de Vos <nix...@us...> wrote: > Just posting my old implementation of the user-space-tool (mentioned > on <http://howto.krisbuytaert.be/openMosixWiki/index.php/UserLandTools2.6 > >) > to this list. Wes wanted to put it in SVN, but probably forgot about > it. With this post the code gets archived and doesn't get lost :) > > Cheers, > Niels > > ---------- Forwarded message ---------- > From: Niels de Vos <ni...@ni...> > Date: Aug 2, 2007 2:59 PM > Subject: Re: openMosix migration daemon > To: Wes Wagner <wes...@gm...> > > > There you are. You're welcome to change the name, so the users can see > there is a difference between my these tools and omuscd. > > If you provide the sources in SVN too (I would prefer that), please > add me as a developer (nix...@us...). > > Thanks > > On 8/2/07, Wes Wagner <wes...@gm...> wrote: > > Niels, > > > > If you send me a tarball of your latest, we can just consider that > version 1 > > and dump it into the 2.6.22 working directory (which may be spawned off > into > > alpha/beta this weekend) - most 2.4 users are accustomed to having the > > userland tools - it would be good to distribute your python code with > the > > distro. > > > > -Wes > > > > > > On 8/2/07, Niels de Vos <ni...@ni...> wrote: > > > Hi Wes, > > > > > > it's more or less working, alpha/beta sounds good... My system crashed > > most often due to oM bugs. Moshe tried them and wrote it works for him > :) > > See this thread on openmosix-devel < > > > http://sourceforge.net/mailarchive/message.php?msg_id=1154867432.8578.9.camel%40localhost.localdomain > > > > for some details. > > > > > > I would happily give these tools to "simple om-2.6" so others can > > use/improve them. It's almost one year ago I did something with the > source > > (except for adding TODO's). But I really do hope I can wor on that soon > > again... no promises here though. Most important thing missing is a > simple > > installer. Now it's just check out the source and run it manually. > > > > > > My server is at some site where the firewall is not cooperating very > well. > > Therefore the SVN is mostly unreachable. However I can get you a copy of > the > > latest version, or maybe even an svnadmin-dump if you like. > > > > > > Let's keep in touch, > > > Niels > > > > > > > > > > > > On 8/2/07, Wes Wagner < wes...@gm...> wrote: > > > > > > http://howto.x-tend.be/openMosixWiki/index.php/UserLandTools2.6 > > > > > > > > I knew I knew Niels' name from somewhere :) What state are the > userland > > tools in? ... what type of % complete do you have things before you > think it > > would be late alpha/early beta? > > > > > > > > -Wes > > > > > > > > > > > > > > > > On 8/1/07, Wes Wagner < wes...@gm...> wrote: > > > > > > > > > Well it is entirely up to you... if I put source out there, people > are > > going to try to hack away at it and fix things. If you want that type of > > help (and the headache that comes with it) then I would do it that way- > > otherwise I would only put in "releases" that we can certify work with a > > particular branch that are prepackaged and setup to build (or in the > case of > > our rpm releases, prebuilt and just install) > > > > > > > > > > -Wes > > > > > > > > > > > > > > > > > > > > On 8/1/07, g4sra < g4...@ya...> wrote: > > > > > > > > > > > Wes Wagner wrote: > > > > > > > Would you like me to host the omuscd source in my svn? (it is > sort > > of > > > > > > > a first step towards automated install...) though I have a > newer > > copy > > > > > > > than what you have in sourceforge, I don't know that what I > have > > is > > > > > > > your best. > > > > > > Entirely up to you Wes, I suppose some people do prefer a > browsable > > svn > > > > > > to a tgz, or it wouldn't exist. It may make your "install" > project a > > > > > > little more coherent (and robust, no broken links :>). The only > > downside > > > > > > that I can think of is that the omuscd project statistics will > not > > be > > > > > > a true refection, for that reason I would prefer that it is only > > > > > > available as part of your "install" and not an individual > > "downloadable" > > > > > > (just hyperlink across if you want to offer that). I don't make > > releases > > > > > > that often, so you should not have to update the svn > ridiculously > > often. > > > > > > I would recommend that you use the officially published versions > > from > > > > > > the omuscd project site though. I will not support any other > > versions > > > > > > (from anywhere else without payment, hence the use of the > embedded > > > > > > serial number & signatures), what is on "master" is a messed up > > "find > > > > > > the segfault" hacked version, it might be timestamped later in > date > > but > > > > > > it is worse code not intended for publication. The latest is sat > on > > my > > > > > > hard drive and will be released when the port to x86_64 works. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Wes Wagner > > > > > > > > > > > > > > > > -- > > > > Wes Wagner > > > > > > > > > > > > > > -- > > Wes Wagner > > ------------------------------------------------------------------------- > SF.Net email is sponsored by: The Future of Linux Business White Paper > from Novell. From the desktop to the data center, Linux is going > mainstream. Let it simplify your IT future. > http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 > _______________________________________________ > Om-2-6-devel mailing list > Om-...@li... > https://lists.sourceforge.net/lists/listinfo/om-2-6-devel > > -- Wes Wagner |
From: Niels de V. <nix...@us...> - 2007-11-28 17:00:06
|
Just posting my old implementation of the user-space-tool (mentioned on <http://howto.krisbuytaert.be/openMosixWiki/index.php/UserLandTools2.6>) to this list. Wes wanted to put it in SVN, but probably forgot about it. With this post the code gets archived and doesn't get lost :) Cheers, Niels ---------- Forwarded message ---------- From: Niels de Vos <ni...@ni...> Date: Aug 2, 2007 2:59 PM Subject: Re: openMosix migration daemon To: Wes Wagner <wes...@gm...> There you are. You're welcome to change the name, so the users can see there is a difference between my these tools and omuscd. If you provide the sources in SVN too (I would prefer that), please add me as a developer (nix...@us...). Thanks On 8/2/07, Wes Wagner <wes...@gm...> wrote: > Niels, > > If you send me a tarball of your latest, we can just consider that version 1 > and dump it into the 2.6.22 working directory (which may be spawned off into > alpha/beta this weekend) - most 2.4 users are accustomed to having the > userland tools - it would be good to distribute your python code with the > distro. > > -Wes > > > On 8/2/07, Niels de Vos <ni...@ni...> wrote: > > Hi Wes, > > > > it's more or less working, alpha/beta sounds good... My system crashed > most often due to oM bugs. Moshe tried them and wrote it works for him :) > See this thread on openmosix-devel < > http://sourceforge.net/mailarchive/message.php?msg_id=1154867432.8578.9.camel%40localhost.localdomain> > for some details. > > > > I would happily give these tools to "simple om-2.6" so others can > use/improve them. It's almost one year ago I did something with the source > (except for adding TODO's). But I really do hope I can wor on that soon > again... no promises here though. Most important thing missing is a simple > installer. Now it's just check out the source and run it manually. > > > > My server is at some site where the firewall is not cooperating very well. > Therefore the SVN is mostly unreachable. However I can get you a copy of the > latest version, or maybe even an svnadmin-dump if you like. > > > > Let's keep in touch, > > Niels > > > > > > > > On 8/2/07, Wes Wagner < wes...@gm...> wrote: > > > > http://howto.x-tend.be/openMosixWiki/index.php/UserLandTools2.6 > > > > > > I knew I knew Niels' name from somewhere :) What state are the userland > tools in? ... what type of % complete do you have things before you think it > would be late alpha/early beta? > > > > > > -Wes > > > > > > > > > > > > On 8/1/07, Wes Wagner < wes...@gm...> wrote: > > > > > > > Well it is entirely up to you... if I put source out there, people are > going to try to hack away at it and fix things. If you want that type of > help (and the headache that comes with it) then I would do it that way- > otherwise I would only put in "releases" that we can certify work with a > particular branch that are prepackaged and setup to build (or in the case of > our rpm releases, prebuilt and just install) > > > > > > > > -Wes > > > > > > > > > > > > > > > > On 8/1/07, g4sra < g4...@ya...> wrote: > > > > > > > > > Wes Wagner wrote: > > > > > > Would you like me to host the omuscd source in my svn? (it is sort > of > > > > > > a first step towards automated install...) though I have a newer > copy > > > > > > than what you have in sourceforge, I don't know that what I have > is > > > > > > your best. > > > > > Entirely up to you Wes, I suppose some people do prefer a browsable > svn > > > > > to a tgz, or it wouldn't exist. It may make your "install" project a > > > > > little more coherent (and robust, no broken links :>). The only > downside > > > > > that I can think of is that the omuscd project statistics will not > be > > > > > a true refection, for that reason I would prefer that it is only > > > > > available as part of your "install" and not an individual > "downloadable" > > > > > (just hyperlink across if you want to offer that). I don't make > releases > > > > > that often, so you should not have to update the svn ridiculously > often. > > > > > I would recommend that you use the officially published versions > from > > > > > the omuscd project site though. I will not support any other > versions > > > > > (from anywhere else without payment, hence the use of the embedded > > > > > serial number & signatures), what is on "master" is a messed up > "find > > > > > the segfault" hacked version, it might be timestamped later in date > but > > > > > it is worse code not intended for publication. The latest is sat on > my > > > > > hard drive and will be released when the port to x86_64 works. > > > > > > > > > > > > > > > > > > > > > -- > > > > Wes Wagner > > > > > > > > > > > > -- > > > Wes Wagner > > > > > > > > -- > Wes Wagner |
From: Alejandro M. <co...@ma...> - 2007-09-23 21:11:53
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, my name is Alejandro Mansilla i am from Mendoza, Argentina. (excuse me for my very BAD english :) ) I write this because im investigating about high performance computing for my degree title at Unversity. My idea is to work about OpenMosix, describing the technologies envolved and showing it working. I need to know about the internals of the OM, what parts of the kernel are patched and why, there is some kind of documentation that i can read? o some few words from the developers about this. i appreciate any help. thanks and keep working. - -- Alejandro -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (GNU/Linux) iD8DBQFG9tbxTYGaXIe2LbkRAsiqAJ4m5sW1IkIBpCxnPZ+U9255GRpLRgCdEoJi ZqdZMkpRXDzF9NdiN/5psg8= =Cody -----END PGP SIGNATURE----- |
From: Wes W. <wes...@gm...> - 2007-08-13 20:32:39
|
Julia, That would be great if you could host it and potentially someone else could pair it up by donating another box. Would you be at all inclined in managing it from the standpoint of checking it out to openmosix developers for a few days at a time so they can use it without trampling on each other? An 8 way box like that would extremely useful to anyone working on validating and making sure the OM code is SMP safe. -Wes On 8/13/07, ri...@vo... <ri...@vo...> wrote: > > On Mon, Aug 13, 2007 at 12:28:41PM -0700, Wes Wagner wrote: > > I have lab space (HVAC + 240V power) that is SSH accessable in my home > but > > that would be a beast... I don't know that we need quite that much > > horsepower for testing. My lab only has 30 amps of 240 so i try to keep > most > > my machine below a half amp of power :) > > > > Anyone have any ideas? > > > > -Wes > > > > > > On 8/13/07, br...@ca... <br...@ca...> wrote: > > > > > > > > > I have a Dell 8450 8 processor x86 system that I would be happy to > > > donate. Unfortunately, it's currently in the United States. It's > very > > > heavy and would cost a small fortune to ship. I'm open to > suggestions... > > > > > > > > > > > > Bruce Kissinger > > > > > > > > > > > > > > > > > > On Fri, 10 Aug 2007 10:10:49 -0700, "Wes Wagner" <wes...@gm... > > > > > wrote: > > > > > > > The openmosix 2.6 project is getting new steam. There have been 2 > $100 > > > > > > > pledges to distribute to the coder(s) (right now it is really just > > > Florian > > > > > > > for the real kernel work) when the successful alpha comes out. Other > > > items > > > > > > > that could help the project is amd64 and i386 test equipment, (procs > and > > > > > > > mbs), for Florian to use in testing. If you have some beat up old > > > > > > > equipment > > > > > > > that works but you don't need and live in Europe so you can ship it > > > cheap, > > > > > > > please consider donating it to the project. > > > > > > > > > > > > > > By the end of the weekend we should have everything together enough > that > > > > > > > we > > > > > > > can start accepting work from other developers. We have been working > on > > > > > > > the > > > > > > > infrastructure needed to support collaboration without anyone > stepping > > > on > > > > > > > each other's toes. > > > > > > > > > > > > > > We are also looking to rename the product/project - so name > suggestions > > > > > > > are > > > > > > > welcome. I have already had some good ideas proposed but we can use > some > > > > > > > more. Moshe suggested rebranding and Florian and I agree entirely > that a > > > > > > > new > > > > > > > name will convey that 2.6 is different than the project objectives > of > > > 2.4, > > > > > > > and has been significantly rethought in light of the changes in > market > > > > > > > needs > > > > > > > and hardware architectures. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > Wes Wagner > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > openMosix-devel mailing list > > ope...@li... > > https://lists.sourceforge.net/lists/listinfo/openmosix-devel > > some quick googling shows the dell 8450 is a 7U machine. not bad. > > where in the US is it located? > > i can offer some space in a financial institution's rack. however, i'm > located in arkansas. > > Julia Longtin <ri...@vo...> > -- Wes Wagner |
From: <ri...@vo...> - 2007-08-13 20:28:16
|
On Mon, Aug 13, 2007 at 12:28:41PM -0700, Wes Wagner wrote: > I have lab space (HVAC + 240V power) that is SSH accessable in my home but > that would be a beast... I don't know that we need quite that much > horsepower for testing. My lab only has 30 amps of 240 so i try to keep most > my machine below a half amp of power :) > > Anyone have any ideas? > > -Wes > > > On 8/13/07, br...@ca... <br...@ca...> wrote: > > > > > > I have a Dell 8450 8 processor x86 system that I would be happy to > > donate. Unfortunately, it's currently in the United States. It's very > > heavy and would cost a small fortune to ship. I'm open to suggestions... > > > > > > > > Bruce Kissinger > > > > > > > > > > > > On Fri, 10 Aug 2007 10:10:49 -0700, "Wes Wagner" <wes...@gm...> > > wrote: > > > > > The openmosix 2.6 project is getting new steam. There have been 2 $100 > > > > > pledges to distribute to the coder(s) (right now it is really just > > Florian > > > > > for the real kernel work) when the successful alpha comes out. Other > > items > > > > > that could help the project is amd64 and i386 test equipment, (procs and > > > > > mbs), for Florian to use in testing. If you have some beat up old > > > > > equipment > > > > > that works but you don't need and live in Europe so you can ship it > > cheap, > > > > > please consider donating it to the project. > > > > > > > > > > By the end of the weekend we should have everything together enough that > > > > > we > > > > > can start accepting work from other developers. We have been working on > > > > > the > > > > > infrastructure needed to support collaboration without anyone stepping > > on > > > > > each other's toes. > > > > > > > > > > We are also looking to rename the product/project - so name suggestions > > > > > are > > > > > welcome. I have already had some good ideas proposed but we can use some > > > > > more. Moshe suggested rebranding and Florian and I agree entirely that a > > > > > new > > > > > name will convey that 2.6 is different than the project objectives of > > 2.4, > > > > > and has been significantly rethought in light of the changes in market > > > > > needs > > > > > and hardware architectures. > > > > > > > > > > > > > > > > > -- > Wes Wagner > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > openMosix-devel mailing list > ope...@li... > https://lists.sourceforge.net/lists/listinfo/openmosix-devel some quick googling shows the dell 8450 is a 7U machine. not bad. where in the US is it located? i can offer some space in a financial institution's rack. however, i'm located in arkansas. Julia Longtin <ri...@vo...> |
From: Wes W. <wes...@gm...> - 2007-08-13 19:28:42
|
I have lab space (HVAC + 240V power) that is SSH accessable in my home but that would be a beast... I don't know that we need quite that much horsepower for testing. My lab only has 30 amps of 240 so i try to keep most my machine below a half amp of power :) Anyone have any ideas? -Wes On 8/13/07, br...@ca... <br...@ca...> wrote: > > > I have a Dell 8450 8 processor x86 system that I would be happy to > donate. Unfortunately, it's currently in the United States. It's very > heavy and would cost a small fortune to ship. I'm open to suggestions... > > > > Bruce Kissinger > > > > > > On Fri, 10 Aug 2007 10:10:49 -0700, "Wes Wagner" <wes...@gm...> > wrote: > > > The openmosix 2.6 project is getting new steam. There have been 2 $100 > > > pledges to distribute to the coder(s) (right now it is really just > Florian > > > for the real kernel work) when the successful alpha comes out. Other > items > > > that could help the project is amd64 and i386 test equipment, (procs and > > > mbs), for Florian to use in testing. If you have some beat up old > > > equipment > > > that works but you don't need and live in Europe so you can ship it > cheap, > > > please consider donating it to the project. > > > > > > By the end of the weekend we should have everything together enough that > > > we > > > can start accepting work from other developers. We have been working on > > > the > > > infrastructure needed to support collaboration without anyone stepping > on > > > each other's toes. > > > > > > We are also looking to rename the product/project - so name suggestions > > > are > > > welcome. I have already had some good ideas proposed but we can use some > > > more. Moshe suggested rebranding and Florian and I agree entirely that a > > > new > > > name will convey that 2.6 is different than the project objectives of > 2.4, > > > and has been significantly rethought in light of the changes in market > > > needs > > > and hardware architectures. > > > > > > > > -- Wes Wagner |
From: <ta...@sn...> - 2007-08-13 10:46:37
|
On Fri, Aug 10, 2007 at 10:10:49AM -0700, Wes Wagner wrote: > We are also looking to rename the product/project - so name suggestions are > welcome. I have already had some good ideas proposed but we can use some > more. Moshe suggested rebranding and Florian and I agree entirely that a new > name will convey that 2.6 is different than the project objectives of 2.4, That is a good idea, and not only that, but I think a new name/rebranding/separating from here/.. is totally *mandatory* for creating a new wave of interested/curious people. Cheers, -- Vincent Hanquez |
From: Jonathan D. <im...@ya...> - 2007-08-10 17:51:08
|
In light of, ummmm, some of the more lively? responses to my posts from earlier, it would be good if we could work on agreeing a summary of objectives for the rebranded project. I'm assuming the shift in user needs will mean that we add functionality previously considered unnecessary for a load-balancing system specific to CPU loads. However, I don't want to get into religious wars over what should or should not be part of the rebranded work. I'd much prefer an agreed-upon list of what users need that we can provide. I know for a fact that HPC users are positively screaming for better network support. Cascading failures and network meltdowns are big issues. I also know for a fact that the Data Center market is increasingly unhappy with solutions that have any single point of failure whatsoever. But are these the users we are going to be concerned with? I'll see what non-ix86 hardware I can find, to see how easy it would be to whip up an experimental (ie: not in the mainstream OM) port to other platforms, in addition to whatever projects I already have simmering that are candidates for putting into the extra projects. Since the absolute core code wants to be as lightweight as possible - for maximum performance, minimum bugs, etc, it makes little sense to merge anything not absolutely critical or universal into the core code, allowing for inevitable special cases and exceptions. Since the core shouldn't care one whit about how data actually arrives or departs, only that it does so in the times and places expected, I honestly can't think of any projects of mine - however controversial - that would have any need or business being in core. Jonathan --- Wes Wagner <wes...@gm...> wrote: > The openmosix 2.6 project is getting new steam. > There have been 2 $100 > pledges to distribute to the coder(s) (right now it > is really just Florian > for the real kernel work) when the successful alpha > comes out. Other items > that could help the project is amd64 and i386 test > equipment, (procs and > mbs), for Florian to use in testing. If you have > some beat up old equipment > that works but you don't need and live in Europe so > you can ship it cheap, > please consider donating it to the project. > > By the end of the weekend we should have everything > together enough that we > can start accepting work from other developers. We > have been working on the > infrastructure needed to support collaboration > without anyone stepping on > each other's toes. > > We are also looking to rename the product/project - > so name suggestions are > welcome. I have already had some good ideas proposed > but we can use some > more. Moshe suggested rebranding and Florian and I > agree entirely that a new > name will convey that 2.6 is different than the > project objectives of 2.4, > and has been significantly rethought in light of the > changes in market needs > and hardware architectures. > > -- > Wes Wagner > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? > Stop. > Now Search log events and configuration files using > AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/> _______________________________________________ > Om-2-6-users mailing list > Om-...@li... > https://lists.sourceforge.net/lists/listinfo/om-2-6-users > ____________________________________________________________________________________ Take the Internet to Go: Yahoo!Go puts the Internet in your pocket: mail, news, photos & more. http://mobile.yahoo.com/go?refer=1GNXIC |
From: Wes W. <wes...@gm...> - 2007-08-10 17:12:08
|
The openmosix 2.6 project is getting new steam. There have been 2 $100 pledges to distribute to the coder(s) (right now it is really just Florian for the real kernel work) when the successful alpha comes out. Other items that could help the project is amd64 and i386 test equipment, (procs and mbs), for Florian to use in testing. If you have some beat up old equipment that works but you don't need and live in Europe so you can ship it cheap, please consider donating it to the project. By the end of the weekend we should have everything together enough that we can start accepting work from other developers. We have been working on the infrastructure needed to support collaboration without anyone stepping on each other's toes. We are also looking to rename the product/project - so name suggestions are welcome. I have already had some good ideas proposed but we can use some more. Moshe suggested rebranding and Florian and I agree entirely that a new name will convey that 2.6 is different than the project objectives of 2.4, and has been significantly rethought in light of the changes in market needs and hardware architectures. -- Wes Wagner |
From: Wes W. <wes...@gm...> - 2007-08-07 20:31:37
|
Just so everyone generally knows, work has been continuing on Openmosix 2.6and Florian has been a great help. Somehow I convinced him to go along with my crazy idea of releasing 2.6.22 as the first major release instead of 2.6.17. We are concentrating right now on making sure that the development environment makes it easy for people to collaborate and submit their work. I am in the process of integrating kgdb into the 2.6.22 line and Florian has been working on the development environment and wants to get to fixing the syscall layer as soon as he can. Our goal for the first 2.6.22 release is to have all basic functionality working (processes can migrate and make all syscalls) for the i386 and x86_64 architectures. If you are interested in continuing to do kernel development work towards this project, please let me know. So far it seems the only active developer is Florian so if someone else out there is or has stepped up to the plate I need to know so I can get you access to the current source repository. -- Wes Wagner |
From: Wes W. <wes...@gm...> - 2007-07-31 14:22:27
|
I think I would have more confidence in the micropledge team after they get the official nod from sourceforge. I expect to be making updates to the SVN before this weekend with scripts that make it quick and easy to download your kernel from kernel.org (or anywhere else) and apply the patches. That is really the only missing component at the moment. g_remlin was less than kind with the current 2.6.17 source you had in repository to get it working. My understanding is that he stripped out alot and the stripping will continue. Right now base functionality (successful process migration) is working and I have not seen a process migration fault in weeks. Omuscd (his cluster daemon) still needs some work. Our goal is to put all core openmosix in a /core directory and move out any special features or addons into extensions directories. By dong this it will be easier to keep the core product stable but allow new features to go through a standard development and release cycle: development, testing, alpha, beta, release. Also that means that individual developers will have more control over their add-ons, but the core product can move on even if they are not ready to. Anyway, that is the way I have been managing the simple openmosix project. We also dumped support for all the architectures except i386 and x86_64. Non-common platforms should be managed as add-ons so they do not hold up availability of core. By doing this we actually moved through kernel version fairly rapidly. We went from your base 2.6.17 to 2.6.19 2.6.21 and 2.6.22with relative ease. I suspect that if we had sufficient testors we could release openmosix on any subkernel version within 1-2 weeks of the kernels GA. Probably closer to 1 so long as we are tracking development and not waiting for official GA. I personally use gentoo which takes a vanilla kernel really well. It might be helpful if you knew anyone who had particular interest in making redhat/debian/etc packages so we can release consumer friendly distributions for people who are not as technically savvy :) Our end goal is to make clustering so simple that even a caveman could do it. If you have google chat, you can find me online most of the time at wes...@gm... - I certainly would like to talk about all of this :) -Wes On 7/31/07, Florian Delizy <flo...@un...> wrote: > > what do you think of this one ?? I am not sure this is not real spam > ... but that may worth something afterall > > BTW, tell me when you update the svn, I'll grab a copy very soon, and > start coding :) (Yeah man, I am back, for real this time :) ) > > > Hi Florian, > > I was looking at your image-clustering Linux kernel extension on > SourceForge.net, and I > thought you could benefit from a collaborative web funding site we've just > released. We're called microPledge, and we're looking for useful and > interesting projects to fill our site before we start advertising to the > general public - within a week if we get enough projects on site. > > You can find us at http://micropledge.com/ > > We help people pledge small amounts (or large amounts!) of money to you > and > others. For much open-source software, people are reluctant to give money, > because they're afraid nothing will come of it. When they pledge money on > microPledge, however, their money is held in trust until they see the > developer's made real progress on the software. They can't pull their > money > out unless the project fails, so developers can be confident they will get > the > money. > > This enables trust on both sides, and it means people are more ready to > support your software. > > We provide a free service for open-source software. To put your project on > microPledge, visit http://micropledge.com/dream and enter your project > details. Make sure you select "I'm the sole developer" (unless you want to > > open up the project for quotes from other developers). > > Some existing projects, perhaps like yours, may prefer our donations-only > scheme. This might work best if you're just bug-fixing and can't aim for > any > feature milestone. It is different from pledges, because the money goes > directly to you instead of being held in trust until you make progress > releases. I'm sure you have other sources of donations, but another might > help. To create a donations-only project, simply visit > http://micropledge.com/dream, create a project and select the > 'donations-only' > option. > > We're working towards partnering with SourceForge.net so that they will > allow > you to place microPledge links on your SourceForge.net project page, > providing > you with easier-to-find sources of revenue. > > The more people who pledge to open-source software, the more pledging will > become a way to provide open-source developers with a stable source of > funds. > You can help us, and help the Open Source movement, by letting others know > about us. > > By all means, let us know how you find our service. We appreciate your > time. > > Bryan, > for the microPledge team. > > -- Wes Wagner |
From: Wes W. <wes...@gm...> - 2007-07-27 01:19:40
|
The 2.6.22 kernel patches for simple open mosix are now available. By the end of the weekend the SVN should also contain the scripts you need to make it easier, but if you are very eager, the source is there now. https://sourceforge.net/projects/om-2-6 The simple openmosix project only supports i386 and x86_64 at this time, and you will want to download and use omuscd as well as a simple cluster daemon. https://sourceforge.net/projects/omuscd -- Wes Wagner -- Wes Wagner |
From: Wes W. <wes...@gm...> - 2007-07-25 23:04:20
|
On 7/25/07, Jonathan Day <im...@ya...> wrote: > > For the purpose of this discussion, I will exclude all > reference to shared memory or the sharing of resources > in a time-sensitive manner. I will also purposely > exclude all reference to sequential coherence. > > One of the popular objections to clusters is you need > special software, using special-purpose communications > methods. This is bogus in the abstract, and only true > if the implementation doesn't support any IPC. > > I will start by describing simple message-passing. > This is the easiest and simply requires that the > message be intercepted, relayed to the remote machine > with the target process, and then re-sent. If the > cluster uses UPIDs, this isn't too bad, as you'll know > if the UPID exists on your node and can always look up > what node the UPID is on if you've not already cached > it. > > For message-passing (and other IPC) that can be > relayed over tipc, you simply use that. Nothing > further is needed. So, the above message-passing > mechanism is only useful for message-passing that > isn't trapped by tipc already. > > Semaphores - and, indeed, any other IPC that requires > only the sending of a single byte or bit - cannot be > done efficiently. The overheads on even the simplest > packet massively exceed the size of data. Semaphores > are also used a lot for synchronization, and networks > have latencies in the order of micoseconds > (Dolphinics, Infiniband), milliseconds (Ethernet) or > longer (IP-over-Avian). Way too slow for a lot of > things. > > However, synchronization over a cluster is always a > pain and unless you can now buy tachyon-based home > networking gear at Best Buy, anything written for a > cluster will be subject to the same limitations, and > MPI barrier operations are going to be a LOT slower > than just delivering a trivial packet over the > network. > > That is sufficient for basic IPC between processes to > occur between nodes on a fairly transparent basis. > When it's too slow, just migrate the processes > together on the same node. Problem solved. > > What about other mechanisms? > > Well, socket connections will be fun, as you will need > to reflect then redirect the connection to not only > the correct node but quite possibly the correct port > on that node - unless port assignments are > cluster-wide. "Localhost" is the worst case, because > you have to go to where the user-side stub is, check > for port mappings, find out the user-side stub > attached to that port, find the machine it is now on, > get the process stub to identify the port, then check > mappings to see what the actual port is. And then make > the connection. > > RPC is about the same, only you'd have to do the RPC > mappings as well. > > CORBA (yes, people use it, mostly for avionics from > what I understand) is totally evil. When a process > migrates, it also has to handle all of the ORB > communication needed to resituate itself in addition > to migrating all of the connections. > > In the cases of RPC and CORBA, you'd need a dedicated > extension for each to support the necessary activity. > > And what do you get, if all of this is done? Well, you > can't cluster shmem operations, but you can cluster > nearly everything else in use without the need of a > solution hard-coded into the user apps. > > What about other possibilities? > > It would be good to have extensions that would allow > brave adventurers in GCC-land to support an OpenMosix > target for a subset of OpenMP. Then even when you did > want hard-coded solutions, the compiler can give you > native ones. > > How central are these? > > I would regard none of them as inherently CORE, but > I'd probably have TIPC and other really primitive > operations plugged as extensions directly on top of > CORE. Everything else listed here I'd build as > extensions on top of those IPC primitives. > > > > > > ____________________________________________________________________________________ > Boardwalk for $500? In 2007? Ha! Play Monopoly Here and Now (it's updated > for today's economy) at Yahoo! Games. > http://get.games.yahoo.com/proddesc?gamekey=monopolyherenow > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > openMosix-devel mailing list > ope...@li... > https://lists.sourceforge.net/lists/listinfo/openmosix-devel > -- Wes Wagner Join a libertarian network of person-to-person lending on Prosper: http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group |
From: Wes W. <wes...@gm...> - 2007-07-25 23:04:07
|
On 7/25/07, Jonathan Day <im...@ya...> wrote: > > Here is my proposal for the CORE segment of the code. > The CORE segment is going to be the relatively stable > component of the code. Code in CORE would exist in two > states: viscous and locked-down. > > Viscous code would be code that is still changing - > it's still partly fluid - but changes relatively > slowly and only as needed. Glue layers between the > kernel and the CORE would be permanently viscous, as > the kernel's internal API is designed to change. The > initial drop into code would be entirely viscous. > > Locked-down code would be code where there is > sufficient confidence in it being complete and > sufficient for the current kernel AND in it being as > stable as can reasonably be achieved. Code should not > be marked as locked-down unless there's high > confidence in it. By implication, modifying > locked-down code should be avoided if possible, and on > modification it reverts to being viscous. > > The general concept is that the "real work" is largely > independent of implementation specifics of how the > data is passed in or out, that the CORE should only do > the generic, basic "real work" anyway, and that > everything else (ie: all the heavy lifting, the fancy > methods, experimental projects and the other code > that's going to be tough to write) can be attached > either directly to the viscous interface or to another > project if you need further abstraction. > > At this point, I simply do not know what would be > needed in the CORE component. Greater minds than mine, > however, exist on this list and can undoubtedly give a > clearer answer to this. > > I imagine that fundamental architecture-specific code > would reside in there - at least for architectures > that are actively-enough maintained that the code > could be viscous or locked-down. I imagine > universally-required helper functions would be there > too, but I'll leave that to the true kernel gurus here > to rule upon. > > Elsewhere, I've talked about reliable multicast and > other such methods. Those would very definitely be in > the realm of extensions. They would shape how a given > OpenMosix system would work, but would not be > fundamental to getting it to work at all. You don't > need the truly fancy stuff to get a basic system to > run. > > > > > > ____________________________________________________________________________________ > Be a better Globetrotter. Get better travel answers from someone who > knows. Yahoo! Answers - Check it out. > http://answers.yahoo.com/dir/?link=list&sid=396545469 > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > openMosix-devel mailing list > ope...@li... > https://lists.sourceforge.net/lists/listinfo/openmosix-devel > -- Wes Wagner Join a libertarian network of person-to-person lending on Prosper: http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group |
From: Wes W. <wes...@gm...> - 2007-07-25 23:03:16
|
On 7/25/07, Jonathan Day <im...@ya...> wrote: > > This is a very short (for me) writeup on a proposal > for accelerating process migration for SIMD-style > applications. It will NOT be useful for MIMD or > general-purpose clustering. > > The way SIMD generally operates is to have an > identical process migrated to, or started on, a very > large number of nodes simultaneously. MPI is designed > to work this way, it is generally not used either > RPC-style or socket-style to connect to any arbitrary > piece of code with the right interface. > > That being the case, it occurred to me that it makes > no sense whatsoever to start N processes on the master > node and then farm them out sequentially. They're all > the same, so starting one is no different from > starting a thousand - except that it'll be faster and > less memory-intensive. > > At the meeting, I proposed the following for the > migration mechanism. Start up a reference copy and > farm that out as many times as you need to all the > recipients. BUT, if you use NORM or FLUTE for the file > transfer mechanism, you only actually transmit one > copy - all intended recipients would listen for and > upload that copy into memory, assign the universal IDs > at that point, and then signal the originator of what > those IDs are, along with the usual state info. > (Normally, the originator would assign the IDs, but > it's better if the recipients do in the case of a > multicast distribution.) > > The originator then creates the necessary stubs and > populates them with the necessary information. > > Why is this a useful method? > > It's not significantly more complex, as > implementations of NORM and FLUTE already exist. We > don't have to write those, we don't have to care about > that part of the operation at all. In fact, in some > ways it is simpler, as you don't have to juggle very > large numbers of processes or flood the network with > traffic. > > However, it should be faster. Clusters with a large > number of nodes, or running very large SIMD programs, > would have far shorter startup times. It should also > be far more stable for very large clusters, where the > number of user processes for migration become large > enough to disrupt scheduling or resource management. > > This is only at the start, though. Does it improve > performance after that? > > Well, maybe. It depends on how we handle the stubs. > Each process must have its own set of state > information, but that can be done just as easily in a > list. Certainly, since one physical device can only be > doing one thing, we only need one local I/O thread. > Since such systems generally operate on the idea of > scatter/gather, you probably only need one > communications thread that can switch between targets. > > This does improve performance and it also improves > scalability as your limit is in kernel memory and not > kernel thread table space. > > Alternatively, you could slave all the stubs to the > master stub for that SIMD process. In that case, you > minimize changes but still get most of the performance > increases - both at the start and when running. > > As with all of these things, the possibilities are > endless. The challenge will be to find a possibility > that looks like it'll do what we want and home in on > it. > > > > > > ____________________________________________________________________________________ > Choose the right car based on your needs. Check out Yahoo! Autos new Car > Finder tool. > http://autos.yahoo.com/carfinder/ > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > openMosix-devel mailing list > ope...@li... > https://lists.sourceforge.net/lists/listinfo/openmosix-devel > -- Wes Wagner Join a libertarian network of person-to-person lending on Prosper: http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group |
From: Wes W. <wes...@gm...> - 2007-07-25 23:02:38
|
On 7/25/07, Jonathan Day <im...@ya...> wrote: > > A number of things came out of the BOF meeting on > Tuesday, but I shall leave most of the discussion on > that to the formal writeup of the minutes of the > meeting by Wes. I will only say that everyone there > was excited by the idea of having development > continue, with good feelings about where we are. > > One of the things examined was the use of a stub that > was not migratable between nodes. I was asked to write > a short piece describing a proposed modification to > this, so here it is. Note that suggestions, comments > and clue-stick beatings are welcome, but if you could > pad the clue-stick a little, I'd be grateful. > > Brief Synopsis: Linux offers a bazillion features, > network engineers have written a bazillion others. The > I/O-side stub, as it is, works great but there may be > ways to make it more powerful and/or faster. We should > consider if this would be a useful/good thing to do, > and if so what our priorities should be. > > > My suggestion is to first extend the stub. Linux has a > whole host of new features which should expand the > usefulness of the stub, and it would be good to > discuss what we'd want the stub to actually do. > > The second step I proposed was to allow the stub to > migrate. If the stub migrates, then all user-side I/O > would migrate with it to the new node, which must > (therefore) support the same I/O features as are being > used. So long as the information is available, I don't > see that being a problem. > > (By migrating the stub rather than just the data, you > carry all of the process state, security/permission > labels, etc, for free and you can reuse existing > code.) > > The problem lies in how to migrate the stubs safely > and maintain contact with the controlled process, > without adding any security complications and with > taking into account any data sent by the process > during the migration. > > Linux - after applying a couple of patches - provides > methods for supporting the failover of network > connections, transient addressing, network mobility, > kernel-level switching for layers 2, 3, 4 and 7, and > virtual network devices. In short, we have so many > ways of getting a connection to switch that it's more > a case of what method we should use rather than > whether we can do it. > > A worthy place to look for inspiration is the research > that has gone into IPv6 mobility. The current version > uses a transient home base that all connections link > to. The home base then talks to the user node. When > the user node moves, a redirect is sent through the > network. Any connection not yet redirected bounces off > the home base to wherever the user is. Redirected > connections go directly to the user. Depending on the > version of the mobility draft, once all connections > are redirected, the user becomes the new home base and > the old home base is closed down. > > Alternatively, why keep track at all? Have a > process-side stub that listens for anycasts. When the > user-side stub migrates, it sends out an anycast > request for the UPID it was connected to. The process' > stub will be the only one to respond, so the > connection can then be restored. > > What about kernel threads? Again, same thing applies. > We're not short of methods. Everything from HA > solutions to suspend-to-file would allow you to > transfer kernel-level operations from one machine to > another without problem. > > What information is needed by the stub? Well, we > definitely need all file descriptors, netlink sockets, > unix sockets, pipes, memory tags, tipc sockets to the > remote process and regular sockets for handling the > I/O. You'll also want the security tokens for both the > stub and the remote process, and the stub's state > information. Almost all of that already exists and the > others are relatively minor additions. > > Is there anything significant that's needed that's > new? Well, the stub should export to userspace any > information needed to perform migration and perhaps > resource allocation. At the meeting, we thought about > various strategies for this. /proc and /sys were > suggested, as was using a page of shared memory. I'd > love to hear other people's thoughts on this. > > (The point is that the data changes extremely > infrequently, very few processes will want it, there > is a potential security hole if anything can access > that data and modify it, and copying from the kernel > is slow so should only ever occur on an event, never > on a per-read basis.) > > I also suggested that since Linux now supports > real-time and that time-slicing is much more > predictable under new scheduler, we might want to > consider all processes under the control of OpenMosix > as real-time. Each process then has a fixed-sized > bucket of time, and the scheduler can use a > conventional packing herustic to pack those buckets > into the cluster. > > How is this better than using, say, the load average? > The load average doesn't tell you what's taking the > time or why, or whether the remaining time is usable. > (I'm not certain how much you can fragment the block > of time that the new scheduler uses to timeslice, but > I can be pretty certain that any desktop system is > going to have enough events to fragment the time > block, leaving unusable chunks.) > > Ok, so we want to export the timeslice requirements. > What else? Any universal ID would be good to export, > along with what machine the process is running on. We > also need to export memory tags, as RDMA should not be > done in the kernel. > > Why not export everything? Because 99% of what is on > the stub is strictly for I/O to other kernel-based > components. Very little is needed or wanted by > userspace, and the less you have to copy, track or > update, the less latency you'll suffer and the lower > your requirements. You also get better security. > > This ends the reading of the stub. > > > > > > ____________________________________________________________________________________ > Pinpoint customers who are looking for what you sell. > http://searchmarketing.yahoo.com/ > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > openMosix-devel mailing list > ope...@li... > https://lists.sourceforge.net/lists/listinfo/openmosix-devel > -- Wes Wagner Join a libertarian network of person-to-person lending on Prosper: http://www.prosper.com/groups/group_home.aspx?group_short_name=freelibertarians&referrer=AiriusTorpora&utm_source=referrer-AiriusTorpora&utm_medium=referral-link&utm_content=join_my_group-160x33&utm_campaign=referrals-group |