Menu
▾
▴
[Openmosix-devel] Java Threads Migrating on openMosix -- successful applications
|
From: Schaad,John G - T. <jgs...@bp...> - 2005-03-29 18:54:25
|
I have had good success with two versions of Java threads migrating = automatically with openMosix on our 39-node cluster, running = 2.4.24-openmosix2smp with an "uptime" today of 202-days. 1) Blackdown Java 1.3.1 (green threads): =20 Used with straight, unmodified linux version of the General Electric = PSLF-14.2 Positive Sequence Load Flow and Stability Program (Java = program), with scripts configured to use the Blackdown Java 1.3.1 = libraries for BATCH MODE system simulations of the entire Western U.S. = Power Grid. For example: 300 simulations run on the cluster complete in = 1-hour. These 300 simulations would take a week on a single node of the = cluster. This also makes use of the "Hyperthreading" through which the = 39-node cluster has 156-CPU's. We run the INTERACTIVE MODE of PSLF on = individual cluster nodes using the Java 1.4.2 libraries, where the PSLF = Interactive User Interface depends to some extent on some of the newer = functions available in Java 1.4.2. ftp://metalab.unc.edu/pub/linux/devel/lang/java/blackdown.org/ http://www.gepower.com/prod_serv/products/utility_software/en/ge_pslf/ind= ex.htm 2) FSU Pthreads Library: Coding up the sample program given with this library, these work very = well as a demo on our 39-node openMosix cluster. We have not yet had = the opportunity to use it in an application. http://filibusta.crema.unimi.it/openmosix/fsu_threads_on_om/benchmark.htm= Test Summary: Cluster Test Program: BDIV -- search range of large integers for prime = numbers using Computational Threads Two Versions created: to "parallelize" the computation, and = parallelize CPU processes. ptDiv_STD -- compiled using Linux Threads, Standard, with Redhat = 8.0/9.0 Linux Distribution ptDiv_FSU -- compiled using FSU-Posix Threads , installed as additional = thread library. Basic Program Command: ./ptDiv_XXX -r 1048576 4194304 -t 40 -s -n -r 1048576 4194304 Range to Seach, min. max. -t 40 Spawn "40" Threads -s Force scheduler invocation for the test -n Force "no output" mode Then, I made three executable scripts: ptDiv_STD-40x5 -- runs "5" instances of the basic program command, = above ptDiv_FSU-40x5 -- runs "5" instances of the basic program command, = above ptDiv_FSU-40x50 -- runs "50" instances of the basic program command, = above -------------------------------------------------------------------------= ----------------------- Test System: Linux Cluster, Redhat Linux 9.0, openMosix-2.4.22-1-SMP = kernel, 39-nodes Nodes: 1-Gb RAM, Dual-CPU Intel 2.4-Ghz XEON [P4] with hyperthreading = enabled -------------------------------------------------------------------------= ----------------------- Test Results: [# nodes automatically used] LinuxThreads1: ptDiv_STD-40x5 -- [1-instance] Completion: = 14:14-min./sec. only 1 FSU-Threads1: ptDiv_FSU-40x5 -- [1-instance] Completion: = 2:28-min./sec. 3 FSU-Threads2: ptDiv_FSU-40x50 -- [2-instances] Completion: = 3:24-min./sec. 34=09 FSU-Threads3: ptDiv_FSU-40x50 -- [4-instances] Completion: = 4:36-min./sec. all 39=09 FSU-Threads4: ptDiv_FSU-40x50 -- [8-instances] Completion: = 8:11-min./sec. all 39 FSU-Threads5: ptDiv_FSU-40x50 -- [16-instances] Completion: = 15:40-min./sec. all 39 -------------------------------------------------------------------------= ----------------------- -------------------------------------------------------------------------= ---------------------------------- Analysis/Comparison of Results, based on equivalent "FSU-Threads5" = Thread-Count/Timing: FSU-Threads5: [40x50x16] =3D 32,000-threads; [total solution time =3D = 15:40-min-sec.] =3D 34.04 threads/sec. LinuxThreads1: [40x5x1] =3D 200-threads; [total solution time =3D = 14:14-min-sec.] =3D 0.234 threads/sec. [40x50x16] =3D 32,000-threads; [equiv.solution time =3D = 37:57:20-hours-min-sec.]=20 -------------------------------------------------------------------------= ---------------------------------- Let me know if you have any quesitons. --John Sent: Monday, March 28, 2005 8:14 PM Today's Topics: 1. Fwd: About migrating POSIX and Java threads (Siddharth) 2. libpthreads optimized for i686 (Paul Bennett) 3. RE: Fwd: About migrating POSIX and Java threads (Huntsinger, Reid) 4. Fwd: About migrating POSIX and Java threads (Siddharth) --__--__-- Message: 1 Date: Mon, 28 Mar 2005 20:33:05 +0530 From: Siddharth <sid...@gm...> Reply-To: Siddharth <sid...@gm...> To: ope...@li... Subject: [Openmosix-devel] Fwd: About migrating POSIX and Java threads Sir, I am Siddharth. My group has implemented a OpenMosix cluster in our Computer Lab. When I ran a program which used POSIX threads, I found that it did not migrate these threads to different nodes of the cluster. So, I would like to do a final year undergraduate project which migrates POSIX threads. I have a few questions.. a. Has it been done before? b. Is it feasible? What I know about POSIX threads is that it is a thread liibrary. A program may create any number of POSIX threads, but the kernel views the program as one process with no threads. And has anybody migrated Java threads over the cluster? We are also interested in migrating Java threads over the cluster. Thanking you, Siddharth --__--__-- Message: 2 Date: Mon, 28 Mar 2005 09:26:25 -0800 (PST) From: Paul Bennett <be...@sn...> To: ope...@li... Subject: [Openmosix-devel] libpthreads optimized for i686 Hi List, Has anyone had success with the i686 optimized libpthread patch from Mulyadi from back in July 2004 on and smp machine? It looks like this never made it into an official oM release. We've recently have run into some issues with perl DBI. When grabbing info serially from Oracle we're getting seg faults. These jobs would run fine unless there was other activity on the cluster, so it looked like the processes would get migrated and then seg faulting. Running gdb it looked to be related to libpthread. Coincidentally a couple weeks ago Christian Kristukat's post to the openMosix-general list about libpthreads led me to look more into this.=20 We had been having a lot of trouble trying to upgrade our old Mosix/RH7.1 cluster to oM/SuSE9.0. It seemed I could get oMtest to run good, but when one of our jobs (molecular modeling) ran we would have seemingly sporatic failures. I ended up reverting back all the way to 2.4.20-2 and things were usably stable.=20 Randomly we would have problems connecting to Oracle to grab data for a new modeling job and also doing a perl make that was part of the job. I'm wondering if this libpthread problem could also be related to these issues, which might help Guarionex Morales with his acid test, becasue his results seem to ring true for my experiences as well. We have an older oM cluster that's running Gentoo on 2.4.19-r7 and this problem has emerged (no pun intended) when we started doing some perl DBI stuff to Oracle. There is a recent glibc on this cluster (Oct 2004)that was added to support a software install so I think this could be where the optimized i686 libpthread crept in. This is also a production machine running lots of stuff so I'd like to minimize version changes. It doesn't look like a lot of work to patch this, but I don't want to mess around with things if it would be futile. =20 The hardware is Dual PIII/1 GHz CPU's, on Supermicro mobos with ServerWorks HE chipset. The first cluster has Gentoo 1.4.2.8 release the second has SuSE 9.0 pro. Both have vanilla kernels with the appropriate oM and NFS kernel patches. Any suggestions or input will be greatly appriciated, Paul --__--__-- Message: 3 From: "Huntsinger, Reid" <rei...@me...> To: "'Siddharth'" <sid...@gm...>, ope...@li... Subject: RE: [Openmosix-devel] Fwd: About migrating POSIX and Java threads Date: Mon, 28 Mar 2005 17:27:22 -0500 Hi Siddharth, The issue is with the thread model implemented by the thread library you use. The pthreads implementation on Linux *does* create separate = processes for threads (which share memory). Other thread libraries don't = necessarily do this; there's an account on the openMosix Wiki of using a user-space thread library called FSU threads, I recall. Reid Huntsinger -----Original Message----- From: ope...@li... [mailto:ope...@li...] On Behalf Of = Siddharth Sent: Monday, March 28, 2005 10:03 AM To: ope...@li... Subject: [Openmosix-devel] Fwd: About migrating POSIX and Java threads Sir, I am Siddharth. My group has implemented a OpenMosix cluster in our Computer Lab. When I ran a program which used POSIX threads, I found that it did not migrate these threads to different nodes of the cluster. So, I would like to do a final year undergraduate project which migrates POSIX threads. I have a few questions.. a. Has it been done before? b. Is it feasible? What I know about POSIX threads is that it is a thread liibrary. A program may create any number of POSIX threads, but the kernel views the program as one process with no threads. And has anybody migrated Java threads over the cluster? We are also interested in migrating Java threads over the cluster. Thanking you, Siddharth ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=3D6595&alloc_id=3D14396&op=3Dclick _______________________________________________ openMosix-devel mailing list ope...@li... https://lists.sourceforge.net/lists/listinfo/openmosix-devel -------------------------------------------------------------------------= ----- Notice: This e-mail message, together with any attachments, contains = information of Merck & Co., Inc. (One Merck Drive, Whitehouse Station, = New Jersey, USA 08889), and/or its affiliates (which may be known = outside the United States as Merck Frosst, Merck Sharp & Dohme or MSD = and in Japan, as Banyu) that may be confidential, proprietary = copyrighted and/or legally privileged. It is intended solely for the use = of the individual or entity named on this message. If you are not the = intended recipient, and have received this message in error, please = notify us immediately by reply e-mail and then delete it from your = system. -------------------------------------------------------------------------= ----- --__--__-- Message: 4 Date: Mon, 28 Mar 2005 20:12:39 +0530 From: Siddharth <sid...@gm...> Reply-To: Siddharth <sid...@gm...> To: Ope...@li... Subject: [Openmosix-devel] Fwd: About migrating POSIX and Java threads ---------- Forwarded message ---------- From: Siddharth <sid...@gm...> Date: Mon, 28 Mar 2005 11:30:10 +0530 Subject: About migrating POSIX and Java threads To: mo...@mo... Sir, I am Siddharth. My group has implemented a OpenMosix cluster in our Computer Lab. When I ran a program which used POSIX threads, I found that it did not migrate these threads to different nodes of the cluster. So, I would like to do a final year undergraduate project which migrates POSIX threads. I have a few questions.. a. Has it been done before? b. Is it feasible? What I know about POSIX threads is that it is a thread liibrary. A program may create any number of POSIX threads, but the kernel views the program as one process with no threads. And has anybody migrated Java threads over the cluster? We are also interested in migrating Java threads over the cluster. Thanking you, Siddharth --__--__-- _______________________________________________ openMosix-devel mailing list ope...@li... https://lists.sourceforge.net/lists/listinfo/openmosix-devel End of openMosix-devel Digest |