Menu
▾
▴
Re: [X10-users] APGAS lib: Adding Places across different mashines
|
From: Jonas P. <jon...@st...> - 2015-09-30 18:21:26
|
Hi Olivier, it works fine :) Another question about the SshLauncher: When I have allocated eg. 5 Nodes in a Cluster with 2 CPUS each, then I want to start 10 places overall, 2 places at every Node. Is there a way to do this easy with parameters? greets Jonas 2015-09-30 16:38 GMT+02:00 Olivier Tardieu <ta...@us...>: > Jonas, > > I pushed an update to convert the paths to absolute paths on the master > node. > It should now be possible to use the ssh launcher with relative paths. > > Olivier > > Jonas Posner <jon...@st...> wrote on 09/29/2015 > 10:08:57 AM: > > > From: Jonas Posner <jon...@st...> > > To: Mailing list for users of the X10 programming language <x10- > > us...@li...> > > Date: 09/29/2015 10:09 AM > > Subject: Re: [X10-users] APGAS lib: Adding Places across different > mashines > > > > Hey Olivier, > > > > it works! Thank you! > > > > Many greetings > > Jonas > > > > 2015-09-29 14:45 GMT+02:00 Olivier Tardieu <ta...@us...>: > > Jonas, > > > > For now the paths (classes and classpath) need to be absolute. > > On the remote nodes, relative paths will be relative to the user > > home directory rather than the launch path. > > > > Olivier > > > > Jonas Posner <jon...@st...> wrote on 09/29/ > > 2015 03:54:50 AM: > > > > > From: Jonas Posner <jon...@st...> > > > To: Mailing list for users of the X10 programming language <x10- > > > us...@li...> > > > Date: 09/29/2015 03:55 AM > > > Subject: Re: [X10-users] APGAS lib: Adding Places across different > mashines > > > > > > Hi Oliver, > > > > > > I tested at a cluster with gpfs filesystem, so the files should be > > > available on every node at the same paths. On every node a "java - > > > version" shows the same: java version "1.8.0_60". I have APGAS not > > > in a jar, but the normal sources in my project. That is not a > > > problem, right? So I tried to start my program with: > > > java -cp .:../lib/hazelcast-3.5.2.jar - > > > Dapgas.launcher=apgas.impl.SshLauncher -Dapgas.hostfile=hostfile - > > > Dapgas.places=2 examples.ResilientHelloWorld > > > > > > This is the output: > > > [APGAS] Spawning new place: ssh -t -t its-cs194.its.uni-kassel.de > > > java -Xbootclasspath:/gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/lib/ > > > resources.jar:/gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/lib/ > > > rt.jar:/gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/lib/ > > > sunrsasign.jar:/gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/lib/ > > > jsse.jar:/gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/lib/jce.jar:/ > > > gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/lib/charsets.jar:/gpfs/ > > > home08/0019/uk000618/jdk1.8.0_60/jre/lib/jfr.jar:/gpfs/home08/0019/ > > > uk000618/jdk1.8.0_60/jre/classes -cp .:../lib/hazelcast-3.5.2.jar - > > > Dapgas.hostfile=hostfile -Dapgas.serialization.exception=true - > > > Dapgas.places=2 -Dapgas.launcher.verbose=true -Dapgas.resilient=true > > > -Dapgas.launcher=apgas.impl.SshLauncher -Dapgas.my.master=192.168. > > 205.103:5701 > > > apgas.GlobalRuntime > > > tcgetattr: Invalid argument > > > Error: Could not find or load main class apgas.GlobalRuntime > > > Connection to its-cs194.its.uni-kassel.de closed. > > > > > > The hostfile only contains: "its-cs194.its.uni-kassel.de" > > > Where I started my programm "pwd" shows: "/home/users/0019/ > > uk000618/APGAS/bin" > > > > > > With a simple "java -cp .:../lib/hazelcast-3.5.2.jar - > > > Dapgas.places=2 examples.ResilientHelloWorld" it starts normal and > > > as expected with output: > > > [APGAS] Spawning new place: java -Xbootclasspath:/gpfs/home08/0019/ > > > uk000618/jdk1.8.0_60/jre/lib/resources.jar:/gpfs/home08/0019/ > > > uk000618/jdk1.8.0_60/jre/lib/rt.jar:/gpfs/home08/0019/uk000618/jdk1. > > > 8.0_60/jre/lib/sunrsasign.jar:/gpfs/home08/0019/uk000618/jdk1.8. > > > 0_60/jre/lib/jsse.jar:/gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/ > > > lib/jce.jar:/gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/lib/ > > > charsets.jar:/gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/lib/ > > > jfr.jar:/gpfs/home08/0019/uk000618/jdk1.8.0_60/jre/classes -cp .:../ > > > lib/hazelcast-3.5.2.jar -Dapgas.serialization.exception=true - > > > Dapgas.places=2 -Dapgas.launcher.verbose=true -Dapgas.resilient=true > > > -Dapgas.my.master=192.168.205.103:5701 apgas.GlobalRuntime > > > Running main at place(0) of 2 places > > > 0: There are 2 places > > > 1: There are 2 places > > > .... > > > > > > Many greetings > > > Jonas > > > > > > 2015-09-29 3:12 GMT+02:00 Olivier Tardieu <ta...@us...>: > > > Jonas, > > > > > > I did not explain the requirements... > > > > > > The ssh launcher expects the file system to look the same across the > nodes. > > > The compiled classes and the jars (apgas, hazelcast) should be > > > available at the same paths. > > > This is typically achieved by using a distributed file system. > > > Pushing the files using rsync, scp... before launching is also an > option. > > > > > > BTW, "java" is also expected to resolve to a compatible Java 8 jvm > > > on all the nodes. > > > But an explicit path can be provided using property -Dapgas.java=/ > > path/to/java > > > > > > You can specify "-Dapgas.launcher.verbose=true" to debug the invoked > > > command line. > > > > > > Olivier > > > > > > > > > Jonas Posner <jon...@st...> wrote on 09/28/ > > > 2015 02:45:48 PM: > > > > > > > From: Jonas Posner <jon...@st...> > > > > To: Mailing list for users of the X10 programming language <x10- > > > > us...@li...> > > > > Date: 09/28/2015 02:46 PM > > > > Subject: Re: [X10-users] APGAS lib: Adding Places across > > different mashines > > > > > > > > Hi Olivier, > > > > > > > > thank you a lot for your effort and replies. > > > > > > > > Until last week I used an older version from your git. After your > > > > first answer I updatet it. The options "-Dapgas.my.ip=" and "- > > > > Dapgas.my.master=" fixed my problems with the ResilientHelloWorld > > > > example. It works with two different mashines. After this simple > > > > test I tried something to let the first place wait for additional > > > > manually started places. Your mail today helps me a lot. The option > > > > "apgas.impl.NoLauncher" does exactly what you described. It works > > > > local and at the cluster, that I use. My "real" program runs very > > > > good with the NoLauncher, too. > > > > > > > > Currently the ssh launcher does not work for me. I created a > > > > hostfile with one node. Then I started the program with: > > > > java -cp .:../lib/hazelcast-3.5.2.jar - > > > > Dapgas.launcher=apgas.impl.SshLauncher -Dapgas.hostfile=hostfile - > > > > Dapgas.places=2 examples.ResilientHelloWorld > > > > The ssh connection to the node in the hostfile is successfull, but > > > > then there is an error: > > > > Error: Could not find or load main class apgas.GlobalRuntime > > > > Connection to xxx closed. > > > > Can you help me there? > > > > > > > > Many greetings > > > > Jonas > > > > > > > > 2015-09-28 16:55 GMT+02:00 Olivier Tardieu <ta...@us...>: > > > > Jonas, > > > > > > > > I pushed new launchers to git. > > > > > > > > I added a new ssh launcher. > > > > java -Dapgas.launcher=apgas.impl.SshLauncher > > > > -Dapgas.hostfile=/home/tardieu/hostfile -Dapgas.places=4 ... > > > > It uses ssh to run on remote nodes. Place 0 always run on localhost. > The > > > > hostfile should contain the hosts for the remaining places. 1 per > line. > > > > > > > > I added a new dummy launcher. > > > > java -Dapgas.launcher=apgas.impl.NoLauncher -Dapgas.places=4 ... > > > > This launcher does not do anything! It makes it possible for the > user to > > > > launch places explicitly instead of relying on a launcher. > > > > Place 0 will wait for 3 other places to connect before running main. > > > > > > > > I added the -Dapgas.laucher.verbose property. > > > > If set, the launcher prints the command executed to launch > additional > > > > places. > > > > Use in combination with apgas.impl.NoLauncher to know the command to > > > > invoke. > > > > > > > > Let me know if you still have issues running on multiple nodes. > > > > > > > > Olivier > > > > > > > > > > > > Jonas Posner <jon...@st...> wrote on > 09/23/2015 > > > > 09:30:58 AM: > > > > > > > > > From: Jonas Posner <jon...@st...> > > > > > To: x10...@li... > > > > > Date: 09/23/2015 09:31 AM > > > > > Subject: [X10-users] APGAS lib: Adding Places across different > mashines > > > > > > > > > > Hi all, > > > > > > > > > > I'm playing around with the APGAS library for Java. I want to add > > > > > some places to a running application using the ip:port adress. I > use > > > > > for testing the ResilientWorld Class in examples. Local works it > > > > > fine with the commands: > > > > > 1.application: java -cp .:../lib/hazelcast-3.4.jar > -Dapgas.places=1 > > > > > examples.ResilientHelloWorld > > > > > 2.application: java -cp .:../lib/hazelcast-3.4.jar > -Dapgas.places=1 > > > > > -Dapgas.master=localhost examples.ResilientHelloWorld > > > > > > > > > > The second application connects to the first and the output shows > > > > > two places as expected. Application two can crash and app one > > > > > catches a DeadPlaceException. Everything fine and as expected. > > > > > > > > > > But now i want to start the applications at tow different > machines. > > > > > Instead of "localhost" i wrote the ip of the first application. > > > > > Additionally I added the port :5701. But application two doesn't > > > > > connect. Both applications run alone. I also switched the port in > > > > > the Transport Class of APGAS, without success. I don't know what > to > > > > > do now. What requirements must be met? > > > > > > > > > > Thanks and many greetings > > > > > Jonas Posner > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > > Monitor Your Dynamic Infrastructure at Any Scale With Datadog! > > > > > Get real-time metrics from all of your servers, apps and tools > > > > > in one place. > > > > > SourceForge users - Click here to start your Free Trial of Datadog > now! > > > > > http://pubads.g.doubleclick.net/gampad/clk?id=241902991&iu=/4140 > > > > > _______________________________________________ > > > > > X10-users mailing list > > > > > X10...@li... > > > > > https://lists.sourceforge.net/lists/listinfo/x10-users > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > _______________________________________________ > > > > X10-users mailing list > > > > X10...@li... > > > > https://lists.sourceforge.net/lists/listinfo/x10-users > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > _______________________________________________ > > > > X10-users mailing list > > > > X10...@li... > > > > https://lists.sourceforge.net/lists/listinfo/x10-users > > > > > > > > > > > > > ------------------------------------------------------------------------------ > > > > > > _______________________________________________ > > > X10-users mailing list > > > X10...@li... > > > https://lists.sourceforge.net/lists/listinfo/x10-users > > > > > > > > > ------------------------------------------------------------------------------ > > > _______________________________________________ > > > X10-users mailing list > > > X10...@li... > > > https://lists.sourceforge.net/lists/listinfo/x10-users > > > > > > > > ------------------------------------------------------------------------------ > > > > _______________________________________________ > > X10-users mailing list > > X10...@li... > > https://lists.sourceforge.net/lists/listinfo/x10-users > > > > > ------------------------------------------------------------------------------ > > _______________________________________________ > > X10-users mailing list > > X10...@li... > > https://lists.sourceforge.net/lists/listinfo/x10-users > > > > > ------------------------------------------------------------------------------ > _______________________________________________ > X10-users mailing list > X10...@li... > https://lists.sourceforge.net/lists/listinfo/x10-users > |