py4j-users Mailing List for Py4J (Page 2)
Status: Beta
Brought to you by:
barthe
You can subscribe to this list here.
2009 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
(1) |
---|---|---|---|---|---|---|---|---|---|---|---|---|
2010 |
Jan
|
Feb
(1) |
Mar
|
Apr
(1) |
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
(1) |
Nov
|
Dec
|
2011 |
Jan
(6) |
Feb
(1) |
Mar
(8) |
Apr
(6) |
May
|
Jun
(1) |
Jul
|
Aug
|
Sep
|
Oct
(4) |
Nov
|
Dec
(4) |
2012 |
Jan
|
Feb
|
Mar
(5) |
Apr
|
May
(7) |
Jun
(9) |
Jul
(10) |
Aug
|
Sep
|
Oct
(6) |
Nov
|
Dec
|
2013 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
(2) |
Jul
|
Aug
|
Sep
(3) |
Oct
|
Nov
|
Dec
(1) |
2014 |
Jan
(3) |
Feb
|
Mar
(3) |
Apr
|
May
(5) |
Jun
|
Jul
(2) |
Aug
(3) |
Sep
|
Oct
(1) |
Nov
|
Dec
(21) |
2015 |
Jan
(2) |
Feb
(5) |
Mar
(4) |
Apr
|
May
|
Jun
(5) |
Jul
(12) |
Aug
|
Sep
(1) |
Oct
(3) |
Nov
(1) |
Dec
(1) |
From: Barthelemy D. <bar...@in...> - 2015-03-28 00:08:13
|
Hi, how about: data = gateway.jvm.java.io.ByteArrayInputStream(jsonData.getBytes("UTF-8")) References in the documentation: http://py4j.sourceforge.net/faq.html#how-to-call-a-constructor http://py4j.sourceforge.net/getting_started.html#collections-help-and-constructors On Fri, Mar 27, 2015 at 1:36 PM, Israel Brewster <is...@ra...> wrote: > Is there a way in py4j to call the java "new" command? That is, in java I > have some code like: > > InputStream data=new ByteArrayInputStream(jsonData.getBytes("UTF-8")); > > Is there a way to pull that across to the python side, or do I just have > to make a bunch of utility functions in Java for this? > > I'm trying to make the Jasper Reports library available to my python code, > and while it works for the most part I haven't yet figured out object > creation. Thanks. > ----------------------------------------------- > Israel Brewster > Systems Analyst II > Ravn Alaska > 5245 Airport Industrial Rd > Fairbanks, AK 99709 > (907) 450-7293 > ----------------------------------------------- > > > > > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming The Go Parallel Website, > sponsored > by Intel and developed in partnership with Slashdot Media, is your hub for > all > things parallel software development, from weekly thought leadership blogs > to > news, videos, case studies, tutorials and more. Take a look and join the > conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > > |
From: Israel B. <is...@ra...> - 2015-03-27 18:38:56
|
Is there a way in py4j to call the java "new" command? That is, in java I have some code like: InputStream data=new ByteArrayInputStream(jsonData.getBytes("UTF-8")); Is there a way to pull that across to the python side, or do I just have to make a bunch of utility functions in Java for this? I'm trying to make the Jasper Reports library available to my python code, and while it works for the most part I haven't yet figured out object creation. Thanks. ----------------------------------------------- Israel Brewster Systems Analyst II Ravn Alaska 5245 Airport Industrial Rd Fairbanks, AK 99709 (907) 450-7293 ----------------------------------------------- |
From: Barthelemy D. <bar...@in...> - 2015-02-18 10:36:54
|
Hi Frank, pyrolite might be suitable for your needs. It runs a server on the python interpreter and definitively do some dynamic evaluation and type conversion. You could achieve something similar with Py4J, but as you noticed, there are more steps involved. The steps I outlined were indeed the same as the Operator example with a few more annotations :-) For example, OperatorExample could have a field, private Operator operator; with a getter and setter. >From Python, you could call: addition = Addition() ... myOperatorExample.setOperator(addition) # From now on, the Java side has a reference to your Python class. # Call a method that will tell the Java side that it is now in control gateway.jvm.yourpackage.YourControllerClass.youHaveControl() Barthelemy On Tue, Feb 17, 2015 at 12:48 PM, Frank J. Iannarilli, Jr. <fr...@ae...> wrote: > On 2/17/2015 4:32 AM, Barthelemy Dagenais wrote: >> >> The process is a little involved so there might be better alternatives >> (jython would probably not work with SciPy. not sure how pyjnius >> handles Java to Python). > > Hi Barthelemy, > > Thanks for your suggestions, to which I have a few follow-up questions. > > But first, do you have any opinions about another possible alternative, that > is to use Pyrolite? > https://pythonhosted.org/Pyro4/pyrolite.html > {pyrolite reportedly eliminates the Jython hop earlier required when using > the following chain: > Java<->Jython<->Pyro<->CPython; so now Java<->pyrolite<->CPython} > .... > > So if I understand your steps below, they are nearly parallel to your > example in > http://py4j.sourceforge.net/advanced_topics.html#implementing-java-interfaces-from-python-callback > > as follows below, and if so, I might simply first attempt to extend your > example: > >> The steps would be: >> >> 1. In Java, declare an interface (e.g., MySciPy). >> 2. In Java, create a class that will hold an instance of MySciPy. This >> is the class you will use in Java to "talk to Python through MySciPy". >> We will call it PythonProxy with setMySciPy and getMySciPy >> 3. In Python, implement the interface in a class (e.g., MySciPyImpl) > > > 1. MySciPy=interface Operator (Java side) > 2. PythonProxy=OperatorExample (Java side) > --> "holding instance of > MySciPy"=OperatorExample.randomBinaryOperator(Operator op) > 3. MySciPyImpl=Addition (Python side) > >> Then, once you are done, the control flow would be: >> ... >> 4. In Python, call PythonProxy.setMySciPy(myscipyimpl_instance) >> ... >> 6. Your Java code can now call Python with >> PythonProxy.getMySciPy().yourMethodHere() > > > 4. In Python, call PythonProxy.setMySciPy(myscipyimpl_instance) <=> > gateway.entry_point.randomBinaryOperator(operator) > OR > gateway.jvm.py4j.examples.OperatorExample().randomBinaryOperator(operator) > > 6. yourMethodHere() = doOperation() > > > But now to achieve what I'm seeking, which is your Step 6, I'm still > mystified but would the following get me close?: > > On Step2, in my Java client code, I would also do: > myOperatorExample = new OperatorExample(); > GatewayServer server = new GatewayServer(myOperatorExample); > server.start(); > > so that: > > 6. In Java, PythonProxy.getMySciPy().yourMethodHere() <=> > numbers = myOperatorExample.randomBinaryOperator(Operator.doOperation) > > I don't see how the above would bind myOperatorExample to the actual Python > proxy. > > > Thanks! > > > > > > > -- > > Frank J. Iannarilli fr...@ae... > Aerodyne Research, Inc., 45 Manning Road, Billerica, MA 01821 USA > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > |
From: Frank J. I. Jr. <fr...@ae...> - 2015-02-17 17:48:16
|
On 2/17/2015 4:32 AM, Barthelemy Dagenais wrote: > The process is a little involved so there might be better alternatives > (jython would probably not work with SciPy. not sure how pyjnius > handles Java to Python). Hi Barthelemy, Thanks for your suggestions, to which I have a few follow-up questions. But first, do you have any opinions about another possible alternative, that is to use Pyrolite? https://pythonhosted.org/Pyro4/pyrolite.html {pyrolite reportedly eliminates the Jython hop earlier required when using the following chain: Java<->Jython<->Pyro<->CPython; so now Java<->pyrolite<->CPython} .... So if I understand your steps below, they are nearly parallel to your example in http://py4j.sourceforge.net/advanced_topics.html#implementing-java-interfaces-from-python-callback as follows below, and if so, I might simply first attempt to extend your example: > The steps would be: > > 1. In Java, declare an interface (e.g., MySciPy). > 2. In Java, create a class that will hold an instance of MySciPy. This > is the class you will use in Java to "talk to Python through MySciPy". > We will call it PythonProxy with setMySciPy and getMySciPy > 3. In Python, implement the interface in a class (e.g., MySciPyImpl) 1. MySciPy=interface Operator (Java side) 2. PythonProxy=OperatorExample (Java side) --> "holding instance of MySciPy"=OperatorExample.randomBinaryOperator(Operator op) 3. MySciPyImpl=Addition (Python side) > Then, once you are done, the control flow would be: > ... > 4. In Python, call PythonProxy.setMySciPy(myscipyimpl_instance) > ... > 6. Your Java code can now call Python with > PythonProxy.getMySciPy().yourMethodHere() 4. In Python, call PythonProxy.setMySciPy(myscipyimpl_instance) <=> gateway.entry_point.randomBinaryOperator(operator) OR gateway.jvm.py4j.examples.OperatorExample().randomBinaryOperator(operator) 6. yourMethodHere() = doOperation() But now to achieve what I'm seeking, which is your Step 6, I'm still mystified but would the following get me close?: On Step2, in my Java client code, I would also do: myOperatorExample = new OperatorExample(); GatewayServer server = new GatewayServer(myOperatorExample); server.start(); so that: 6. In Java, PythonProxy.getMySciPy().yourMethodHere() <=> numbers = myOperatorExample.randomBinaryOperator(Operator.doOperation) I don't see how the above would bind myOperatorExample to the actual Python proxy. Thanks! -- Frank J. Iannarilli fr...@ae... Aerodyne Research, Inc., 45 Manning Road, Billerica, MA 01821 USA |
From: Barthelemy D. <bar...@in...> - 2015-02-17 09:33:01
|
Hi Frank, Py4J can be used to call Python code from Java, but because of the static nature of Java and the design of Py4J this can be only achieved through Java interfaces. Py4J enables a communication channel between a Python interpreter and a JVM. Usually the control flow is driven by Python, but you could certainly spawn a Python interpreter from Java and control it with some synchronization. Because Python is dynamic, it does not need to create in advance all possible interfaces/classes present in the JVM and it can dynamically wrap any Java instance. On the Java side, Java needs to have access to all interfaces at compilation time so Py4J requires that Python code implements a Java interface to be used by the Java side. The process is a little involved so there might be better alternatives (jython would probably not work with SciPy. not sure how pyjnius handles Java to Python). The steps would be: 1. In Java, declare an interface (e.g., MySciPy). 2. In Java, create a class that will hold an instance of MySciPy. This is the class you will use in Java to "talk to Python through MySciPy". We will call it PythonProxy with setMySciPy and getMySciPy 3. In Python, implement the interface in a class (e.g., MySciPyImpl) Then, once you are done, the control flow would be: 1. Start a Python interpreter (this could be started from your Java program) 2. Connect the Python interpreter to your JVM (i.e., with JavaGateway(...)) 3. In Python, create an instance of the class MySciPyImpl 4. In Python, call PythonProxy.setMySciPy(myscipyimpl_instance) 5. In Python, notifies your Java code that it can take control (e.g., by calling a run loop, waking up a thread, etc.) 6. Your Java code can now call Python with PythonProxy.getMySciPy().yourMethodHere() There are some limitations: the Python code cannot send Python objects to the Java side, it can only send primitives or references to Java objects. It could maintain references to Python objects that would be accessed through the other methods of your MySciPy interface though. Some workarounds: a) You could call eval or exec in your Python code. From Java, you would then just send strings. b) I'm open to feature requests such as automatically wrapping Python collections to Java interfaces (e.g., you could return a Python sequence and it would appear to be a java.util.List on the Java side). HTH, Barthelemy On Tue, Feb 17, 2015 at 2:21 AM, Stephen Boesch <ja...@gm...> wrote: > HI py4j is for going the other way: the Gateway serves java classes for use > by python. > > 2015-02-16 20:38 GMT-08:00 Frank J. Iannarilli, Jr. <fr...@ae...>: >> >> Hi, >> >> >> Barthelemy, thanks for contributing Py4J! >> >> Is Py4J suitable for enabling me to call Python methods (e.g. SciPy >> library) from my Java program? >> >> I've succeeded in exercising the Getting Started example: >> http://py4j.sourceforge.net/getting_started.html >> >> I am trying to grok your example: >> >> http://py4j.sourceforge.net/advanced_topics.html#implementing-java-interfaces-from-python-callback >> which was pointed to in response to a similar StackOverflow question: >> >> http://stackoverflow.com/questions/23157424/py4j-how-would-i-go-about-on-calling-a-python-method-in-java?rq=1 >> >> Although I can see (from the OperatorExample.java file in the >> distribution) that Java File2 is not showing this method: >> public static void main(String[] args) { >> GatewayServer server = new GatewayServer(new OperatorExample()); >> server.start(); >> } >> , >> I still can't see from this example how to call Python functions from my >> Java program. Where this example (final lines in Python code) show: >> # "Sends" python object to the Java side. >> numbers = operator_example.randomBinaryOperator(operator) >> the meaning of "Sends" seems to imply a "non-stop round-trip" starting >> from and ending on the Python side. >> >> Is Py4J the wrong tool to be using for what I'm seeking? Any advice >> appreciated! >> Thanks, >> Frank >> >> >> >> -- >> >> Frank J. Iannarilli fr...@ae... >> Aerodyne Research, Inc., 45 Manning Road, Billerica, MA 01821 USA >> >> >> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> >> http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users >> > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > |
From: Stephen B. <ja...@gm...> - 2015-02-17 07:22:03
|
HI py4j is for going the other way: the Gateway serves java classes for use by python. 2015-02-16 20:38 GMT-08:00 Frank J. Iannarilli, Jr. <fr...@ae...>: > Hi, > > > Barthelemy, thanks for contributing Py4J! > > Is Py4J suitable for enabling me to call Python methods (e.g. SciPy > library) from my Java program? > > I've succeeded in exercising the Getting Started example: > http://py4j.sourceforge.net/getting_started.html > > I am trying to grok your example: > http://py4j.sourceforge.net/advanced_topics.html# > implementing-java-interfaces-from-python-callback > which was pointed to in response to a similar StackOverflow question: > http://stackoverflow.com/questions/23157424/py4j-how- > would-i-go-about-on-calling-a-python-method-in-java?rq=1 > > Although I can see (from the OperatorExample.java file in the > distribution) that Java File2 is not showing this method: > public static void main(String[] args) { > GatewayServer server = new GatewayServer(new OperatorExample()); > server.start(); > } > , > I still can't see from this example how to call Python functions from my > Java program. Where this example (final lines in Python code) show: > # "Sends" python object to the Java side. > numbers = operator_example.randomBinaryOperator(operator) > the meaning of "Sends" seems to imply a "non-stop round-trip" starting > from and ending on the Python side. > > Is Py4J the wrong tool to be using for what I'm seeking? Any advice > appreciated! > Thanks, > Frank > > > > -- > > Frank J. Iannarilli fr...@ae... > Aerodyne Research, Inc., 45 Manning Road, Billerica, MA 01821 USA > > > > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > > http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > > |
From: Frank J. I. Jr. <fr...@ae...> - 2015-02-17 04:38:37
|
Hi, Barthelemy, thanks for contributing Py4J! Is Py4J suitable for enabling me to call Python methods (e.g. SciPy library) from my Java program? I've succeeded in exercising the Getting Started example: http://py4j.sourceforge.net/getting_started.html I am trying to grok your example: http://py4j.sourceforge.net/advanced_topics.html#implementing-java-interfaces-from-python-callback which was pointed to in response to a similar StackOverflow question: http://stackoverflow.com/questions/23157424/py4j-how-would-i-go-about-on-calling-a-python-method-in-java?rq=1 Although I can see (from the OperatorExample.java file in the distribution) that Java File2 is not showing this method: public static void main(String[] args) { GatewayServer server = new GatewayServer(new OperatorExample()); server.start(); } , I still can't see from this example how to call Python functions from my Java program. Where this example (final lines in Python code) show: # "Sends" python object to the Java side. numbers = operator_example.randomBinaryOperator(operator) the meaning of "Sends" seems to imply a "non-stop round-trip" starting from and ending on the Python side. Is Py4J the wrong tool to be using for what I'm seeking? Any advice appreciated! Thanks, Frank -- Frank J. Iannarilli fr...@ae... Aerodyne Research, Inc., 45 Manning Road, Billerica, MA 01821 USA |
From: Barthelemy D. <bar...@in...> - 2015-01-14 01:56:02
|
Hi, Apologies for the delay. Variadic function parameters are not supported yet. I'll have to take a look at the reflection API and revisit the Java specs to determine how to support this feature. Any help would be appreciated (e.g., opening a ticket on Github, providing examples or tests clearly showing method signature precedence when methods are overloaded, etc.). As always, a workaround would be to implement a wrapper in Java and call the wrapper from Python. That's clearly not ideal though. Thanks, Barthelemy On Thu, Jan 1, 2015 at 3:15 PM, Wilfried Fauvel <wil...@gm...> wrote: > Hi, > > I would like to know if there is a way to deal with java variadic > function parameters from python ? > Thank you. > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users |
From: Wilfried F. <wil...@gm...> - 2015-01-01 20:15:45
|
Hi, I would like to know if there is a way to deal with java variadic function parameters from python ? Thank you. |
From: Barthelemy D. <bar...@in...> - 2014-12-31 01:28:15
|
Hi, The TLDR: you are trying to access a class unknown to the JVM. I believe it is not correctly added to the Java classpath OR the classloader of the GatewayServer instance on the JVM does not have access to your class. Someone from the PySpark team may better help you. To check if this is a problem with the classpath/classloader, try something like that: # Sanity test string_class = gateway.jvm.java.lang.Class.forName("java.lang.String") # Will return java.lang.String string_class.getName() # Will return java.lang.Class string_class.getClass().getName() # Will raise an exception if the class is not found by the JVM jar_test_class = gateway.jvm.java.lang.Class.forName("com.mycompany.spark.test.JarTest") Now for the long version: Py4J is extremely lazy and dynamic. It does not try to preload or verify that something exists on the JVM until it absolutely has to. Because of the limited introspection capabilities of the JVM when it comes to available packages, Py4J does not know in advance all available packages and classes. What happens here is that Py4J tries to find a class "JarTest" in the com.mycompany.spark.test package. Because it cannot find such as class, it considers JarTest to be a package. When you try to call ping(), Py4J tells you that you cannot call a method on a package. I agree that the error message could be improved, but it is essentially the same error as if you were trying to call java.lang.ping() Regarding java_import(): this serves the same purpose as the import statement in Java, i.e., it lets you refer to a class with its unqualified name. It does not even try to check if the class or package exists. It does not "load" the class as the import statement does in python. So you could do: # ArrayList2 does not exist, py4j does not complain java_import(gateway.jvm, "java.util.ArrayList2") # ArrayList exists java_import(gateway.jvm, "java.util.ArrayList") # No need to use qualified name. a_list = gateway.jvm.ArrayList() # No need to import a class to use it with a FQN another_list = gateway.jvm.java.util.LinkedList() Hope this helps, Barthelemy On Tue, Dec 30, 2014 at 1:21 PM, Stephen Boesch <ja...@gm...> wrote: > > My team has added a module for pyspark which is a heavy user of py4j. I > have not been successful to invoke the newly added scala/java classes from > python (pyspark) via their java gateway. > > > The pyspark code creates a java gateway: > gateway = JavaGateway(GatewayClient(port=gateway_port), > auto_convert=False) > > Here is an example of existing (/working) pyspark java_gateway code: > > java_import(gateway.jvm, "org.apache.spark.sql.hive.HiveContext") > > I have attempted to import a trivial custom class: > > java_import(gateway.jvm, "com.mycompany.spark.test.JarTest") > > > But when invoking I get: > > tclass = self._jvm.com.mycompany.spark.test.JarTest > tclass.ping() # Error happens on this line > > > Py4JError: Trying to call a package. > > > So I would request guidance / documentation on how to > (a) incorporate java classes into an existing java gateway > (b) how to invoke those classes from the python client side > (c) how to get more visibility/insight into exactly what is going on / > what the meaning is of > "Trying to call a package" > > Thanks! > > > > > > > > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > |
From: Stephen B. <ja...@gm...> - 2014-12-30 18:21:10
|
My team has added a module for pyspark which is a heavy user of py4j. I have not been successful to invoke the newly added scala/java classes from python (pyspark) via their java gateway. The pyspark code creates a java gateway: gateway = JavaGateway(GatewayClient(port=gateway_port), auto_convert=False) Here is an example of existing (/working) pyspark java_gateway code: java_import(gateway.jvm, "org.apache.spark.sql.hive.HiveContext") I have attempted to import a trivial custom class: java_import(gateway.jvm, "com.mycompany.spark.test.JarTest") But when invoking I get: tclass = self._jvm.com.mycompany.spark.test.JarTest tclass.ping() # Error happens on this line Py4JError: Trying to call a package. So I would request guidance / documentation on how to (a) incorporate java classes into an existing java gateway (b) how to invoke those classes from the python client side (c) how to get more visibility/insight into exactly what is going on / what the meaning is of "Trying to call a package" Thanks! |
From: Barthelemy D. <bar...@in...> - 2014-12-30 16:35:29
|
Hi, when printing a Java List in Python, Py4J retrieves an iterator and iterates over the entire list to print the individual elements. NoSuchElementException is thrown by Java once the iterator has reached the end of the list (this is the way Java iterators work). Py4J then catches this exception on the Python side and raises a StopIteration error which will stop the list iteration. The exception you see on the Java side is normal. What I am trying to assess is what happens on the Python side. From what you said, it seems the list is not printed and the gateway shuts down. Is this correct? If so, this is strange. Enabling logging on the Python side might help: import logging logger = logging.getLogger("py4j") logger.setLevel(logging.DEBUG) logger.addHandler(logging.StreamHandler()) Regarding the Java logging, the FAQ already provides two working ways to enable logging. You made a small mistake when you tried to set the log level, it should be: logger.setLevel(logging.Level.ALL) I believe you may have mixed the instructions on enabling logging in Java and Python :-) This is really a strange bug: there are many unit tests testing this specific use case (retrieving a string representation of a Java list). I will try to run the tests in cygwin/windows later this week. Cheers, Barthelemy On Mon, Dec 29, 2014 at 10:16 PM, <hg...@it...> wrote: > Hi, > > (A) exception from accessing list > > The versions are here: > py4j 0.8.2.1 > python 2.7.8 > Java 1.8.0_20 > > I open two cygwin consoles. One for running the Gateway Server, and the other is to run python script. > > When I run the python script to print an ArrayList as your advice, the same exception raised in the server side. Part of the trace stack is as below: > ... > Caused by: java.util.NoSuchElementException > at java.util.ArrayList$Itr.next(ArrayList.java:854) > > It's OK to access the list individually by indices, but it causes exception when directly typing the "a_list" or "print(a_ list)". > > When the list is accessed as a whole, what would the gateway server do the list? It seems the server tries to iterate to elements that don't exist. > > (B) exception from setting logging in python script > > I follow your faq page of how to turn logging on-off and get the following exception: > >>>> logging = gw.jvm.java.util.logging >>>> logger = logging.Logger.getLogger("py4j") >>>> logger.setLevel(logging.DEBUG) > Traceback (most recent call last): > File "<stdin>", line 1, in <module> > File "/usr/lib/python2.7/site-packages/py4j/java_gateway.py", line 529, in __call__ > [get_command_part(arg, self.pool) for arg in new_args]) > File "/usr/lib/python2.7/site-packages/py4j/protocol.py", line 242, in get_command_part > if parameter == None: > File "/usr/lib/python2.7/site-packages/py4j/java_gateway.py", line 726, in __getattr__ > raise Py4JError('Trying to call a package.') > py4j.protocol.Py4JError: Trying to call a package. > > Maybe faq page needs to add more information on this situation. > > Frank Yang > > > ________________________________________ > 從: Barthelemy Dagenais <bar...@in...> > 寄件日期: 2014年12月28日 04:42 > 至: Support and Comments about Py4J > 主旨: Re: [Py4j-users] 回覆: Exception when printing a returned List<String> object > > Hi, > > apologies for the delays :-) > > The behavior you are seeing is strange. > > Here are a few pointers: > > a) Error Logging can be turned on and off on both sides: > http://py4j.sourceforge.net/faq.html#how-to-turn-logging-on-off > > b) Which version of py4j, python, and Java are you using? > > c) Are you able to print a standard list, e.g., ArrayList? e.g., > > a_list = gateway.jvm.java.util.ArrayList() > a_list.append("a") > a_list.append("b") > print(a_list) > > d) The exceptions you shown seems to come from the JVM. Do you see any > exception on the Python side? > > Regards, > Barthelemy > > On Tue, Dec 23, 2014 at 9:56 PM, <hg...@it...> wrote: >> Hi, >> >> Yes, it prints <class 'py4j.java_collections.JavaIterator'>. >> So couldn't I directly assign the return List<String> to a python list like the following and then print it? >> >>>>>ret = [] >>>>>ret = gateway.entry_point.getMyStringList(inputStr) >>>>>print ret >> >> If that's the case, for now my workaround for this is below: >> >>>>>ans = [ret[i] for i in range(len(ret))] >> >> and then use "ans" instead. >> >> Could you have any suggestion on this otherwise? >> >> Besides, because the exception message only appears partially, how can I log full error messages and debug when the server stops? I think it's especially necessary when the error behavior is in the server side. >> >> Thanks, >> Frank Yang >> ________________________________________ >> 從: Barthelemy Dagenais <bar...@in...> >> 寄件日期: 2014年12月23日 21:14 >> 至: Support and Comments about Py4J >> 主旨: Re: [Py4j-users] Exception when printing a returned List<String> object >> >> Hi, >> >> When printing the list, Py4J retrieves an Iterator and wraps it with >> py4j.java_collections.JavaIterator. It then cycles through the list >> with the iterator. JavaIterator should catch the >> NoSuchElementException and raise instead a StopIteration. Because it >> does not, I suspect something is messing with the automated wrapping >> of collections. >> >> could you do: >> >> iter = ret.iterator() >> print(type(iter)) >> >> This should print something like <class 'py4j.java_collections.JavaIterator'> >> >> Thanks, >> Barthelemy >> >> On Tue, Dec 23, 2014 at 6:10 AM, <hg...@it...> wrote: >>> Hi, >>> >>> >>> The method in my gateway server will return a List<String> object. When I >>> access it in my python client code, the server has exception and then >>> closed. The following is part of the exceptions >>> >>> >>> >>> >>> ▒Q▒G▒▒ 23, 2014 5:18:14 ▒U▒▒ py4j.reflection.MethodInvoker invoke >>> ĵ▒i: Exception occurred in client code. >>> java.lang.reflect.InvocationTargetException >>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>> at >>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:483) >>> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) >>> at >>> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) >>> at py4j.Gateway.invoke(Gateway.java:259) >>> at >>> py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) >>> at py4j.commands.CallCommand.execute(CallCommand.java:79) >>> at py4j.GatewayConnection.run(GatewayConnection.java:207) >>> at java.lang.Thread.run(Thread.java:745) >>> Caused by: java.util.NoSuchElementException >>> at java.util.AbstractList$Itr.next(AbstractList.java:364) >>> ... 11 more >>> >>> What I do in client code is the following: >>> >>>>>>ret = gateway.entry_point.getMyStringList(inputStr) # it's ok to run >>>>>> this line >>>>>>print ret # when running to this line, the server raised exception >>> >>> >>> Below is the method in server side: >>> >>> public List<String> getMyStringList(String input) throws IOException { >>> List<String> strList = new ArrayList<String>(); >>> strList = segmenter.segmentString(input); // this is some 3rd party >>> API for string tokenization and has been tested OK in console, the return >>> object is List<String> >>> >>> return strList; >>> } >>> >>> It's OK to access "ret" using indices and len(), but when I just type "ret" >>> or "print ret", then it would display the content of "ret", but then the >>> server poped Exception and stopped. What would be the problemt? Thanks for >>> any help! >>> >>> >>> >>> Frank Yang >>> >>> >>> 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 >>> This email may contain confidential information. Please do not use or >>> disclose it in any way and delete it if you are not the intended recipient. >>> ------------------------------------------------------------------------------ >>> Dive into the World of Parallel Programming! The Go Parallel Website, >>> sponsored by Intel and developed in partnership with Slashdot Media, is your >>> hub for all things parallel software development, from weekly thought >>> leadership blogs to news, videos, case studies, tutorials and more. Take a >>> look and join the conversation now. http://goparallel.sourceforge.net >>> _______________________________________________ >>> Py4j-users mailing list >>> Py4...@li... >>> https://lists.sourceforge.net/lists/listinfo/py4j-users >>> >> >> ------------------------------------------------------------------------------ >> Dive into the World of Parallel Programming! The Go Parallel Website, >> sponsored by Intel and developed in partnership with Slashdot Media, is your >> hub for all things parallel software development, from weekly thought >> leadership blogs to news, videos, case studies, tutorials and more. Take a >> look and join the conversation now. http://goparallel.sourceforge.net >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users >> >> ==================================================================== >> 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 >> This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. >> ------------------------------------------------------------------------------ >> Dive into the World of Parallel Programming! The Go Parallel Website, >> sponsored by Intel and developed in partnership with Slashdot Media, is your >> hub for all things parallel software development, from weekly thought >> leadership blogs to news, videos, case studies, tutorials and more. Take a >> look and join the conversation now. http://goparallel.sourceforge.net >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > > > ==================================================================== > 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 > This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users |
From: <hg...@it...> - 2014-12-30 03:17:38
|
Hi, (A) exception from accessing list The versions are here: py4j 0.8.2.1 python 2.7.8 Java 1.8.0_20 I open two cygwin consoles. One for running the Gateway Server, and the other is to run python script. When I run the python script to print an ArrayList as your advice, the same exception raised in the server side. Part of the trace stack is as below: ... Caused by: java.util.NoSuchElementException at java.util.ArrayList$Itr.next(ArrayList.java:854) It's OK to access the list individually by indices, but it causes exception when directly typing the "a_list" or "print(a_ list)". When the list is accessed as a whole, what would the gateway server do the list? It seems the server tries to iterate to elements that don't exist. (B) exception from setting logging in python script I follow your faq page of how to turn logging on-off and get the following exception: >>> logging = gw.jvm.java.util.logging >>> logger = logging.Logger.getLogger("py4j") >>> logger.setLevel(logging.DEBUG) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.7/site-packages/py4j/java_gateway.py", line 529, in __call__ [get_command_part(arg, self.pool) for arg in new_args]) File "/usr/lib/python2.7/site-packages/py4j/protocol.py", line 242, in get_command_part if parameter == None: File "/usr/lib/python2.7/site-packages/py4j/java_gateway.py", line 726, in __getattr__ raise Py4JError('Trying to call a package.') py4j.protocol.Py4JError: Trying to call a package. Maybe faq page needs to add more information on this situation. Frank Yang ________________________________________ 從: Barthelemy Dagenais <bar...@in...> 寄件日期: 2014年12月28日 04:42 至: Support and Comments about Py4J 主旨: Re: [Py4j-users] 回覆: Exception when printing a returned List<String> object Hi, apologies for the delays :-) The behavior you are seeing is strange. Here are a few pointers: a) Error Logging can be turned on and off on both sides: http://py4j.sourceforge.net/faq.html#how-to-turn-logging-on-off b) Which version of py4j, python, and Java are you using? c) Are you able to print a standard list, e.g., ArrayList? e.g., a_list = gateway.jvm.java.util.ArrayList() a_list.append("a") a_list.append("b") print(a_list) d) The exceptions you shown seems to come from the JVM. Do you see any exception on the Python side? Regards, Barthelemy On Tue, Dec 23, 2014 at 9:56 PM, <hg...@it...> wrote: > Hi, > > Yes, it prints <class 'py4j.java_collections.JavaIterator'>. > So couldn't I directly assign the return List<String> to a python list like the following and then print it? > >>>>ret = [] >>>>ret = gateway.entry_point.getMyStringList(inputStr) >>>>print ret > > If that's the case, for now my workaround for this is below: > >>>>ans = [ret[i] for i in range(len(ret))] > > and then use "ans" instead. > > Could you have any suggestion on this otherwise? > > Besides, because the exception message only appears partially, how can I log full error messages and debug when the server stops? I think it's especially necessary when the error behavior is in the server side. > > Thanks, > Frank Yang > ________________________________________ > 從: Barthelemy Dagenais <bar...@in...> > 寄件日期: 2014年12月23日 21:14 > 至: Support and Comments about Py4J > 主旨: Re: [Py4j-users] Exception when printing a returned List<String> object > > Hi, > > When printing the list, Py4J retrieves an Iterator and wraps it with > py4j.java_collections.JavaIterator. It then cycles through the list > with the iterator. JavaIterator should catch the > NoSuchElementException and raise instead a StopIteration. Because it > does not, I suspect something is messing with the automated wrapping > of collections. > > could you do: > > iter = ret.iterator() > print(type(iter)) > > This should print something like <class 'py4j.java_collections.JavaIterator'> > > Thanks, > Barthelemy > > On Tue, Dec 23, 2014 at 6:10 AM, <hg...@it...> wrote: >> Hi, >> >> >> The method in my gateway server will return a List<String> object. When I >> access it in my python client code, the server has exception and then >> closed. The following is part of the exceptions >> >> >> >> >> ▒Q▒G▒▒ 23, 2014 5:18:14 ▒U▒▒ py4j.reflection.MethodInvoker invoke >> ĵ▒i: Exception occurred in client code. >> java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:483) >> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) >> at >> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) >> at py4j.Gateway.invoke(Gateway.java:259) >> at >> py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) >> at py4j.commands.CallCommand.execute(CallCommand.java:79) >> at py4j.GatewayConnection.run(GatewayConnection.java:207) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.util.NoSuchElementException >> at java.util.AbstractList$Itr.next(AbstractList.java:364) >> ... 11 more >> >> What I do in client code is the following: >> >>>>>ret = gateway.entry_point.getMyStringList(inputStr) # it's ok to run >>>>> this line >>>>>print ret # when running to this line, the server raised exception >> >> >> Below is the method in server side: >> >> public List<String> getMyStringList(String input) throws IOException { >> List<String> strList = new ArrayList<String>(); >> strList = segmenter.segmentString(input); // this is some 3rd party >> API for string tokenization and has been tested OK in console, the return >> object is List<String> >> >> return strList; >> } >> >> It's OK to access "ret" using indices and len(), but when I just type "ret" >> or "print ret", then it would display the content of "ret", but then the >> server poped Exception and stopped. What would be the problemt? Thanks for >> any help! >> >> >> >> Frank Yang >> >> >> 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 >> This email may contain confidential information. Please do not use or >> disclose it in any way and delete it if you are not the intended recipient. >> ------------------------------------------------------------------------------ >> Dive into the World of Parallel Programming! The Go Parallel Website, >> sponsored by Intel and developed in partnership with Slashdot Media, is your >> hub for all things parallel software development, from weekly thought >> leadership blogs to news, videos, case studies, tutorials and more. Take a >> look and join the conversation now. http://goparallel.sourceforge.net >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users >> > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > > ==================================================================== > 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 > This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users ------------------------------------------------------------------------------ Dive into the World of Parallel Programming! The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net _______________________________________________ Py4j-users mailing list Py4...@li... https://lists.sourceforge.net/lists/listinfo/py4j-users ==================================================================== 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. |
From: Barthelemy D. <bar...@in...> - 2014-12-29 09:13:51
|
Hi, apologies for the delays :-) The behavior you are seeing is strange. Here are a few pointers: a) Error Logging can be turned on and off on both sides: http://py4j.sourceforge.net/faq.html#how-to-turn-logging-on-off b) Which version of py4j, python, and Java are you using? c) Are you able to print a standard list, e.g., ArrayList? e.g., a_list = gateway.jvm.java.util.ArrayList() a_list.append("a") a_list.append("b") print(a_list) d) The exceptions you shown seems to come from the JVM. Do you see any exception on the Python side? Regards, Barthelemy On Tue, Dec 23, 2014 at 9:56 PM, <hg...@it...> wrote: > Hi, > > Yes, it prints <class 'py4j.java_collections.JavaIterator'>. > So couldn't I directly assign the return List<String> to a python list like the following and then print it? > >>>>ret = [] >>>>ret = gateway.entry_point.getMyStringList(inputStr) >>>>print ret > > If that's the case, for now my workaround for this is below: > >>>>ans = [ret[i] for i in range(len(ret))] > > and then use "ans" instead. > > Could you have any suggestion on this otherwise? > > Besides, because the exception message only appears partially, how can I log full error messages and debug when the server stops? I think it's especially necessary when the error behavior is in the server side. > > Thanks, > Frank Yang > ________________________________________ > 從: Barthelemy Dagenais <bar...@in...> > 寄件日期: 2014年12月23日 21:14 > 至: Support and Comments about Py4J > 主旨: Re: [Py4j-users] Exception when printing a returned List<String> object > > Hi, > > When printing the list, Py4J retrieves an Iterator and wraps it with > py4j.java_collections.JavaIterator. It then cycles through the list > with the iterator. JavaIterator should catch the > NoSuchElementException and raise instead a StopIteration. Because it > does not, I suspect something is messing with the automated wrapping > of collections. > > could you do: > > iter = ret.iterator() > print(type(iter)) > > This should print something like <class 'py4j.java_collections.JavaIterator'> > > Thanks, > Barthelemy > > On Tue, Dec 23, 2014 at 6:10 AM, <hg...@it...> wrote: >> Hi, >> >> >> The method in my gateway server will return a List<String> object. When I >> access it in my python client code, the server has exception and then >> closed. The following is part of the exceptions >> >> >> >> >> ▒Q▒G▒▒ 23, 2014 5:18:14 ▒U▒▒ py4j.reflection.MethodInvoker invoke >> ĵ▒i: Exception occurred in client code. >> java.lang.reflect.InvocationTargetException >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> at java.lang.reflect.Method.invoke(Method.java:483) >> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) >> at >> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) >> at py4j.Gateway.invoke(Gateway.java:259) >> at >> py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) >> at py4j.commands.CallCommand.execute(CallCommand.java:79) >> at py4j.GatewayConnection.run(GatewayConnection.java:207) >> at java.lang.Thread.run(Thread.java:745) >> Caused by: java.util.NoSuchElementException >> at java.util.AbstractList$Itr.next(AbstractList.java:364) >> ... 11 more >> >> What I do in client code is the following: >> >>>>>ret = gateway.entry_point.getMyStringList(inputStr) # it's ok to run >>>>> this line >>>>>print ret # when running to this line, the server raised exception >> >> >> Below is the method in server side: >> >> public List<String> getMyStringList(String input) throws IOException { >> List<String> strList = new ArrayList<String>(); >> strList = segmenter.segmentString(input); // this is some 3rd party >> API for string tokenization and has been tested OK in console, the return >> object is List<String> >> >> return strList; >> } >> >> It's OK to access "ret" using indices and len(), but when I just type "ret" >> or "print ret", then it would display the content of "ret", but then the >> server poped Exception and stopped. What would be the problemt? Thanks for >> any help! >> >> >> >> Frank Yang >> >> >> 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 >> This email may contain confidential information. Please do not use or >> disclose it in any way and delete it if you are not the intended recipient. >> ------------------------------------------------------------------------------ >> Dive into the World of Parallel Programming! The Go Parallel Website, >> sponsored by Intel and developed in partnership with Slashdot Media, is your >> hub for all things parallel software development, from weekly thought >> leadership blogs to news, videos, case studies, tutorials and more. Take a >> look and join the conversation now. http://goparallel.sourceforge.net >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users >> > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > > ==================================================================== > 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 > This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users |
From: <hg...@it...> - 2014-12-24 02:57:21
|
Hi, Yes, it prints <class 'py4j.java_collections.JavaIterator'>. So couldn't I directly assign the return List<String> to a python list like the following and then print it? >>>ret = [] >>>ret = gateway.entry_point.getMyStringList(inputStr) >>>print ret If that's the case, for now my workaround for this is below: >>>ans = [ret[i] for i in range(len(ret))] and then use "ans" instead. Could you have any suggestion on this otherwise? Besides, because the exception message only appears partially, how can I log full error messages and debug when the server stops? I think it's especially necessary when the error behavior is in the server side. Thanks, Frank Yang ________________________________________ 從: Barthelemy Dagenais <bar...@in...> 寄件日期: 2014年12月23日 21:14 至: Support and Comments about Py4J 主旨: Re: [Py4j-users] Exception when printing a returned List<String> object Hi, When printing the list, Py4J retrieves an Iterator and wraps it with py4j.java_collections.JavaIterator. It then cycles through the list with the iterator. JavaIterator should catch the NoSuchElementException and raise instead a StopIteration. Because it does not, I suspect something is messing with the automated wrapping of collections. could you do: iter = ret.iterator() print(type(iter)) This should print something like <class 'py4j.java_collections.JavaIterator'> Thanks, Barthelemy On Tue, Dec 23, 2014 at 6:10 AM, <hg...@it...> wrote: > Hi, > > > The method in my gateway server will return a List<String> object. When I > access it in my python client code, the server has exception and then > closed. The following is part of the exceptions > > > > > ▒Q▒G▒▒ 23, 2014 5:18:14 ▒U▒▒ py4j.reflection.MethodInvoker invoke > ĵ▒i: Exception occurred in client code. > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) > at > py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) > at py4j.Gateway.invoke(Gateway.java:259) > at > py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) > at py4j.commands.CallCommand.execute(CallCommand.java:79) > at py4j.GatewayConnection.run(GatewayConnection.java:207) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.NoSuchElementException > at java.util.AbstractList$Itr.next(AbstractList.java:364) > ... 11 more > > What I do in client code is the following: > >>>>ret = gateway.entry_point.getMyStringList(inputStr) # it's ok to run >>>> this line >>>>print ret # when running to this line, the server raised exception > > > Below is the method in server side: > > public List<String> getMyStringList(String input) throws IOException { > List<String> strList = new ArrayList<String>(); > strList = segmenter.segmentString(input); // this is some 3rd party > API for string tokenization and has been tested OK in console, the return > object is List<String> > > return strList; > } > > It's OK to access "ret" using indices and len(), but when I just type "ret" > or "print ret", then it would display the content of "ret", but then the > server poped Exception and stopped. What would be the problemt? Thanks for > any help! > > > > Frank Yang > > > 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 > This email may contain confidential information. Please do not use or > disclose it in any way and delete it if you are not the intended recipient. > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > ------------------------------------------------------------------------------ Dive into the World of Parallel Programming! The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net _______________________________________________ Py4j-users mailing list Py4...@li... https://lists.sourceforge.net/lists/listinfo/py4j-users ==================================================================== 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. |
From: Barthelemy D. <bar...@in...> - 2014-12-23 13:15:12
|
Hi, When printing the list, Py4J retrieves an Iterator and wraps it with py4j.java_collections.JavaIterator. It then cycles through the list with the iterator. JavaIterator should catch the NoSuchElementException and raise instead a StopIteration. Because it does not, I suspect something is messing with the automated wrapping of collections. could you do: iter = ret.iterator() print(type(iter)) This should print something like <class 'py4j.java_collections.JavaIterator'> Thanks, Barthelemy On Tue, Dec 23, 2014 at 6:10 AM, <hg...@it...> wrote: > Hi, > > > The method in my gateway server will return a List<String> object. When I > access it in my python client code, the server has exception and then > closed. The following is part of the exceptions > > > > > ▒Q▒G▒▒ 23, 2014 5:18:14 ▒U▒▒ py4j.reflection.MethodInvoker invoke > ĵ▒i: Exception occurred in client code. > java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) > at > py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) > at py4j.Gateway.invoke(Gateway.java:259) > at > py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) > at py4j.commands.CallCommand.execute(CallCommand.java:79) > at py4j.GatewayConnection.run(GatewayConnection.java:207) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.util.NoSuchElementException > at java.util.AbstractList$Itr.next(AbstractList.java:364) > ... 11 more > > What I do in client code is the following: > >>>>ret = gateway.entry_point.getMyStringList(inputStr) # it's ok to run >>>> this line >>>>print ret # when running to this line, the server raised exception > > > Below is the method in server side: > > public List<String> getMyStringList(String input) throws IOException { > List<String> strList = new ArrayList<String>(); > strList = segmenter.segmentString(input); // this is some 3rd party > API for string tokenization and has been tested OK in console, the return > object is List<String> > > return strList; > } > > It's OK to access "ret" using indices and len(), but when I just type "ret" > or "print ret", then it would display the content of "ret", but then the > server poped Exception and stopped. What would be the problemt? Thanks for > any help! > > > > Frank Yang > > > 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 > This email may contain confidential information. Please do not use or > disclose it in any way and delete it if you are not the intended recipient. > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > |
From: <hg...@it...> - 2014-12-23 11:55:10
|
Hi, The method in my gateway server will return a List<String> object. When I access it in my python client code, the server has exception and then closed. The following is part of the exceptions ▒Q▒G▒▒ 23, 2014 5:18:14 ▒U▒▒ py4j.reflection.MethodInvoker invoke ĵ▒i: Exception occurred in client code. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:483) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379) at py4j.Gateway.invoke(Gateway.java:259) at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133) at py4j.commands.CallCommand.execute(CallCommand.java:79) at py4j.GatewayConnection.run(GatewayConnection.java:207) at java.lang.Thread.run(Thread.java:745) Caused by: java.util.NoSuchElementException at java.util.AbstractList$Itr.next(AbstractList.java:364) ... 11 more What I do in client code is the following: >>>ret = gateway.entry_point.getMyStringList(inputStr) # it's ok to run this line >>>print ret # when running to this line, the server raised exception Below is the method in server side: public List<String> getMyStringList(String input) throws IOException { List<String> strList = new ArrayList<String>(); strList = segmenter.segmentString(input); // this is some 3rd party API for string tokenization and has been tested OK in console, the return object is List<String> return strList; } It's OK to access "ret" using indices and len(), but when I just type "ret" or "print ret", then it would display the content of "ret", but then the server poped Exception and stopped. What would be the problemt? Thanks for any help! Frank Yang ==================================================================== 本信件可能包含工研院機密資訊,非指定之收件者,請勿使用或揭露本信件內容,並請銷毀此信件。 This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient. |
From: Barthelemy D. <bar...@in...> - 2014-12-09 01:27:33
|
Hi, here are a few pointers that may be useful to you: 1. The py4j.java_gateway.launch_gateway enables you to start a JVM and kill it when the Python process dies. 2. It's possible to launch the Java server with port 0 (ephemeral port): this takes the first accessible port. You then need a way to communicate this port to the Python side (launch_gateway uses a pipe) 3. Py4J supports garbage collection: if a python reference to a Java object is garbage collected, the reference to the Java object is released. This uses the weakref.ref(obj, callback) method. You may want to use something like that to know when to stop the Java server. Please note that finalizers, as it is the case with most programming languages, are not guaranteed to be called, but they are good enough for most use cases. Barthelemy On Mon, Dec 8, 2014 at 4:19 PM, Thomas Nyberg <tom...@gm...> wrote: > Great that's a nice way to avoid sending any explicit kill signals. Also > that's probably easy enough for my users to manage. > > What would be really nice is if there could be some sort of garbage > collector hook that would send the shutdown signal automatically. That > way you could maybe create a class that automatically started the server > and then when whatever you're using goes out of scope, your hook could > shutdown the server. I'm sure something like this is theoretically > possible, but I've never worked with that sort of thing. Then maybe on > init one could check if the port is in use (i.e. by another py4j server) > and choose a different port in that case. > > This is all probably overkill though. Your method is great for now. Thanks! > > On 12/08/2014 04:09 PM, Barthelemy Dagenais wrote: >> Hi, >> >> it depends on your use case. Most of the examples start a server in >> the Java main method so if you do gateway.shutdown() on the Python >> side, it will (1) send a shutdown signal on the Java side, (2) the >> Java server will stops, and (3) because there are no more active >> threads, the JVM will exit. >> >> You can even do something harsher from Python: >> >> gateway.jvm.System.exit(0) >> >> Does that help? >> >> Barthelemy >> >> On Mon, Dec 8, 2014 at 3:46 PM, Thomas Nyberg <tom...@gm...> wrote: >>> Hello, >>> >>> What would be the best way for me to start and stop the py4j server from >>> within python? I know I could use >>> >>> > os.system("java ...") >>> >>> to start the server. I guess I would need a subprocess for that not to >>> block. Then to stop it I could send it a kill signal. Is this the best >>> way to do it? Is there a way for me to do this in a less >>> directly-managed fashion? This is okay for me, but the users of what I'm >>> making probably would have trouble... >>> >>> Cheers, >>> Thomas >>> >>> ------------------------------------------------------------------------------ >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >>> with Interactivity, Sharing, Native Excel Exports, App Integration & more >>> Get technology previously reserved for billion-dollar corporations, FREE >>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Py4j-users mailing list >>> Py4...@li... >>> https://lists.sourceforge.net/lists/listinfo/py4j-users >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users >> > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users |
From: Thomas N. <tom...@gm...> - 2014-12-08 21:19:20
|
Great that's a nice way to avoid sending any explicit kill signals. Also that's probably easy enough for my users to manage. What would be really nice is if there could be some sort of garbage collector hook that would send the shutdown signal automatically. That way you could maybe create a class that automatically started the server and then when whatever you're using goes out of scope, your hook could shutdown the server. I'm sure something like this is theoretically possible, but I've never worked with that sort of thing. Then maybe on init one could check if the port is in use (i.e. by another py4j server) and choose a different port in that case. This is all probably overkill though. Your method is great for now. Thanks! On 12/08/2014 04:09 PM, Barthelemy Dagenais wrote: > Hi, > > it depends on your use case. Most of the examples start a server in > the Java main method so if you do gateway.shutdown() on the Python > side, it will (1) send a shutdown signal on the Java side, (2) the > Java server will stops, and (3) because there are no more active > threads, the JVM will exit. > > You can even do something harsher from Python: > > gateway.jvm.System.exit(0) > > Does that help? > > Barthelemy > > On Mon, Dec 8, 2014 at 3:46 PM, Thomas Nyberg <tom...@gm...> wrote: >> Hello, >> >> What would be the best way for me to start and stop the py4j server from >> within python? I know I could use >> >> > os.system("java ...") >> >> to start the server. I guess I would need a subprocess for that not to >> block. Then to stop it I could send it a kill signal. Is this the best >> way to do it? Is there a way for me to do this in a less >> directly-managed fashion? This is okay for me, but the users of what I'm >> making probably would have trouble... >> >> Cheers, >> Thomas >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > |
From: Barthelemy D. <bar...@in...> - 2014-12-08 21:09:20
|
Hi, it depends on your use case. Most of the examples start a server in the Java main method so if you do gateway.shutdown() on the Python side, it will (1) send a shutdown signal on the Java side, (2) the Java server will stops, and (3) because there are no more active threads, the JVM will exit. You can even do something harsher from Python: gateway.jvm.System.exit(0) Does that help? Barthelemy On Mon, Dec 8, 2014 at 3:46 PM, Thomas Nyberg <tom...@gm...> wrote: > Hello, > > What would be the best way for me to start and stop the py4j server from > within python? I know I could use > > > os.system("java ...") > > to start the server. I guess I would need a subprocess for that not to > block. Then to stop it I could send it a kill signal. Is this the best > way to do it? Is there a way for me to do this in a less > directly-managed fashion? This is okay for me, but the users of what I'm > making probably would have trouble... > > Cheers, > Thomas > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users |
From: Thomas N. <tom...@gm...> - 2014-12-08 20:47:02
|
Hello, What would be the best way for me to start and stop the py4j server from within python? I know I could use > os.system("java ...") to start the server. I guess I would need a subprocess for that not to block. Then to stop it I could send it a kill signal. Is this the best way to do it? Is there a way for me to do this in a less directly-managed fashion? This is okay for me, but the users of what I'm making probably would have trouble... Cheers, Thomas |
From: Thomas N. <tom...@gm...> - 2014-12-08 18:45:21
|
(No need to spam the list.) Actually I totally understand. Generally using the command line and knowing how things work "under the hood" is helpful, but it can backfire when almost all people in that programming community use certain IDEs and other tools. I haven't worked with Java enough to ever warrant using an IDE (it drove me crazy not understanding what the IDE was doing for me...). No room for old-fashioned folks like us I guess... Once again thanks a lot! On 12/08/2014 01:31 PM, Barthelemy Dagenais wrote: > Hi, > > glad it worked! Point taken regarding the documentation. I never > thought people would try to compile Java source files from the command > line and most IDEs will properly organize and compile source files, > but this is not the first time I see Py4J users struggling with javac > so I will improve the documentation accordingly. > > For the record, I mostly work in vim and on the CLI except when I have > to work in Java ;-) > > Barthelemy > > On Mon, Dec 8, 2014 at 1:06 PM, listserve <tom...@gm...> wrote: >> Yeah that seems to be the problem. The following works for me: >> >> Stack.java >> ------------------------------------- >> import java.util.LinkedList; >> import java.util.List; >> >> public class Stack { >> private List<String> internalList = new LinkedList<String>(); >> >> public void push(String element) { >> internalList.add(0, element); >> } >> >> public String pop() { >> return internalList.remove(0); >> } >> >> public List<String> getInternalList() { >> return internalList; >> } >> >> public void pushAll(List<String> elements) { >> for (String element : elements) { >> this.push(element); >> } >> } >> } >> ------------------------------------- >> >> StackEntryPoint.java >> ------------------------------------- >> import py4j.GatewayServer; >> >> public class StackEntryPoint { >> >> private Stack stack; >> >> public StackEntryPoint() { >> stack = new Stack(); >> stack.push("Initial Item"); >> } >> >> public Stack getStack() { >> return stack; >> } >> >> public static void main(String[] args) { >> GatewayServer gatewayServer = new GatewayServer(new >> StackEntryPoint()); >> gatewayServer.start(); >> System.out.println("Gateway Server Started"); >> } >> >> } >> ------------------------------------- >> >> I then compiled and ran with: >> >> > export JARFILE=/usr/local/share/py4j/py4j0.8.2.1.jar >> > javac -cp $JARFILE *.java >> > java -cp .:$JARFILE StackEntryPoint >> >> >> Thanks a lot for the package! Now that I'm able to I hope to make use of >> it. :p >> >> Just a little. It might be better to use the code as I wrote it instead >> of the way it currently is in the docs. I've used Java in the past, but >> it's been a while and I could never make sense of the error codes nor >> could I make it work without emailing the list. Searching on the >> internet wasn't really that helpful either because the errors were >> fairly generic. I'm pretty sure other Java neophytes might have similar >> issues since they're probably often like me not actually good at Java, >> but have some java that needs to be wrapped in Python. I think making >> this as brain-dead easy as possible will probably help adoption. >> >> In any case, thanks so much for writing this. I'm excited to make use of it! >> >> Cheers, >> Thomas >> >> On 12/08/2014 12:36 PM, Barthelemy Dagenais wrote: >>> Hi, >>> >>> thanks for using Py4J! >>> >>> I believe the problem might be the package. If you copied/pasted the >>> code exactly, the Java source files need to be in the appropriate >>> directories (e.g., py4j/examples/StackEntryPoint). After compiling the >>> classes, you need to specify the fully qualified name of the main >>> class when invoking java: >>> >>> java -cp .:$JARFILE py4j.examples.StackEntryPoint >>> >>> I suggest though that you just remove the "package py4j.examples;" >>> line in both files. You should then be able to compile and run the >>> example code. >>> >>> Alternatively, you can use the StackEntryPoint class that is provided by py4j: >>> java -cp $JARFILE py4j.examples.StackEntryPoint >>> >>> HTH, >>> Barthelemy >>> >>> On Mon, Dec 8, 2014 at 11:39 AM, listserve <tom...@gm...> wrote: >>>> Hello, >>>> >>>> I'm trying to run the code in the docs from here: >>>> >>>> http://py4j.sourceforge.net/getting_started.html#writing-the-java-program >>>> >>>> I'm not that experienced with Java so I may very well be making a >>>> mistake unrelated to py4j, but I'm unable to run the code regardless. >>>> This is what I've done: I saved the Stack code in a file called >>>> Stack.java and the other code in StackEntryPoint.java. I compiled it at >>>> the command line with the following: >>>> >>>> JARFILE=/usr/local/share/py4j/py4j0.8.2.1.jar >>>> javac -cp $JARFILE *.java >>>> >>>> That gave no error. I then tried to run it with: >>>> >>>> java -cp $JARFILE:. StackEntryPoint >>>> >>>> and I get the following error: >>>> >>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>> StackEntryPoint (wrong name: py4j/examples/StackEntryPoint) >>>> at java.lang.ClassLoader.defineClass1(Native Method) >>>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >>>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>>> at java.security.AccessController.doPrivileged(Native Method) >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >>>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) >>>> >>>> I am running Linux Mint 17 (which is basically Ubuntu) and the following >>>> is my versions of java/javac: >>>> >>>> $ java -version >>>> java version "1.7.0_65" >>>> OpenJDK Runtime Environment (IcedTea 2.5.3) (7u71-2.5.3-0ubuntu0.14.04.1) >>>> OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) >>>> >>>> $ javac -version >>>> javac 1.7.0_65 >>>> >>>> Thanks for any help. >>>> >>>> Cheers, >>>> Thomas >>>> >>>> ------------------------------------------------------------------------------ >>>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >>>> with Interactivity, Sharing, Native Excel Exports, App Integration & more >>>> Get technology previously reserved for billion-dollar corporations, FREE >>>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>>> _______________________________________________ >>>> Py4j-users mailing list >>>> Py4...@li... >>>> https://lists.sourceforge.net/lists/listinfo/py4j-users >>> >>> ------------------------------------------------------------------------------ >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >>> with Interactivity, Sharing, Native Excel Exports, App Integration & more >>> Get technology previously reserved for billion-dollar corporations, FREE >>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Py4j-users mailing list >>> Py4...@li... >>> https://lists.sourceforge.net/lists/listinfo/py4j-users >>> >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > |
From: Barthelemy D. <bar...@in...> - 2014-12-08 18:32:03
|
Hi, glad it worked! Point taken regarding the documentation. I never thought people would try to compile Java source files from the command line and most IDEs will properly organize and compile source files, but this is not the first time I see Py4J users struggling with javac so I will improve the documentation accordingly. For the record, I mostly work in vim and on the CLI except when I have to work in Java ;-) Barthelemy On Mon, Dec 8, 2014 at 1:06 PM, listserve <tom...@gm...> wrote: > Yeah that seems to be the problem. The following works for me: > > Stack.java > ------------------------------------- > import java.util.LinkedList; > import java.util.List; > > public class Stack { > private List<String> internalList = new LinkedList<String>(); > > public void push(String element) { > internalList.add(0, element); > } > > public String pop() { > return internalList.remove(0); > } > > public List<String> getInternalList() { > return internalList; > } > > public void pushAll(List<String> elements) { > for (String element : elements) { > this.push(element); > } > } > } > ------------------------------------- > > StackEntryPoint.java > ------------------------------------- > import py4j.GatewayServer; > > public class StackEntryPoint { > > private Stack stack; > > public StackEntryPoint() { > stack = new Stack(); > stack.push("Initial Item"); > } > > public Stack getStack() { > return stack; > } > > public static void main(String[] args) { > GatewayServer gatewayServer = new GatewayServer(new > StackEntryPoint()); > gatewayServer.start(); > System.out.println("Gateway Server Started"); > } > > } > ------------------------------------- > > I then compiled and ran with: > > > export JARFILE=/usr/local/share/py4j/py4j0.8.2.1.jar > > javac -cp $JARFILE *.java > > java -cp .:$JARFILE StackEntryPoint > > > Thanks a lot for the package! Now that I'm able to I hope to make use of > it. :p > > Just a little. It might be better to use the code as I wrote it instead > of the way it currently is in the docs. I've used Java in the past, but > it's been a while and I could never make sense of the error codes nor > could I make it work without emailing the list. Searching on the > internet wasn't really that helpful either because the errors were > fairly generic. I'm pretty sure other Java neophytes might have similar > issues since they're probably often like me not actually good at Java, > but have some java that needs to be wrapped in Python. I think making > this as brain-dead easy as possible will probably help adoption. > > In any case, thanks so much for writing this. I'm excited to make use of it! > > Cheers, > Thomas > > On 12/08/2014 12:36 PM, Barthelemy Dagenais wrote: >> Hi, >> >> thanks for using Py4J! >> >> I believe the problem might be the package. If you copied/pasted the >> code exactly, the Java source files need to be in the appropriate >> directories (e.g., py4j/examples/StackEntryPoint). After compiling the >> classes, you need to specify the fully qualified name of the main >> class when invoking java: >> >> java -cp .:$JARFILE py4j.examples.StackEntryPoint >> >> I suggest though that you just remove the "package py4j.examples;" >> line in both files. You should then be able to compile and run the >> example code. >> >> Alternatively, you can use the StackEntryPoint class that is provided by py4j: >> java -cp $JARFILE py4j.examples.StackEntryPoint >> >> HTH, >> Barthelemy >> >> On Mon, Dec 8, 2014 at 11:39 AM, listserve <tom...@gm...> wrote: >>> Hello, >>> >>> I'm trying to run the code in the docs from here: >>> >>> http://py4j.sourceforge.net/getting_started.html#writing-the-java-program >>> >>> I'm not that experienced with Java so I may very well be making a >>> mistake unrelated to py4j, but I'm unable to run the code regardless. >>> This is what I've done: I saved the Stack code in a file called >>> Stack.java and the other code in StackEntryPoint.java. I compiled it at >>> the command line with the following: >>> >>> JARFILE=/usr/local/share/py4j/py4j0.8.2.1.jar >>> javac -cp $JARFILE *.java >>> >>> That gave no error. I then tried to run it with: >>> >>> java -cp $JARFILE:. StackEntryPoint >>> >>> and I get the following error: >>> >>> Exception in thread "main" java.lang.NoClassDefFoundError: >>> StackEntryPoint (wrong name: py4j/examples/StackEntryPoint) >>> at java.lang.ClassLoader.defineClass1(Native Method) >>> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >>> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >>> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >>> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >>> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) >>> >>> I am running Linux Mint 17 (which is basically Ubuntu) and the following >>> is my versions of java/javac: >>> >>> $ java -version >>> java version "1.7.0_65" >>> OpenJDK Runtime Environment (IcedTea 2.5.3) (7u71-2.5.3-0ubuntu0.14.04.1) >>> OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) >>> >>> $ javac -version >>> javac 1.7.0_65 >>> >>> Thanks for any help. >>> >>> Cheers, >>> Thomas >>> >>> ------------------------------------------------------------------------------ >>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >>> with Interactivity, Sharing, Native Excel Exports, App Integration & more >>> Get technology previously reserved for billion-dollar corporations, FREE >>> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >>> _______________________________________________ >>> Py4j-users mailing list >>> Py4...@li... >>> https://lists.sourceforge.net/lists/listinfo/py4j-users >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users >> > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users |
From: listserve <tom...@gm...> - 2014-12-08 18:07:02
|
Yeah that seems to be the problem. The following works for me: Stack.java ------------------------------------- import java.util.LinkedList; import java.util.List; public class Stack { private List<String> internalList = new LinkedList<String>(); public void push(String element) { internalList.add(0, element); } public String pop() { return internalList.remove(0); } public List<String> getInternalList() { return internalList; } public void pushAll(List<String> elements) { for (String element : elements) { this.push(element); } } } ------------------------------------- StackEntryPoint.java ------------------------------------- import py4j.GatewayServer; public class StackEntryPoint { private Stack stack; public StackEntryPoint() { stack = new Stack(); stack.push("Initial Item"); } public Stack getStack() { return stack; } public static void main(String[] args) { GatewayServer gatewayServer = new GatewayServer(new StackEntryPoint()); gatewayServer.start(); System.out.println("Gateway Server Started"); } } ------------------------------------- I then compiled and ran with: > export JARFILE=/usr/local/share/py4j/py4j0.8.2.1.jar > javac -cp $JARFILE *.java > java -cp .:$JARFILE StackEntryPoint Thanks a lot for the package! Now that I'm able to I hope to make use of it. :p Just a little. It might be better to use the code as I wrote it instead of the way it currently is in the docs. I've used Java in the past, but it's been a while and I could never make sense of the error codes nor could I make it work without emailing the list. Searching on the internet wasn't really that helpful either because the errors were fairly generic. I'm pretty sure other Java neophytes might have similar issues since they're probably often like me not actually good at Java, but have some java that needs to be wrapped in Python. I think making this as brain-dead easy as possible will probably help adoption. In any case, thanks so much for writing this. I'm excited to make use of it! Cheers, Thomas On 12/08/2014 12:36 PM, Barthelemy Dagenais wrote: > Hi, > > thanks for using Py4J! > > I believe the problem might be the package. If you copied/pasted the > code exactly, the Java source files need to be in the appropriate > directories (e.g., py4j/examples/StackEntryPoint). After compiling the > classes, you need to specify the fully qualified name of the main > class when invoking java: > > java -cp .:$JARFILE py4j.examples.StackEntryPoint > > I suggest though that you just remove the "package py4j.examples;" > line in both files. You should then be able to compile and run the > example code. > > Alternatively, you can use the StackEntryPoint class that is provided by py4j: > java -cp $JARFILE py4j.examples.StackEntryPoint > > HTH, > Barthelemy > > On Mon, Dec 8, 2014 at 11:39 AM, listserve <tom...@gm...> wrote: >> Hello, >> >> I'm trying to run the code in the docs from here: >> >> http://py4j.sourceforge.net/getting_started.html#writing-the-java-program >> >> I'm not that experienced with Java so I may very well be making a >> mistake unrelated to py4j, but I'm unable to run the code regardless. >> This is what I've done: I saved the Stack code in a file called >> Stack.java and the other code in StackEntryPoint.java. I compiled it at >> the command line with the following: >> >> JARFILE=/usr/local/share/py4j/py4j0.8.2.1.jar >> javac -cp $JARFILE *.java >> >> That gave no error. I then tried to run it with: >> >> java -cp $JARFILE:. StackEntryPoint >> >> and I get the following error: >> >> Exception in thread "main" java.lang.NoClassDefFoundError: >> StackEntryPoint (wrong name: py4j/examples/StackEntryPoint) >> at java.lang.ClassLoader.defineClass1(Native Method) >> at java.lang.ClassLoader.defineClass(ClassLoader.java:800) >> at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) >> at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) >> at java.net.URLClassLoader.access$100(URLClassLoader.java:71) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:361) >> at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:425) >> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:358) >> at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) >> >> I am running Linux Mint 17 (which is basically Ubuntu) and the following >> is my versions of java/javac: >> >> $ java -version >> java version "1.7.0_65" >> OpenJDK Runtime Environment (IcedTea 2.5.3) (7u71-2.5.3-0ubuntu0.14.04.1) >> OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) >> >> $ javac -version >> javac 1.7.0_65 >> >> Thanks for any help. >> >> Cheers, >> Thomas >> >> ------------------------------------------------------------------------------ >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards >> with Interactivity, Sharing, Native Excel Exports, App Integration & more >> Get technology previously reserved for billion-dollar corporations, FREE >> http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk >> _______________________________________________ >> Py4j-users mailing list >> Py4...@li... >> https://lists.sourceforge.net/lists/listinfo/py4j-users > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users > |
From: Barthelemy D. <bar...@in...> - 2014-12-08 17:36:49
|
Hi, thanks for using Py4J! I believe the problem might be the package. If you copied/pasted the code exactly, the Java source files need to be in the appropriate directories (e.g., py4j/examples/StackEntryPoint). After compiling the classes, you need to specify the fully qualified name of the main class when invoking java: java -cp .:$JARFILE py4j.examples.StackEntryPoint I suggest though that you just remove the "package py4j.examples;" line in both files. You should then be able to compile and run the example code. Alternatively, you can use the StackEntryPoint class that is provided by py4j: java -cp $JARFILE py4j.examples.StackEntryPoint HTH, Barthelemy On Mon, Dec 8, 2014 at 11:39 AM, listserve <tom...@gm...> wrote: > Hello, > > I'm trying to run the code in the docs from here: > > http://py4j.sourceforge.net/getting_started.html#writing-the-java-program > > I'm not that experienced with Java so I may very well be making a > mistake unrelated to py4j, but I'm unable to run the code regardless. > This is what I've done: I saved the Stack code in a file called > Stack.java and the other code in StackEntryPoint.java. I compiled it at > the command line with the following: > > JARFILE=/usr/local/share/py4j/py4j0.8.2.1.jar > javac -cp $JARFILE *.java > > That gave no error. I then tried to run it with: > > java -cp $JARFILE:. StackEntryPoint > > and I get the following error: > > Exception in thread "main" java.lang.NoClassDefFoundError: > StackEntryPoint (wrong name: py4j/examples/StackEntryPoint) > at java.lang.ClassLoader.defineClass1(Native Method) > at java.lang.ClassLoader.defineClass(ClassLoader.java:800) > at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) > at java.net.URLClassLoader.defineClass(URLClassLoader.java:449) > at java.net.URLClassLoader.access$100(URLClassLoader.java:71) > at java.net.URLClassLoader$1.run(URLClassLoader.java:361) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:425) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:358) > at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:482) > > I am running Linux Mint 17 (which is basically Ubuntu) and the following > is my versions of java/javac: > > $ java -version > java version "1.7.0_65" > OpenJDK Runtime Environment (IcedTea 2.5.3) (7u71-2.5.3-0ubuntu0.14.04.1) > OpenJDK 64-Bit Server VM (build 24.65-b04, mixed mode) > > $ javac -version > javac 1.7.0_65 > > Thanks for any help. > > Cheers, > Thomas > > ------------------------------------------------------------------------------ > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server > from Actuate! Instantly Supercharge Your Business Reports and Dashboards > with Interactivity, Sharing, Native Excel Exports, App Integration & more > Get technology previously reserved for billion-dollar corporations, FREE > http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk > _______________________________________________ > Py4j-users mailing list > Py4...@li... > https://lists.sourceforge.net/lists/listinfo/py4j-users |