Re: [Py4j-users] 回覆: 回覆: Exception when printing a returned List<String> object
Status: Beta
Brought to you by:
barthe
Menu
▾
▴
Re: [Py4j-users] 回覆: 回覆: Exception when printing a returned List<String> object
|
From: Barthelemy D. <bar...@in...> - 2014-12-30 16:35:29
|
Hi,
when printing a Java List in Python, Py4J retrieves an iterator and
iterates over the entire list to print the individual elements.
NoSuchElementException is thrown by Java once the iterator has reached
the end of the list (this is the way Java iterators work). Py4J then
catches this exception on the Python side and raises a StopIteration
error which will stop the list iteration.
The exception you see on the Java side is normal. What I am trying to
assess is what happens on the Python side. From what you said, it
seems the list is not printed and the gateway shuts down. Is this
correct? If so, this is strange. Enabling logging on the Python side
might help:
import logging
logger = logging.getLogger("py4j")
logger.setLevel(logging.DEBUG)
logger.addHandler(logging.StreamHandler())
Regarding the Java logging, the FAQ already provides two working ways
to enable logging. You made a small mistake when you tried to set the
log level, it should be:
logger.setLevel(logging.Level.ALL)
I believe you may have mixed the instructions on enabling logging in
Java and Python :-)
This is really a strange bug: there are many unit tests testing this
specific use case (retrieving a string representation of a Java list).
I will try to run the tests in cygwin/windows later this week.
Cheers,
Barthelemy
On Mon, Dec 29, 2014 at 10:16 PM, <hg...@it...> wrote:
> Hi,
>
> (A) exception from accessing list
>
> The versions are here:
> py4j 0.8.2.1
> python 2.7.8
> Java 1.8.0_20
>
> I open two cygwin consoles. One for running the Gateway Server, and the other is to run python script.
>
> When I run the python script to print an ArrayList as your advice, the same exception raised in the server side. Part of the trace stack is as below:
> ...
> Caused by: java.util.NoSuchElementException
> at java.util.ArrayList$Itr.next(ArrayList.java:854)
>
> It's OK to access the list individually by indices, but it causes exception when directly typing the "a_list" or "print(a_ list)".
>
> When the list is accessed as a whole, what would the gateway server do the list? It seems the server tries to iterate to elements that don't exist.
>
> (B) exception from setting logging in python script
>
> I follow your faq page of how to turn logging on-off and get the following exception:
>
>>>> logging = gw.jvm.java.util.logging
>>>> logger = logging.Logger.getLogger("py4j")
>>>> logger.setLevel(logging.DEBUG)
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/usr/lib/python2.7/site-packages/py4j/java_gateway.py", line 529, in __call__
> [get_command_part(arg, self.pool) for arg in new_args])
> File "/usr/lib/python2.7/site-packages/py4j/protocol.py", line 242, in get_command_part
> if parameter == None:
> File "/usr/lib/python2.7/site-packages/py4j/java_gateway.py", line 726, in __getattr__
> raise Py4JError('Trying to call a package.')
> py4j.protocol.Py4JError: Trying to call a package.
>
> Maybe faq page needs to add more information on this situation.
>
> Frank Yang
>
>
> ________________________________________
> 從: Barthelemy Dagenais <bar...@in...>
> 寄件日期: 2014年12月28日 04:42
> 至: Support and Comments about Py4J
> 主旨: Re: [Py4j-users] 回覆: Exception when printing a returned List<String> object
>
> Hi,
>
> apologies for the delays :-)
>
> The behavior you are seeing is strange.
>
> Here are a few pointers:
>
> a) Error Logging can be turned on and off on both sides:
> http://py4j.sourceforge.net/faq.html#how-to-turn-logging-on-off
>
> b) Which version of py4j, python, and Java are you using?
>
> c) Are you able to print a standard list, e.g., ArrayList? e.g.,
>
> a_list = gateway.jvm.java.util.ArrayList()
> a_list.append("a")
> a_list.append("b")
> print(a_list)
>
> d) The exceptions you shown seems to come from the JVM. Do you see any
> exception on the Python side?
>
> Regards,
> Barthelemy
>
> On Tue, Dec 23, 2014 at 9:56 PM, <hg...@it...> wrote:
>> Hi,
>>
>> Yes, it prints <class 'py4j.java_collections.JavaIterator'>.
>> So couldn't I directly assign the return List<String> to a python list like the following and then print it?
>>
>>>>>ret = []
>>>>>ret = gateway.entry_point.getMyStringList(inputStr)
>>>>>print ret
>>
>> If that's the case, for now my workaround for this is below:
>>
>>>>>ans = [ret[i] for i in range(len(ret))]
>>
>> and then use "ans" instead.
>>
>> Could you have any suggestion on this otherwise?
>>
>> Besides, because the exception message only appears partially, how can I log full error messages and debug when the server stops? I think it's especially necessary when the error behavior is in the server side.
>>
>> Thanks,
>> Frank Yang
>> ________________________________________
>> 從: Barthelemy Dagenais <bar...@in...>
>> 寄件日期: 2014年12月23日 21:14
>> 至: Support and Comments about Py4J
>> 主旨: Re: [Py4j-users] Exception when printing a returned List<String> object
>>
>> Hi,
>>
>> When printing the list, Py4J retrieves an Iterator and wraps it with
>> py4j.java_collections.JavaIterator. It then cycles through the list
>> with the iterator. JavaIterator should catch the
>> NoSuchElementException and raise instead a StopIteration. Because it
>> does not, I suspect something is messing with the automated wrapping
>> of collections.
>>
>> could you do:
>>
>> iter = ret.iterator()
>> print(type(iter))
>>
>> This should print something like <class 'py4j.java_collections.JavaIterator'>
>>
>> Thanks,
>> Barthelemy
>>
>> On Tue, Dec 23, 2014 at 6:10 AM, <hg...@it...> wrote:
>>> Hi,
>>>
>>>
>>> The method in my gateway server will return a List<String> object. When I
>>> access it in my python client code, the server has exception and then
>>> closed. The following is part of the exceptions
>>>
>>>
>>>
>>>
>>> ▒Q▒G▒▒ 23, 2014 5:18:14 ▒U▒▒ py4j.reflection.MethodInvoker invoke
>>> ĵ▒i: Exception occurred in client code.
>>> java.lang.reflect.InvocationTargetException
>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>> at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>> at java.lang.reflect.Method.invoke(Method.java:483)
>>> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
>>> at
>>> py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:379)
>>> at py4j.Gateway.invoke(Gateway.java:259)
>>> at
>>> py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
>>> at py4j.commands.CallCommand.execute(CallCommand.java:79)
>>> at py4j.GatewayConnection.run(GatewayConnection.java:207)
>>> at java.lang.Thread.run(Thread.java:745)
>>> Caused by: java.util.NoSuchElementException
>>> at java.util.AbstractList$Itr.next(AbstractList.java:364)
>>> ... 11 more
>>>
>>> What I do in client code is the following:
>>>
>>>>>>ret = gateway.entry_point.getMyStringList(inputStr) # it's ok to run
>>>>>> this line
>>>>>>print ret # when running to this line, the server raised exception
>>>
>>>
>>> Below is the method in server side:
>>>
>>> public List<String> getMyStringList(String input) throws IOException {
>>> List<String> strList = new ArrayList<String>();
>>> strList = segmenter.segmentString(input); // this is some 3rd party
>>> API for string tokenization and has been tested OK in console, the return
>>> object is List<String>
>>>
>>> return strList;
>>> }
>>>
>>> It's OK to access "ret" using indices and len(), but when I just type "ret"
>>> or "print ret", then it would display the content of "ret", but then the
>>> server poped Exception and stopped. What would be the problemt? Thanks for
>>> any help!
>>>
>>>
>>>
>>> Frank Yang
>>>
>>>
>>> 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。
>>> This email may contain confidential information. Please do not use or
>>> disclose it in any way and delete it if you are not the intended recipient.
>>> ------------------------------------------------------------------------------
>>> Dive into the World of Parallel Programming! The Go Parallel Website,
>>> sponsored by Intel and developed in partnership with Slashdot Media, is your
>>> hub for all things parallel software development, from weekly thought
>>> leadership blogs to news, videos, case studies, tutorials and more. Take a
>>> look and join the conversation now. http://goparallel.sourceforge.net
>>> _______________________________________________
>>> Py4j-users mailing list
>>> Py4...@li...
>>> https://lists.sourceforge.net/lists/listinfo/py4j-users
>>>
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming! The Go Parallel Website,
>> sponsored by Intel and developed in partnership with Slashdot Media, is your
>> hub for all things parallel software development, from weekly thought
>> leadership blogs to news, videos, case studies, tutorials and more. Take a
>> look and join the conversation now. http://goparallel.sourceforge.net
>> _______________________________________________
>> Py4j-users mailing list
>> Py4...@li...
>> https://lists.sourceforge.net/lists/listinfo/py4j-users
>>
>> ====================================================================
>> 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。
>> This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient.
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming! The Go Parallel Website,
>> sponsored by Intel and developed in partnership with Slashdot Media, is your
>> hub for all things parallel software development, from weekly thought
>> leadership blogs to news, videos, case studies, tutorials and more. Take a
>> look and join the conversation now. http://goparallel.sourceforge.net
>> _______________________________________________
>> Py4j-users mailing list
>> Py4...@li...
>> https://lists.sourceforge.net/lists/listinfo/py4j-users
>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming! The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. Take a
> look and join the conversation now. http://goparallel.sourceforge.net
> _______________________________________________
> Py4j-users mailing list
> Py4...@li...
> https://lists.sourceforge.net/lists/listinfo/py4j-users
>
>
> ====================================================================
> 本信件可能包含工研院機密資訊，非指定之收件者，請勿使用或揭露本信件內容，並請銷毀此信件。
> This email may contain confidential information. Please do not use or disclose it in any way and delete it if you are not the intended recipient.
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming! The Go Parallel Website,
> sponsored by Intel and developed in partnership with Slashdot Media, is your
> hub for all things parallel software development, from weekly thought
> leadership blogs to news, videos, case studies, tutorials and more. Take a
> look and join the conversation now. http://goparallel.sourceforge.net
> _______________________________________________
> Py4j-users mailing list
> Py4...@li...
> https://lists.sourceforge.net/lists/listinfo/py4j-users
|
