jython-dev — Discussion of developing and enhancing Jython itself(!) (not how to use Jython)

Re: [Jython-dev] Help with parsing

[<bc...@wo...>]

[Javier Fradiletti]

>Hi!
>I'm triying to parse an expression in jython sintaxis and with the result
>tree do something.
>I'm having some troubles when i try to get the information in the tree
>nodes.
>I have to know the 'structure' of the tree (an if statment with a
>condition -and have acces to that condition-) to represent it in a graphic
>interfase.
>There is not much java api documentation in the jython site, so i really
>need your help.

I don't quite understand what you mean. Is your "result tree" the syntax
tree create by the org.python.parser.* classes?

I'll assume that it is, because I don't know of any other way tree
representaions of jython expressions. You can investigate the actual
tree structure with the SimpleNode.dump(..) method:

    from java import io
    from org.python.core import parser

    istream = io.ByteArrayInputStream("if 1: print 'true'")
    tree = parser.parse(istream, "exec", "dummy");

    tree.dump("if-stmt:")

The language grammar in jython.jjt is also a necessary tool when
understanding the syntax tree.

Keep in mind that the syntax tree is an internal feature of the python
language and that the tree may change between releases.

regards,
finn

[Jython-dev] Help with parsing

[Javier F. <je...@ad...>]

Hi!
I'm triying to parse an expression in jython sintaxis and with the result
tree do something.
I'm having some troubles when i try to get the information in the tree
nodes.
I have to know the 'structure' of the tree (an if statment with a
condition -and have acces to that condition-) to represent it in a graphic
interfase.
There is not much java api documentation in the jython site, so i really
need your help.

Best regards and awaiting your answer,

Javier Fradiletti
je...@ad...

Re: [Jython-dev] poor man freezing and loading from other sources: proposals and ideas

[Samuele P. <pe...@in...>]

Hi.

[Finn]
> >> Maybe a different worded ImportError should be printed when
> >> sys.classLoader is set.
> >I can hear them too <wink>, once I imagined the following ?ugly? solution
> >(but no import * or dir) that technically means having the packageManager
> >existsPacakage return always true <wink>. Not a good default but as special
> >option ...
> 
> I suppose it would silence a lot of the valid complaints. It would make
> debugging a erroneous setup more difficult and it feels somewhat
> unpythonic: "Errors should never pass silently".
> 
> I can't think of a good way of enabling this right now, but as a special
> option it is worth remembering.
There is no good way <wink>, if I remember well my code and the rules it 
enforces it is a matter of ~5 (a bit more with changes to Options) lines in the 
right place. We can even still use the gathered information to selectively issue 
warnings:
" java package 'foo' forced, no dir or import * support, could even not
exist ;)"
But yes it should be a special option, in any case  people could then complain 
about import * and dir failing:

 "You cannot stop people from complaining, but you can influence what
they complain about." - Tim Peters

> 
> >> >To be honest this one  is the more complicated of the proposed changes,
> >> >here some concrete ideas:
> >> >
> >> >0. have a tool to "index" a jar in a way that supports dir (maybe the same
> >> >  as above under different options)
> >> 
> >> I believe that a .jar should only be scanned once. That means that if
> >> the scan result isn't cached we should reject un-indexed jars. Why don't
> >> you want to cache the scan result?
> >> 
> >> All in all this is my favorite.
> >Having the tool? caching things?
> 
> I don't mind caching the scan result, but if we decide that we shouldn't
> cache it, a tool that stores the scan result inside the jar is just as
> good IMO.
> 
> Rescanning for every session is wrong. I can't remember any complains
> about the initial scanning of jars but it is unacceptable to do it every
> time.
> ...
I will try all the possibilities and see the differences. (*)

> 
> >> >c. the overkill solution [maybe not that much but implies that we must 
live
> >> >   forever with the following CPython incompatibility (which is quite 
marginal):
> >> >      L=UserList.UserList()
> >> >      sys.path=L
> >> >   works in CPython but not in Jython, for c) we will have to even enforce 
this]
> >> >   add to builtin lists the possibility to have an observer (it costs
> >> >   the comparision between a ref and a null on every update method)
> >> >   that is notified for changes, so we can open, scan and close things
> >> >   on sys.path changes. Why every list? to make rare things work:
> >> >      L=['.','foodir']
> >> >      sys.path=L
> >> >      L.append('bazdir')
> >> >   When a list is assigned to sys.path its observer field is assigned,
> >> >   the field of the old one is reset...
> >> 
> >...
> >> 
> >> Or wrap the list 'L' above with an observable wrapper that intercepts
> >> __XXXitem__ calls. That would close the archive immeditatly but cause
> >> the value returned by 'sys.path' to be different from the value
> >> assigned. My favorite solution.
> >Ok for me but the example above that modifies sys.path trough L will not work 
(?)
> 
> ohh right. No it won't work. My bad.
Sorry, this is too much emotionally and with irony overloaded.
Do you mean you were aware? we both are.
In any case if it is ok for you to live with this problem, it is ok for me
too. But that's a point where we need a decision...
It is still your favorite? with some logic to deal with "off-line" changes
it should be fine up to the problem.

Otherwise there is the overkill solution or keeping them just open, which
maybe is not that bad.

<+
For the poor man freezing case we need a tool anyway  (*),
I think that a resource org/python/inf/frozen.list could be ok (?)
for storing the collected info.

regards.

Re: [Jython-dev] poor man freezing and loading from other sources: proposals and ideas

[<bc...@wo...>]

On Wed, 21 Mar 2001 15:44:02 +0100 (MET), you wrote:

>> ... It's just that I can already hear the
>> complaints that jython can't load java class: "I know the classloader
>> works because I can load classes from java". It is painfull that the
>> complaint is actually valid.
>> 
>> Maybe a different worded ImportError should be printed when
>> sys.classLoader is set.
>I can hear them too <wink>, once I imagined the following ?ugly? solution
>(but no import * or dir) that technically means having the packageManager
>existsPacakage return always true <wink>. Not a good default but as special
>option ...

I suppose it would silence a lot of the valid complaints. It would make
debugging a erroneous setup more difficult and it feels somewhat
unpythonic: "Errors should never pass silently".

I can't think of a good way of enabling this right now, but as a special
option it is worth remembering.

>> >To be honest this one  is the more complicated of the proposed changes,
>> >here some concrete ideas:
>> >
>> >0. have a tool to "index" a jar in a way that supports dir (maybe the same
>> >  as above under different options)
>> 
>> I believe that a .jar should only be scanned once. That means that if
>> the scan result isn't cached we should reject un-indexed jars. Why don't
>> you want to cache the scan result?
>> 
>> All in all this is my favorite.
>Having the tool? caching things?

I don't mind caching the scan result, but if we decide that we shouldn't
cache it, a tool that stores the scan result inside the jar is just as
good IMO.

Rescanning for every session is wrong. I can't remember any complains
about the initial scanning of jars but it is unacceptable to do it every
time.

>I would cache things per session but not across sessions. The problem is that
>we never cleanup the global cache and this is absolute-path based, for trying
>an api once that then is maybe discarded this is bad, for things moving around
>the file-system is bad too, it's ok for jdk libraries or extension or ...

I just checked my cachedir/packages directory. It only contained 290Kb.
Even the 13Mb rt.jar only takes up 45Kb of index. I don't mind that the
cache can accumulate old crud. A user can delete it whenever she wants
without severe side effects.

>> >c. the overkill solution [maybe not that much but implies that we must live
>> >   forever with the following CPython incompatibility (which is quite marginal):
>> >      L=UserList.UserList()
>> >      sys.path=L
>> >   works in CPython but not in Jython, for c) we will have to even enforce this]
>> >   add to builtin lists the possibility to have an observer (it costs
>> >   the comparision between a ref and a null on every update method)
>> >   that is notified for changes, so we can open, scan and close things
>> >   on sys.path changes. Why every list? to make rare things work:
>> >      L=['.','foodir']
>> >      sys.path=L
>> >      L.append('bazdir')
>> >   When a list is assigned to sys.path its observer field is assigned,
>> >   the field of the old one is reset...
>> 
>...
>> 
>> Or wrap the list 'L' above with an observable wrapper that intercepts
>> __XXXitem__ calls. That would close the archive immeditatly but cause
>> the value returned by 'sys.path' to be different from the value
>> assigned. My favorite solution.
>Ok for me but the example above that modifies sys.path trough L will not work (?)

ohh right. No it won't work. My bad.

regards,
finn

[Jython-dev] Problem loading main class manifest attribute

[lambert <ta...@si...>]

Dear Sir/Madam,
    Sorry to trouble you. I have encountered a problem in starting =
JPython.
The message that it return is "Failed to load Main-Class manifest =
attribute from jython.jar.
Can you please kindly tell me what went wrong.
Your Sincerely,
Lambert

Re: [Jython-dev] poor man freezing and loading from other sources: proposals and ideas

[Samuele P. <pe...@in...>]

Hi.

We should agree on where to put the result of our pre-scanning both
for sys.path jars and poor man freezing

use sun META-INF? don't know, in any case it is a bad location for poor man freezing info,
could make sense for sys.path jars, but I imagine sun consider that space reserved
only for its needs;

org/python/inf or something similar?

regards

Re: [Jython-dev] poor man freezing and loading from other sources: proposals and ideas

[Samuele P. <pe...@in...>]

Hi.

[Finn]
> [Samuele]
> 
...
> 
> Yeah.
> 
> >On the other hand even if the only solution is to define packages
> >through sys.add_* apis, I think that opening the sys.classLoader API
> >and making imports even more regular is something necessary for many embedding 
> >related situations, in any case it is something not for the casual user, the 
> >others can pay the necessary attention and lack of comfort...
> 
> Yes, at least in principle. It's just that I can already hear the
> complaints that jython can't load java class: "I know the classloader
> works because I can load classes from java". It is painfull that the
> complaint is actually valid.
> 
> Maybe a different worded ImportError should be printed when
> sys.classLoader is set.
I can hear them too <wink>, once I imagined the following ?ugly? solution
(but no import * or dir) that technically means having the packageManager
existsPacakage return always true <wink>. Not a good default but as special
option ...
 
> ...
> I think listeners are less likely to be edited, and when they are
> changed, the loading/linking of the adapter will most likely fail.
> 
> Proxies are much more fragile since changes to both the java class and
> the python subclass can/should force a new proxy class to be generated.
> But to avoid two $py.class formats, we could always compile in the proxy
> classname and only use the name when running as a poor-man frozen
> application.
I see, it's a matter of coding that <wink>

> ...
> >Unfortunaly jar -i does not store enough information to support
> >dir and import *. In any case we would have to build a tool for
> >non java SE2 1.3 env (?).
> 
> Yes. I called this tool of ours for "jarindex". It would be a shell/.bat
> script written by the installer.
Ok.

> >To be honest this one  is the more complicated of the proposed changes,
> >here some concrete ideas:
> >
> >0. have a tool to "index" a jar in a way that supports dir (maybe the same
> >  as above under different options)
> 
> I believe that a .jar should only be scanned once. That means that if
> the scan result isn't cached we should reject un-indexed jars. Why don't
> you want to cache the scan result?
> 
> All in all this is my favorite.
Having the tool? caching things?

I would cache things per session but not across sessions. The problem is that
we never cleanup the global cache and this is absolute-path based, for trying
an api once that then is maybe discarded this is bad, for things moving around
the file-system is bad too, it's ok for jdk libraries or extension or ...

> >a. ignore performance issues, open and close jars on every import, scan
> > or read the index also lazily on import
> 
> There shouldn't be noticeable performance difference between putting a
> jar on sys.path or on classpath. I think we have to keep it open.
I agree.

> >b. like a) but keep jars open, but I don't see any good place to lazily
> >   close them that work at least in many situations if not for all.
> >   Just keeping them open sounds bad.
> 
> That is what the JVM does for the .zip files found on CLASSPATH. As for
> changes to sys.path se below.
Aware of that.


> >c. the overkill solution [maybe not that much but implies that we must live
> >   forever with the following CPython incompatibility (which is quite marginal):
> >      L=UserList.UserList()
> >      sys.path=L
> >   works in CPython but not in Jython, for c) we will have to even enforce this]
> >   add to builtin lists the possibility to have an observer (it costs
> >   the comparision between a ref and a null on every update method)
> >   that is notified for changes, so we can open, scan and close things
> >   on sys.path changes. Why every list? to make rare things work:
> >      L=['.','foodir']
> >      sys.path=L
> >      L.append('bazdir')
> >   When a list is assigned to sys.path its observer field is assigned,
> >   the field of the old one is reset...
> 
...
> 
> Or wrap the list 'L' above with an observable wrapper that intercepts
> __XXXitem__ calls. That would close the archive immeditatly but cause
> the value returned by 'sys.path' to be different from the value
> assigned. My favorite solution.
Ok for me but the example above that modifies sys.path trough L will not work (?)

and we still have to deal with unexpected modifications.

> I suppose that putting the zipfile instance itself into sys.path would
> solve the close problem. Until CPython decides to go the full length, I
> would prefer a less neat solution where we keep only strings in
> sys.path.
Yes.

regards.

Re: [Jython-dev] poor man freezing and loading from other sources: proposals and ideas

[<bc...@wo...>]

[Samuele]

>Hi.
>
>[Finn]
>> >Please, no pure "it would be nice if" requests.
>> 
>> But it would be so nice, if we had all this <wink>.
>Maybe not all in time for 2.1, but in principle <wink>.
>Let's get a bit concrete.
>
>> Opening the sys.classLoader API would be a good thing, but it also opens
>> the question of "scanning" the sys.classLoader for java packages. We
>> could perhaps make it a requirement that sys.classLoader.getPackages()
>> should return something usefull. Unfortunately I think most custom
>> classloaders ignore the package support completely.
>In any case getPackages() is protected, and my interpretation of the docs
>is that even a good-willing classloader can limit itself to define packages 
>lazily only when classes are loaded from them.

Yeah.

>On the other hand even if the only solution is to define packages
>through sys.add_* apis, I think that opening the sys.classLoader API
>and making imports even more regular is something necessary for many embedding 
>related situations, in any case it is something not for the casual user, the 
>others can pay the necessary attention and lack of comfort...

Yes, at least in principle. It's just that I can already hear the
complaints that jython can't load java class: "I know the classloader
works because I can load classes from java". It is painfull that the
complaint is actually valid.

Maybe a different worded ImportError should be printed when
sys.classLoader is set.

>> > iv) <poor man freezing>
>...
>> > idea: have a tool that take a jar or a set of directories
>> > and collect which java packages-classes/python packages-modules are defined
>> > and produce a report in a textual format (for maximal flexibility).
>> >
>> > This report can be put in the jar and be read (impl/expl) by jython runtime
>> > as a _resource_  in    
>> > order to initiliaze its internal tables for special and java loading (see 
>*)).
>> 
>> A tool sound fine to me. Having such an additional build step is truly a
>> minor thing.
>> 
>> > open issues: $py.class files still require that proxies and adapters
>> > are built at runtime, so classloader creation permission is required,
>> > this should not be an issue for server side deployment or trusted
>> > applets, for normal applets jythonc solve this. Is this a serious 
>> > problem?
>> 
>> I'm sure it depends on the deployment environment, whether or not it is
>> a problem <wink>.
>> 
>> An idea could be a runtime option that caused proxies and adapters to be
>> saved in user specified directory. It would also cause the $py.class
>> file to be rewritten with a compiled-in proxy reference. That could
>> perhaps maintain the illusion that we don't have to pre-compile the
>> application.
>There is something in that direction (at least for adapters) for debugging 
>purpose. I think what you propose is doable. Maybe we can even
>avoid to put explicit references (and have 2 $py.class formats),
>at least for adapters the actual code works both for the interpreter
>case and jythonc compiled code without using explicit refs. We
>can try to extend and adapt this.

I think listeners are less likely to be edited, and when they are
changed, the loading/linking of the adapter will most likely fail.

Proxies are much more fragile since changes to both the java class and
the python subclass can/should force a new proxy class to be generated.
But to avoid two $py.class formats, we could always compile in the proxy
classname and only use the name when running as a poor-man frozen
application.

>Clearly we cannot support classes redefined multiple times or defined
>only along one execution path but nobody is asking for that or I'm wrong?
><wink>

At least that is easy to explain and to suggest workarounds if any users
have such expectations.

>And yes the tool solution is still the best I can conceive.
>
>> >v) eventually jreload.makeLoadSet can be extended to accept
>> > a classloader (this will not allow reloading but makes sense for
>> > the loading from other srcs issue) 
>> 
>> My senses tells me that this is a minor corner. The real issue for other
>> load sources and sys.classLoader when jython is embedded.
>> 
>Ok, I will not do that. For the moment I have received 1 positive
>feedback about jreload and the tiny bug report. Nothing personal
>but without feeback (+,-,?) I will leave it untouched.
>
>> >i) allow to have jars specified in sys.path from which both
>> >   java classes or python modules can be imported.
>> > open issues:
>...
>> 
>> >  - should the jars added to sys.path be kept open for performance
>> >    reasons as long as they are in sys.path?
>> 
>> Dunno. I suppose it depend on the actual performance of opening an
>> archive, but I would expect the boost to be significant.
>> 
>> >  - when should the "slow" scan for java packages be scheduled, when
>> >    the jar is added to sys.path (more deterministic) or at the next 
>> >    import, should the results be cached (IMHO no), should there be
>> >    a tool to prescan a jar and store the information? in the
>> >    normal jython cache? or together or inside the jar?
>> 
>> Have you seen the "-i" option to the JDK1.3 jar command? I hadn't until
>> now. It seems very nifty. I may change my mind later, but for a first
>> attempt I feel that it is OK to require that some kind of indexing is
>> run on the .jar files on sys.path. Printing a message for non-indexed
>> jars:
>> 
>>    "xxx.jar is ignored. Please run the jarindex command on xxx.jar"
>> 
>Unfortunaly jar -i does not store enough information to support
>dir and import *. In any case we would have to build a tool for
>non java SE2 1.3 env (?).

Yes. I called this tool of ours for "jarindex". It would be a shell/.bat
script written by the installer.

>To be honest this one  is the more complicated of the proposed changes,
>here some concrete ideas:
>
>0. have a tool to "index" a jar in a way that supports dir (maybe the same
>  as above under different options)

I believe that a .jar should only be scanned once. That means that if
the scan result isn't cached we should reject un-indexed jars. Why don't
you want to cache the scan result?

All in all this is my favorite.

>a. ignore performance issues, open and close jars on every import, scan
> or read the index also lazily on import

There shouldn't be noticeable performance difference between putting a
jar on sys.path or on classpath. I think we have to keep it open.

>b. like a) but keep jars open, but I don't see any good place to lazily
>   close them that work at least in many situations if not for all.
>   Just keeping them open sounds bad.

That is what the JVM does for the .zip files found on CLASSPATH. As for
changes to sys.path se below.

>c. the overkill solution [maybe not that much but implies that we must live
>   forever with the following CPython incompatibility (which is quite marginal):
>      L=UserList.UserList()
>      sys.path=L
>   works in CPython but not in Jython, for c) we will have to even enforce this]
>   add to builtin lists the possibility to have an observer (it costs
>   the comparision between a ref and a null on every update method)
>   that is notified for changes, so we can open, scan and close things
>   on sys.path changes. Why every list? to make rare things work:
>      L=['.','foodir']
>      sys.path=L
>      L.append('bazdir')
>   When a list is assigned to sys.path its observer field is assigned,
>   the field of the old one is reset...

Or keep a copy of the sys.path internally. Whenever we want to access
Py.getSystemState().path from java we compare the actual list with the
internal copy and deal with the differences. That would close the
archive at the next import.

Or wrap the list 'L' above with an observable wrapper that intercepts
__XXXitem__ calls. That would close the archive immeditatly but cause
the value returned by 'sys.path' to be different from the value
assigned. My favorite solution.

>d. the "ugly" solution
>   sys.jar_open('foobaz.jar') # it is also scanned or index read
>   sys.path.append('foobaz.jar')
>   ...
>   sys.path.remove('foobaz.jar')
>   sys.jar_close('foobaz.jar')

No!

>   one can put a jar in sys.path without opening it => the jar
>   will be opened and closed on each import, fair enough ;)
>
>All of them have pros and cons, b) needs an answer to the open question,
>c) is somehow "user-friendly", maybe even too much and maybe should be
>discussed with python-dev people because if CPython takes the decision
>of extending sys.path usage, this too implicit approach can be difficult
>to port to the new situation. 

I suppose that putting the zipfile instance itself into sys.path would
solve the close problem. Until CPython decides to go the full length, I
would prefer a less neat solution where we keep only strings in
sys.path.

regards,
finn

Re: [Jython-dev] Jython and the curses library

[<bc...@wo...>]

[Jayson S Baird]

>Hiho all,
>
>  I know this will seem a bit odd, but in a research project, I'm using
>python as the embedded scripting language from java. The hard part is, we
>need to use the curses library. Python, of course has a curses module, but
>can iot be used with Jython? 

No. AFAIK, nobody have made a curses library for java.

>Another question is, can another file, or
>string buffer(python code), be put in the namespace so an import can be
>used from another python script? I know it seems odd, but any redesign
>suggestions or help would be appreciated.

Maybe I'm not parsing your question correctly, but you can fake a module
import with code like this:


--------------- BEGIN ---------------
import new, sys

s = """
def bar():
   print 'bar'
"""

c = compile(s, "dummy", "exec")
m = new.module("dummy")
exec c in m.__dict__, m.__dict__

sys.modules['dummy'] = m


--------------- END ---------------

Later and in a different module, you can import the faked "dummy" module
with code like this:

import dummy
dummy.bar()


regards,
finn

Re: [Jython-dev] re v. sre

[<bc...@wo...>]

[Steven Marcus]

>The re.py module gives preference to the new sre module.
>However, I notice that the java version of the re module is
>implemented using the ORO matcher -- 

Right.

>and I assume that the java
>version is used instead of the py version in jython.

No. In general we have two regular expression engines in jython: "pre"
and "sre", where "pre" is the ORO implementation and "sre" is a port of
Fredrik Lundh's unicode aware engine. The "re" module can map to either
one of these and by default it maps to "sre".

Technically the "sre" engine is doing its compilation in python and the
actual matching is implemented in the "_sre" java module.

>I am a heavy user of regex in jython. What is the future?

sre. Mainly because it is 100% feature compatible with CPython.

>Will the java version of the re module go away and the ORO
>matcher be eliminated?

Maybe. As long as there is no technical advantages for removing it, it
will stay.

>Will I have to continue to explicitly
>import sre to use the python-based regex code?

?

You don't have to import "sre" now. Importing "re" will do.

>Has anyone done any performance comparisons of sre v. ORO?

Not me. If you manage to create some real life timing numbers, please
post them.

regards,
finn

[Jython-dev] Jython and the curses library

[Jayson S B. <js...@ci...>]

Hiho all,

  I know this will seem a bit odd, but in a research project, I'm using
python as the embedded scripting language from java. The hard part is, we
need to use the curses library. Python, of course has a curses module, but
can iot be used with Jython? Another question is, can another file, or
string buffer(python code), be put in the namespace so an import can be
used from another python script? I know it seems odd, but any redesign
suggestions or help would be appreciated.

Thanks in advance,

   Jayson

[Jython-dev] re v. sre

[Steven M. <sr...@ya...>]

Hello all,

The re.py module gives preference to the new sre module.
However, I notice that the java version of the re module is
implemented using the ORO matcher -- and I assume that the java
version is used instead of the py version in jython.

I am a heavy user of regex in jython. What is the future?
Will the java version of the re module go away and the ORO
matcher be eliminated? Will I have to continue to explicitly
import sre to use the python-based regex code? Has anyone done
any performance comparisons of sre v. ORO?

thanks much!
Steven Marcus


__________________________________________________
Do You Yahoo!?
Get email at your own domain with Yahoo! Mail. 
http://personal.mail.yahoo.com/

[Jython-dev] Accessing non-public classes with Jython

[Christian H. <ch...@nu...>]

Hi,

There doesn't seem to be a way to access and test non-public classes with
jython. Would it be possible to extend respectJavaAccessibility to also not
respect accessibility restrictions on classes when set to false?

  Christian

# Setting this to false will allow Jython to provide access to
# non-public fields, methods, and constructors of Java objects.
python.security.respectJavaAccessibility = false

[Jython-dev] Jython GUI debugger/IDE

[Kexx <ke...@in...>]

I've given up trying to make Idle (from Python) work with Jython (no =
TkInter module).
I can edit the files using it, of course, but I would like to debug in =
GUI mode, and do the rest of the wonderful things the IDE supports.

Any alternatives I should consider?
I use JDK 1.3 on Win2k, without "X Window System", so DDD frontends =
won't work, I guess...

Should I bear with pdb?


Best regards,
Juris.

P.S. I would appreciate replys to ke...@in... if at all possible =
(sourceforge mailinglists confuse me).

Re: [Jython-dev] poor man freezing and loading from other sources: proposals and ideas

[Samuele P. <pe...@in...>]

Hi.

[Finn]
> >Please, no pure "it would be nice if" requests.
> 
> But it would be so nice, if we had all this <wink>.
Maybe not all in time for 2.1, but in principle <wink>.
Let's get a bit concrete.

> Opening the sys.classLoader API would be a good thing, but it also opens
> the question of "scanning" the sys.classLoader for java packages. We
> could perhaps make it a requirement that sys.classLoader.getPackages()
> should return something usefull. Unfortunately I think most custom
> classloaders ignore the package support completely.
In any case getPackages() is protected, and my interpretation of the docs
is that even a good-willing classloader can limit itself to define packages 
lazily only when classes are loaded from them.
On the other hand even if the only solution is to define packages
through sys.add_* apis, I think that opening the sys.classLoader API
and making imports even more regular is something necessary for many embedding 
related situations, in any case it is something not for the casual user, the 
others can pay the necessary attention and lack of comfort...

> > iv) <poor man freezing>
...
> > idea: have a tool that take a jar or a set of directories
> > and collect which java packages-classes/python packages-modules are defined
> > and produce a report in a textual format (for maximal flexibility).
> >
> > This report can be put in the jar and be read (impl/expl) by jython runtime
> > as a _resource_  in    
> > order to initiliaze its internal tables for special and java loading (see 
*)).
> 
> A tool sound fine to me. Having such an additional build step is truly a
> minor thing.
> 
> > open issues: $py.class files still require that proxies and adapters
> > are built at runtime, so classloader creation permission is required,
> > this should not be an issue for server side deployment or trusted
> > applets, for normal applets jythonc solve this. Is this a serious 
> > problem?
> 
> I'm sure it depends on the deployment environment, whether or not it is
> a problem <wink>.
> 
> An idea could be a runtime option that caused proxies and adapters to be
> saved in user specified directory. It would also cause the $py.class
> file to be rewritten with a compiled-in proxy reference. That could
> perhaps maintain the illusion that we don't have to pre-compile the
> application.
There is something in that direction (at least for adapters) for debugging 
purpose. I think what you propose is doable. Maybe we can even
avoid to put explicit references (and have 2 $py.class formats),
at least for adapters the actual code works both for the interpreter
case and jythonc compiled code without using explicit refs. We
can try to extend and adapt this.
Clearly we cannot support classes redefined multiple times or defined
only along one execution path but nobody is asking for that or I'm wrong?
<wink>
And yes the tool solution is still the best I can conceive.

> >v) eventually jreload.makeLoadSet can be extended to accept
> > a classloader (this will not allow reloading but makes sense for
> > the loading from other srcs issue) 
> 
> My senses tells me that this is a minor corner. The real issue for other
> load sources and sys.classLoader when jython is embedded.
> 
Ok, I will not do that. For the moment I have received 1 positive
feedback about jreload and the tiny bug report. Nothing personal
but without feeback (+,-,?) I will leave it untouched.

> >i) allow to have jars specified in sys.path from which both
> >   java classes or python modules can be imported.
> > open issues:
...
> 
> >  - should the jars added to sys.path be kept open for performance
> >    reasons as long as they are in sys.path?
> 
> Dunno. I suppose it depend on the actual performance of opening an
> archive, but I would expect the boost to be significant.
> 
> >  - when should the "slow" scan for java packages be scheduled, when
> >    the jar is added to sys.path (more deterministic) or at the next 
> >    import, should the results be cached (IMHO no), should there be
> >    a tool to prescan a jar and store the information? in the
> >    normal jython cache? or together or inside the jar?
> 
> Have you seen the "-i" option to the JDK1.3 jar command? I hadn't until
> now. It seems very nifty. I may change my mind later, but for a first
> attempt I feel that it is OK to require that some kind of indexing is
> run on the .jar files on sys.path. Printing a message for non-indexed
> jars:
> 
>    "xxx.jar is ignored. Please run the jarindex command on xxx.jar"
> 
Unfortunaly jar -i does not store enough information to support
dir and import *. In any case we would have to build a tool for
non java SE2 1.3 env (?).

To be honest this one  is the more complicated of the proposed changes,
here some concrete ideas:

0. have a tool to "index" a jar in a way that supports dir (maybe the same
  as above under different options)
a. ignore performance issues, open and close jars on every import, scan
 or read the index also lazily on import
b. like a) but keep jars open, but I don't see any good place to lazily
   close them that work at least in many situations if not for all.
   Just keeping them open sounds bad.
c. the overkill solution [maybe not that much but implies that we must live
   forever with the following CPython incompatibility (which is quite marginal):
      L=UserList.UserList()
      sys.path=L
   works in CPython but not in Jython, for c) we will have to even enforce this]
   add to builtin lists the possibility to have an observer (it costs
   the comparision between a ref and a null on every update method)
   that is notified for changes, so we can open, scan and close things
   on sys.path changes. Why every list? to make rare things work:
      L=['.','foodir']
      sys.path=L
      L.append('bazdir')
   When a list is assigned to sys.path its observer field is assigned,
   the field of the old one is reset...
d. the "ugly" solution
   sys.jar_open('foobaz.jar') # it is also scanned or index read
   sys.path.append('foobaz.jar')
   ...
   sys.path.remove('foobaz.jar')
   sys.jar_close('foobaz.jar')

   one can put a jar in sys.path without opening it => the jar
   will be opened and closed on each import, fair enough ;)

All of them have pros and cons, b) needs an answer to the open question,
c) is somehow "user-friendly", maybe even too much and maybe should be
discussed with python-dev people because if CPython takes the decision
of extending sys.path usage, this too implicit approach can be difficult
to port to the new situation. Suggestions?

regards.

Re: [Jython-dev] Under Win2K SP1, the simple jpython loader test fails.

[<bc...@wo...>]

[Jeff Fried]

>Under Win2K SP1, the simple jpython loader test fails.
>
>http://jython.sourceforge.net/applets/index.html
>
>Using Netscape 4.76.
>
>Here is the Java Console:
>
>Netscape Communications Corporation -- Java 1.1.5
>
>Type '?' for options.
>
>Symantec Java! ByteCode Compiler Version 210.065
>Copyright (C) 1996-97 Symantec Corporation
>...
># Applet exception: exception: java.lang.NullPointerException
>
>java.lang.NullPointerException
>
>  at org.python.core.PySystemState.initStaticFields(Compiled Code)

Thanks for the bugreport. This bug was introduced in the 2.1a1 release
and will be fixed in the 2.1a2 release. The applet on webpage have been
updated and now works for my version of netscape.

>The failure is that after 7 minutes on a very fast LAN, it still 
>hasn't finished loading.

It never will. The clock will continue counting until the HelloWorld
applet is loaded. If that does not happen because of the bug, the
counter will never stop.

regards,
finn

[Jython-dev] Re: Under Win2K SP1, the simple jpython loader test fails.

[Jeff F. <jc...@ho...>]

The failure is that after 7 minutes on a very fast LAN, it still hasn't finished loading.

... jeff

Jeff Fried wrote:

> Under Win2K SP1, the simple jpython loader test fails.
>
> http://jython.sourceforge.net/applets/index.html
>
> Using Netscape 4.76.
>
> Here is the Java Console:
>
> Netscape Communications Corporation -- Java 1.1.5
>
> Type '?' for options.
>
> Symantec Java! ByteCode Compiler Version 210.065
> Copyright (C) 1996-97 Symantec Corporation
> # Applet exception: class accessLog could not be loaded
>
> # Applet exception: class accessLog could not be loaded
>
> # Applet exception: class accessLog could not be loaded
>
> # Error: Invalid digital signature file (-7885)
> # jar file: C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\jzip85UV.TMP
> # path:     C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\jzip85UV.TMP
> # Error: internal error - error in manifest file (-1)
> # jar file: C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\jzip85UV.TMP
> # path:     /applets/
> # Applet exception: exception: java.lang.NullPointerException
>
> java.lang.NullPointerException
>
>   at org.python.core.PySystemState.initStaticFields(Compiled Code)
>
>   at org.python.core.PySystemState.initialize(Compiled Code)
>
>   at org.python.core.Py.initProperties(Compiled Code)
>
>   at org.python.core.Py.initProxy(Compiled Code)
>
> * at HelloWorld.__initProxy__(Compiled Code)
>
>   at HelloWorld.<init>(Compiled Code)
>
>   at netscape.applet.DerivedAppletFrame$LoadAppletEvent.dispatch(Compiled Code)
>
>   at java.awt.EventDispatchThread$EventPump.dispatchEvents(Compiled Code)
>
>   at java.awt.EventDispatchThread.run(Compiled Code)
>
>   at netscape.applet.DerivedAppletFrame$AppletEventDispatchThread.run(Compiled Code)

[Jython-dev] Under Win2K SP1, the simple jpython loader test fails.

[Jeff F. <jc...@ho...>]

Under Win2K SP1, the simple jpython loader test fails.

http://jython.sourceforge.net/applets/index.html

Using Netscape 4.76.

Here is the Java Console:

Netscape Communications Corporation -- Java 1.1.5

Type '?' for options.

Symantec Java! ByteCode Compiler Version 210.065
Copyright (C) 1996-97 Symantec Corporation
# Applet exception: class accessLog could not be loaded

# Applet exception: class accessLog could not be loaded

# Applet exception: class accessLog could not be loaded

# Error: Invalid digital signature file (-7885)
# jar file: C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\jzip85UV.TMP
# path:     C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\jzip85UV.TMP
# Error: internal error - error in manifest file (-1)
# jar file: C:\DOCUME~1\ADMINI~1\LOCALS~1\Temp\jzip85UV.TMP
# path:     /applets/
# Applet exception: exception: java.lang.NullPointerException

java.lang.NullPointerException

  at org.python.core.PySystemState.initStaticFields(Compiled Code)

  at org.python.core.PySystemState.initialize(Compiled Code)

  at org.python.core.Py.initProperties(Compiled Code)

  at org.python.core.Py.initProxy(Compiled Code)

* at HelloWorld.__initProxy__(Compiled Code)

  at HelloWorld.<init>(Compiled Code)

  at netscape.applet.DerivedAppletFrame$LoadAppletEvent.dispatch(Compiled Code)

  at java.awt.EventDispatchThread$EventPump.dispatchEvents(Compiled Code)

  at java.awt.EventDispatchThread.run(Compiled Code)

  at netscape.applet.DerivedAppletFrame$AppletEventDispatchThread.run(Compiled Code)

[Jython-dev] [Announce] jywiki

[Brian Z. <bri...@ya...>]

Hi list,

This is a wiki clone ported from Martin Pool's PikiPiki to Jython Servlet.
I'm sure porting it to java servlet will be a lot more difficult.

The project is hosted here: http://sourceforge.net/projects/jywiki

And there is a wiki (using phpwiki for now): http://jywiki.sourceforge.net
with information about Jython Servlet in general and jywiki in specific.

I hope Jython servlet users can also use this wiki as a place to share tips.

It's still work in progress since I don't have a lot of time to spare.

Thanks for all the excellent work and support from jython developers.

Cheers,

-Brian Zhou

[Jython-dev] jGrasp integration?

[Stephen L A. <sa...@ea...>]

Howdy:

I'm just getting started with Python (and boy is it cool :) but I 
have a general need to analyze (and reverse-engineer) code in 
multiple languages.  One of the nicer tools I've found is Grasp:

http://www.eng.auburn.edu/department/cse/research/grasp/

It supports java, Ada95, C, C++, and VHDL, but it would be 
ultimately cooler with Python support.  The Auburn University guys 
are not exactly hard-core GNU guys, but I believe the source code 
is available.  There are the original windoze and Linux/UNIX 
versions of Grasp, but only jGrasp is currently under active 
development.

I don't really have the background to do any serious code 
development yet (but I'm trying to learn) nor do I have any time at 
the moment; I think I have one-too-many jobs or kids, but I can't 
figure out which...  I did put a bug in their ear about adding 
Python support to Grasp, but they seem stuck on Java for some 
reason I can't understand.

So I was wondering if any of you folks would be interested in the 
idea of integrating jython with jGrasp, and adding full Python 
support to the Grasp environment.  It would be really cool for 
large Python (and multi-language) projects, analyzing and 
"grokking" someone else's code, etc.  Grasp/jGrasp already supports 
gcc, GNAT, JDK (sun and blackdown), M$(crap)C++, etc, and would be 
that much cooler with Python...  You could visualize code and build 
projects using all of the above from the same environment.  Cool, 
no?

I just had the chance to attend Python 9 (my first real nerd 
conference :) and I was extremely impressed with Python and what 
everyone was doing with it.  Just like Eric Jones said in some of 
his Numeric Python slides - "Amazing..."

Anyway, thanks for listening (reading) - Steve Arnold

PS. To those I met at the conference - It was really cool meeting 
you all.  Keep it up; I was completely blown away.  It was a 180 
degree change from watching a large well-known firm totally screw 
up a $1.4 billion program (yes, I said billion).  You guys are so 
much cooler, it's not even funny...

************************************************
Steve Arnold  CLE (Certifiable Linux Evangelist)

http://arnolds.dhs.org

[Jython-dev] Pretty cool

[John A. T. <jt...@co...>]

We have developed a machine controller written in Java (with real-time
portions in C++), and are looking for a good language that allows scripting
our application. We have found Jython to be great for this, in general, but
have some performance concerns. Currently, we call
"InterativeConsole.push(String)" to download our code (including functions),
line by line to the Jython environment. Then we can invoke the functions
just by calling the function name. When I load a Jython function, does it
get interpreted to some intermediate form that executes faster, or is every
line always parsed every time it is executed?

Also, we find that our functions can have no full line comments; if they do,
when we push the comment line, it interprets it as an "end of function".

Is there any list that we can subscribe to for Jython users?

--------------------------------
John A. Tenney
PRI Automation, Inc.
850 Montgomery Street, Suite 300
San Francisco, CA  94133
Tel: 415-391-9050
E-Mail: jt...@pr...

[Jython-dev] Please add a section for Available Books/Articles etc

[Kasi C. <KCh...@in...>]

Re: [Jython-dev] poor man freezing and loading from other sources: proposals and ideas

[<bc...@wo...>]

[Samuele]

>Hi.
>
>Jython 2.0 more clean semantic wrt import and java loading
>was an improvement over JPython. 

That's an understatement!

>Now often more features have been requested in the directions of what has
>been called:
>
>- "poor man freezing" (the ability to take interpreter $py.class files
>  and package them somehow for deployment)
>- (java) loading from other sources (the possibility to have
>   import working from jars or classloaders specified at runtime) .
>
>Here are some ideas and proposals toward something concrete in those
>directions for 2.1.
>
>Note: jython 2.0 'jreload' module in principle can be used to enable
> imports from jars at runtime.
>
>i) allow to have jars specified in sys.path from which both
>   java classes or python modules can be imported.
> open issues:
>  - will that produce comp. problem with pre-existent code
>    that expects only dirs in sys.path?  

I doubt it. Existing uses of sys.path shouldn't blow up if a directory
is renamed or removed.

>  - should the jars added to sys.path be kept open for performance
>    reasons as long as they are in sys.path?

Dunno. I suppose it depend on the actual performance of opening an
archive, but I would expect the boost to be significant.

>  - when should the "slow" scan for java packages be scheduled, when
>    the jar is added to sys.path (more deterministic) or at the next 
>    import, should the results be cached (IMHO no), should there be
>    a tool to prescan a jar and store the information? in the
>    normal jython cache? or together or inside the jar?

Have you seen the "-i" option to the JDK1.3 jar command? I hadn't until
now. It seems very nifty. I may change my mind later, but for a first
attempt I feel that it is OK to require that some kind of indexing is
run on the .jar files on sys.path. Printing a message for non-indexed
jars:

   "xxx.jar is ignored. Please run the jarindex command on xxx.jar"

>  Some of this behaviour require sys.path changes to have side effects,
>  this could be difficult to obtain in a clean way.
>  
>ii) with i) jython -jar foo.jar will put foo.jar in sys.path ;)

Yes.

>iii) make jython behaviour even more regular:
>   (some definitions:
>     rt-classloader: is the classloader of jython runtime,
>     sys.classLoader is sys.classLoader ;) )
>   actually jython behaviour is as follows:
>   sys.classLoader not set:
>     java loading: rt-classloader > sys.path
>     python loading: sys.path
>   sys.classLoader set:
>     java loading: sys.classLoader
>     python loading: sys.path
>   special case: jythonc compiled code (=> sys.classLoader set)
>     java loading rt-classloader
>     python loading: special from rt-classloader > sys.path
>     but in this case as required sys.path==[]
>     
>   ideas:
>     applicative code should be able to set sys.classLoader
>     obtaining a meaningful behaviour (typically this should not
>     be done or at most once but this should not be enforced),
>     there should be a version of PythonInterpreter.initialize
>     that takes a classLoader and set it in sys.classLoader
>     
>   new behaviour: always:
>    java loading: rt-classloader > sys.classLoader (if set) > sys.path

Opening the sys.classLoader API would be a good thing, but it also opens
the question of "scanning" the sys.classLoader for java packages. We
could perhaps make it a requirement that sys.classLoader.getPackages()
should return something usefull. Unfortunately I think most custom
classloaders ignore the package support completely.

>    python loading: special* from rt-classloader & sys.classLoader > sys.path
>
>   *) by now jythonc compiled code comes with an hardcoded list that specifies
>     which python modules/packages should be loaded from java classloaders,
>     this should be mantained and extended, see iv)
>     
> iv) <poor man freezing>
> how to deal with java -jar foo.jar where foo.jar contains jython runtime,
> java classes using PythonInterpreter API and a set of packaged $py.class
> or .py files ; or the similar applet case, etc ?
>
> First it should be noted that in general we cannot assume that we can scan
> the jar at runtime.
> 
> idea: have a tool that take a jar or a set of directories
> and collect which java packages-classes/python packages-modules are defined
> and produce a report in a textual format (for maximal flexibility).
>
> This report can be put in the jar and be read (impl/expl) by jython runtime
> as a _resource_  in    
> order to initiliaze its internal tables for special and java loading (see *)).

A tool sound fine to me. Having such an additional build step is truly a
minor thing.

> open issues: $py.class files still require that proxies and adapters
> are built at runtime, so classloader creation permission is required,
> this should not be an issue for server side deployment or trusted
> applets, for normal applets jythonc solve this. Is this a serious 
> problem?

I'm sure it depends on the deployment environment, whether or not it is
a problem <wink>.

An idea could be a runtime option that caused proxies and adapters to be
saved in user specified directory. It would also cause the $py.class
file to be rewritten with a compiled-in proxy reference. That could
perhaps maintain the illusion that we don't have to pre-compile the
application.

>Note: the open issue is somehow related  (could be partially solved by) to the 
>possibility of loading jythonc compiled code in the interpreter (this actually 
>fails), some of the proposals here could help to construct a framework for that
>but for the moment I'm not addressing such a situation.
>
>
>v) eventually jreload.makeLoadSet can be extended to accept
> a classloader (this will not allow reloading but makes sense for
> the loading from other srcs issue) 

My senses tells me that this is a minor corner. The real issue for other
load sources and sys.classLoader when jython is embedded.

> and load-sets could have add_package- like features.

Yes, but it is a poor workaround when compared to automatic java package
scanning.

>I would appreciate some feedback about what is/is not necessary,
>design ideas and input about the open issues.
>
>Please, no pure "it would be nice if" requests.

But it would be so nice, if we had all this <wink>.

regards,
finn

[Jython-dev] Unable to download Jython2.0 or 2.1a1

[Jere D. <jda...@Ca...>]

Hello,

When I attempt to download either Jython-20.class or 2.1a1.class I get a
cannot find server 
or DNS error.  Is there someother way for me to get the Jython release?

Thank you,

Jere A. Darling
CardioNow

RE: [Jython-dev] Catching java.lang.OutOfMemoryError

[Garcia, M. <mg...@Bu...>]

Hi Ype,
I don't believe you can catch an OutOfMemoryError.  It does not inherit from
java.lang.Exception but is a VirtualMachineError and signals that the
garbage collector is unable to free any memory.  

If you are getting this error you should check your code because something
serious is happening that should not be.  Sounds like you have a memory leak
somewhere, which yes, can happen in Java.

regards,
Mick 

-----Original Message-----
From: Ype Kingma
To: jyt...@li...
Sent: 3/15/01 2:53 PM
Subject: [Jython-dev] Catching java.lang.OutOfMemoryError

Dear jtyhon developers,

I had google look for some earlier discussion on this subject
as I vaguely recall java and jython exceptions being discussed
not too long ago. However I could not find anything specific
on my problem.

I have roughly the following situation:

import java
somejythonfile = ....

try:
    execfile(somejythonfile)
except SomePythonDeclaredException, ke:
    nicelyCaught = ke
except java.lang.OutOfMemoryError, oome:
    neverCaught = oome
except Exception, e:
    someTimesCaught = e
except:
    # some releases ago, I tried this to catch an occasional
    # internal compiler error, and then it never occurred again...
    ei = sys.exc_info
    ...


My problem is that java.lang.OutOfMemoryError is never caught,
even when 'somejythonfile' causes it.
I also tried to catch java.lang.Throwable, but it flew along as well.

Do I have to go through some java code:

try {
    org.python.core.__builtin__.execfile(somejythonfile);
    // or whatever it is called in java
} catch (java.lang.OutOfMemoryException oome) {
    throw org.python.core.PyException(oome.toString());
}

or can it be done directly in jython?

I realize that running out of memory is tricky business, but
it would be nice if catching this exception would be possible...

Thanks in advance,
Ype

_______________________________________________
Jython-dev mailing list
Jyt...@li...
http://lists.sourceforge.net/lists/listinfo/jython-dev