jython-dev — Discussion of developing and enhancing Jython itself(!) (not how to use Jython)

Re: [Jython-dev] kevin's environ module

[Phil S. <phi...@ho...>]

Thanks Kevin! Great stuff.

I tested it out on my Win 2000 box. 
My linux box is unavailable right now, but I can test
it on that soon if no one else does.

I'm a version of the script with
the following modifications:

The os checks were limited to Windows NT, so I went over
to http://www.tolstoy.com/samizdat/sysprops.html
and got a list of the os.name strings there and
added them in along with some guesses.

Under the Win95 stream I believe the correct command
is 'command /c' so I poked that in, hope someone can
verify that this works. I'm relying on 
http://www.easydos.com/dosindex.html for this info.

Under win32, os.environ should be case-insensitive,
so I added that in as well (this follows
cpython's behaviour). Case-sensitivity should be
maintained for everything else (but this is untested).

I didn't really take an in-depth look at the module
other than to verify that it seems to work as
expected. 

Kevin Butler wrote:
> 
> Finn Bock wrote:
> >
> > [Phil Surette]
> >
> > >I am planning to look into adding os.environment and os.system
> > >support in, basically cribbing from how ant does this, but
> > >I have not had time yet.
> >
> > I hope you and/or someone else find some time to look into it, it would
> > be a valuable addition. I'm not sure what the right way to enable the
> > os.environment should be. It isn't right to fill the os.environment dict
> > whenever "os" is imported, that would be way to slow for all the program
> > that never uses any enviroment variables. Any thoughts?
> 
> So I started playing with this, and didn't stop until I was basically functional.
> 
> Here's the code I came up with (it even has tests!)
> 
> I'm afraid I wrote in on NT, so testing on Unixes would be appreciated...
> 
> Comments/suggestions appreciated.
> 
> kb
> 
>   ------------------------------------------------------------------------
> # os module enhancements for Jython
> # includes System command and lazily populated environ object
> # Kevin Butler <kev...@bi...>
> #
> # Features:
> # - transparently populates environment on first access
> # - transparently disables lazy evaluation for subsequent accesses
> # - passes changed environment to system command
> 
> #
> # Does not:
> # - ensure thread-safe initial population of environment
> # - read both stderror & stdin, probably in separate threads
> # - pass changed environment to child processes started with Runtime.exec (is this possible?)
> # - handle commands with '"' in them correctly on systems that use sh
> from java.lang import Runtime, System
> from java.io import BufferedReader, InputStreamReader
> from UserDict import UserDict
> 
> class ShellExec:
>     defaultTemplate = 'sh -c "%s"'
>     templates = {
>         "Windows NT": 'cmd /c %s',
>         }
> 
>     def __init__( self, template=None ):
>         """template is a string formatting template to format command to execute."""
>         if template:
>             self.template = template
>         else:
>             # determine the appropriate command template for the current OS
>             self.template = ShellExec.templates.get(
>                 System.getProperty( "os.name" ),
>                 ShellExec.defaultTemplate
>                 )
> 
>     def execute( self, cmd ):
>         """Execute cmd in a shell, and then return process"""
>         shellCmd = self.formatCmd( cmd )
>         if _environPopulated:
>             # already populated environment, so pass environment to subprocess
>             env = self.formatEnvironment( environ )
>             p = Runtime.getRuntime().exec( shellCmd, env )
>         else:
>             # use default environment
>             p = Runtime.getRuntime().exec( shellCmd )
>         return p
> 
>     def system( self, cmd ):
>         """Act like the standard library 'system' call.
>         Execute a command in a shell, and send output to stdout.
>         """
>         p = self.execute( cmd )
> 
>         # should read both stderror and stdout in separate threads...
>         stream = BufferedReader( InputStreamReader( p.getInputStream() ))
>         # we want immediate output, so don't call readLines
>         while 1:
>             line = stream.readLine()
>             if line is None: break
>             print line
> 
>         return p.waitFor()
> 
>     def readLines( self, stream ):
>         lines = []
>         while 1:
>             line = stream.readLine()
>             if line is None: break
>             lines.append( line )
>         return lines
> 
>     def formatCmd( self, cmd ):
>         """Format a command for execution in a shell.
>         """
>         return self.template % cmd
> 
>     def formatEnvironment( self, env ):
>         """Format enviroment in lines suitable for Runtime.exec"""
>         lines = []
>         for keyValue in env.items():
>             lines.append( "%s=%s" % keyValue )
>         return lines
> 
> _shellExec = ShellExec()
> 
> def system( cmd ):
>     return _shellExec.system( cmd )
> 
> def _populateEnviron():
>     """Populate this module's 'environ' variable.
>     This unwraps the 'envelope' pattern of the default lazy dictionary.
>     """
>     defaultEnvironCmd = "env"
>     environCmds = {
>         "Windows NT": "set"
>         }
>     # determine the appropriate command template for the current OS
>     environCmd = environCmds.get(
>         System.getProperty( "os.name" ),
>         defaultEnvironCmd
>         )
> 
>     p = _shellExec.execute( environCmd )
>     env = {}
>     stream = BufferedReader( InputStreamReader( p.getInputStream()))
>     for line in _shellExec.readLines( stream ):
>         i = line.index( '=' )
>         env[ line[:i]] = line[i+1:]
> 
>     # assign to os.environ, so future accesses incur no overhead
>     global environ, _environPopulated
>     environ = env
>     _environPopulated = 1
>     return env
> 
> class LazyDict( UserDict ):
>     """A lazy-populating User Dictionary.
>     Lazy initialization is not thread-safe.
>     """
>     def __init__( self, dict=None, populate=lambda: {} ):
>         """populate is a function that returns the populated dictionary"""
>         UserDict.__init__( self, dict )
>         self.initialized = 0
>         self.populate = populate
> 
>     def __populate( self ):
>         if not self.initialized:
>             self.initialized = 1 # not thread-safe!
>             # store as self.data so any 'set' sets in original as well
>             self.data = self.populate()
> 
>     ########## extend methods from UserDict by pre-populating
>     def __repr__(self):
>         self.__populate()
>         return UserDict.__repr__( self )
>     def __cmp__(self, dict):
>         self.__populate()
>         return UserDict.__cmp__( self, dict )
>     def __len__(self):
>         self.__populate()
>         return UserDict.__len__( self )
>     def __getitem__(self, key):
>         self.__populate()
>         return UserDict.__getitem__( self, key )
>     def __setitem__(self, key, item):
>         self.__populate()
>         UserDict.__setitem__( self, key, item )
>     def __delitem__(self, key):
>         self.__populate()
>         UserDict.__delitem__( self, key )
>     def clear(self):
>         self.__populate()
>         UserDict.clear( self )
>     def copy(self):
>         self.__populate()
>         return UserDict.copy( self )
>     def keys(self):
>         self.__populate()
>         return UserDict.keys( self )
>     def items(self):
>         self.__populate()
>         return UserDict.items( self )
>     def values(self):
>         self.__populate()
>         return UserDict.values( self )
>     def has_key(self, key):
>         self.__populate()
>         return UserDict.has_key( self, key )
>     def update(self, dict):
>         self.__populate()
>         UserDict.update( self, dict )
>     def get(self, key, failobj=None):
>         self.__populate()
>         return UserDict.get( self, key, failobj )
>     def setdefault(self, key, failobj=None):
>         self.__populate()
>         return UserDict.setdefault( self, key, failobj )
>     def popitem(self):
>         self.__populate()
>         return UserDict.popitem( self )
> 
> # initialize environ to the placeholder
> environ = LazyDict( populate=_populateEnviron )
> _environPopulated = 0
> 
> def putenv( key, value ):
>     environ[ key ] = value
> 
> def getenv( key ):
>     return environ[ key ]
> 
> def __test():
>     key, value = "testKey", "testValue"
>     org = environ
>     testCmds = [
>         "echo hello there",             # no quotes, should output both words
>         "echo PATH=%PATH%",             # should print PATH (on NT)
>         "echo %s=%%%s%%" % (key,key),   # should print 'testKey=%testKey%' on NT before initialization, and 'testKey=testValue' after
>         "echo PATH=$PATH",              # should print PATH (on Unix)
>         "echo %s=$%s" % (key,key),      # should print 'testKey=testValue' on Unix after initialization
> 
> # the following tests have double quotes in the commands. They work w/ NT's cmd shell, but not w/ sh. Escaping the quotes doesn't seem to help.
> #        'echo "hello there"',           # should output quotes on NT, no quotes on Unix.
> #        r'''python -c "import sys;sys.stdout.write( 'why\n' )"''', # should print 'why' to stdout.
> #        r'''python -c "import sys;sys.stderr.write( 'why\n' )"''', # should print 'why' to stderr, but it won't right now.  Should complete, though!  :-)
>         ]
> 
>     assert not _environPopulated, "before population, _environPopulated should be false"
> 
>     # test system - we should really grab the output of the system command, but we aren't
>     for cmd in testCmds:
>         print "\nExecuting %s with default environment" % cmd
>         assert not __shellExec.system( cmd ), "%s failed with default environment" % cmd
> 
>     # trigger initialization of environment
>     environ[ key ] = value
> 
>     assert _environPopulated, "after population, _environPopulated should be true"
>     assert org.get( key, None ) == value, "expected stub to have %s set" % key
>     assert environ.get( key, None ) == value, "expected real environment to have %s set" % key
> 
>     # test system using the non-default environment - should really grab output, but oh, well.
>     for cmd in testCmds:
>         print "\nExecuting %s with initialized environment" % cmd
>         assert not __shellExec.system( cmd ), "%s failed with initialized environment" % cmd
> 
>     assert environ.has_key( "PATH" ), "expected environment to have PATH attribute (this may not apply to all platforms!)"
> 
>

[Jython-dev] java->python problems

[dman <ds...@ri...>]

Hi all.  This probably isn't appropriate for the -dev list, but too
frequently I don't get a response on the -users list.

I'm trying to use Jython(c) to allow using Moshe Zadka's PMS framework
along with a Java/Swing GUI for a school project.  (The other group
members are writing the gui and they don't know python)  I've run into
a couple of problems, though, all related to java's type system.

There are several modules in the PMS package.  I have taken the base
classes of the inheritance tree and made them inherit from
java.lang.Object and added the @sig lines.  A simplified example :

----- PMS/AbstractFolder.py
class AbstractFolder( java.lang.Object ) : pass

----- PMS/Folder.py
import AbstractFolder
class Folder( AbstractFolder.AbstractFolder ) : pass


I also have the following class as a factory for the GUI to create a
hardcoded folder for the prototype demo.

----- DemoFolder1.py
class DemoFolder1( java.lang.Object ) :
    def make( self , name ) :
        """
        @sig public Object make( String name )
        """

        server = PMS.MaildirServer.MaildirServer( "server_name" , "." )
        PMS.Configuration.Configuration().servers[ server ] = lambda x : x
        folder = PMS.Folder.Folder( server , name )
        folder.create()

        print folder.__class__ # correctly prints "PMS.Folder.Folder"
        return folder



I used jythonc to compile the stuff, and everything compiles fine.
The Java code that uses this looks like :


            Object folder = (new DemoFolder1()).make( "Demo1" );
            MailMessage.Message message = new MailMessage.Message();

            // This line prints out
            // org.python.proxies.PMS.Folder$Folder$2
            System.out.println( folder.getClass() ) ;

            try
            {
                ((Folder)folder).save_message( message );
            }
            catch ( Exception err )
            {
                // I get a ClassCastException here
            }


Even though the Python factory returns an object that should match the
generated java class, it doesn't match and java can't handle it.


I also tried :
            Folder f = new Folder(
                new MaildirServer( "name" , "." ) , "Demo2" ) ;
            f.save_message( message ) ;

This works if I write it in Python, but written in Java gives :

    Exception occurred during event dispatching:
    Traceback (innermost last):
      (no code object) at line 0
    TypeError: __init__() takes at least 3 arguments (1 given)

As you can see, too much debugging info was lost in the translation to
java so I have no idea where this error is occurring.


I can think of 2 workarounds to the problem, but neither is very
desirable, so any suggestions, hints, pointers, or even wild guesses
are greatly appreciated.


As an additional annoyance, it seems that a jythonc-compiled python
module can't import any python modules that are not translated to
java.  Is this limitation real, or am I just doing something wrong?

TIA!
-D

Re: [Jython-dev] jythons module os

[dman <ds...@ri...>]

On Thu, Nov 01, 2001 at 05:08:06PM +0000, Finn Bock wrote:
| On Thu, 1 Nov 2001 10:26:25 -0500, you wrote:
| 
| >On Thu, Nov 01, 2001 at 03:14:17PM +0000, Finn Bock wrote:
| >| [Phil Surette]
| > 
| >| >I haven't thought writing through yet, but I figure 
| >| >reading will hit a large percentage of the use cases.
| >| 
| >| Writing is normally done by inserting values in the os.environment dict
| >| and using that dict for the os.system() call. We can't possibly do any
| >| better in core jython.
| >
| >How does os.setenv() figure into this? 
| 
| I guess you mean os.putenv()?

Umm ... <checks docs> ... yeah, I didn't look at the docs before I
posted -- I always mess up the name one way or another.

| >I imagine that this operation is probably not possible in Java,
| 
| Right.
| 
| >thus the call should either silently
| >ignore it, or raise an exception.
| 
| Either os.putenv() shouldn't be implemented at all or it should be
| implemented as:
| 
| def putenv(varname, value):
|    os.enviroment[varname] = value

This depends on what the semantics of os.environ should be.  In
CPython, the line 
    os.environ[ key ] = value
doesn't work.  At least, it _may_ change theh dict, but it has no
effect on the environment.

| >I was thinking that, perhaps, some environment values can be gotten
| >from Java directly.  For example,
| >
| >    print os.eviron[ "HOME" ]
| >
| >    could be the same as
| >
| >    System.out.println( System.getProperty( "user.home" ) ) ;
| >
| >though on some JVMs, user.home is really messed up (ex, jdk1.1.8 on
| >win).  I think Java ignores the user's preferences (ie setting $HOME
| >in bash or cmd.exe) and using whatever it feels like, so this may not
| >be a good idea.
| 
| The property route should not be mixed with the "sh -c env" route. Using

Ahh, I see, _that's_ how you get ahold of the data!

| properties have been mentioned before, but nobody have picked it up and
| suggested a patch. I guess it is because the defined set of properties
| is too limited to be really usefull as a replacement for environment
| variables.

I am using Moshe's PMS framework with Jython right now for a school
project.  My group has not used python at all, but is familiar with
Java hence the choice to use Jython.  Anyways, in the system as it is
shipped, there is the line

    DIR = os.path.join( os.environ[ "HOME" ] , "something" )

when it threw an exception, I decided to quickly wrap it in a
try-except and use java's system property (this maintains
compatibility with CPython, though I doubt Moshe would like a patch
like that).  It works for now, though I agree that java's system
properties are no replacement for the environment.

-D

Re: [Jython-dev] jythons module os

[Kevin B. <kb...@ca...>]

Finn Bock wrote:
> 
> [Phil Surette]
> 
> >I am planning to look into adding os.environment and os.system
> >support in, basically cribbing from how ant does this, but
> >I have not had time yet.
> 
> I hope you and/or someone else find some time to look into it, it would
> be a valuable addition. I'm not sure what the right way to enable the
> os.environment should be. It isn't right to fill the os.environment dict
> whenever "os" is imported, that would be way to slow for all the program
> that never uses any enviroment variables. Any thoughts?

So I started playing with this, and didn't stop until I was basically functional.

Here's the code I came up with (it even has tests!)

I'm afraid I wrote in on NT, so testing on Unixes would be appreciated...

Comments/suggestions appreciated.

kb

Re: [Jython-dev] Jython code coverage testing: setline() calls, linenumbertable question, execfile() bug?

[Samuele P. <ped...@bl...>]

[Ype Kingma]
> > >def tracer(frame, why, arg):
> > >    print why, frame.f_lineno, arg
> > >    return tracer
[snip foo def]
> > >import sys
> > >sys.settrace(tracer)
> > >foo()
> >
> > I'll give it a try,
> >
> 
> This works fine when called interactively.
> However when I use it via execfile() i get a:
> NameError: tracer
> on the "return tracer" statement.

Sorry, what do you mean by execution via
execfile? I tried to put the above code
in y.py and to call it both from a interactive
shell with execfile("y.py") or a 
z.py file containing execfile("y.py").

In both cases I could not reproduce the error.

Samuele.

Re: [Jython-dev] Jython code coverage testing: setline() calls, linenumbertable question, execfile() bug?

[<bc...@wo...>]

[Ype Kingma]

>My class file parser does not find a linenumber table in java $py.class
>(21a3) files. It finds is an UTF8 'LineNumberTable' in the constant pool,
>but no attribute seems to be using that name.

The LineNumberTable is an attribute on each Code attribute.

>(Is this a bug?).
>In fact the only attributes it finds are an  'UTF8' 'SourceFile'
>attribute with the file name in the constant pool and
>an  'UTF8' 'org.python.APIVersion' with a 4byte of value 9.

That is the class file attributes.

>This coincides with Module.java in the compiler package.
>Method write() (line 550) does the attributes above,
>but no LineNumberTable. Adding it to a $py.class file
>would take 4 bytes per 'executable' line.
>
>The line number table is completely
>redundant given the setline() calls in the code.

Correct. It is completely and utterly redundant. It is *only* used when
the JVM prints a java stacktrace. With the LineNumberTable in place, the
java stacktrace will include nice and correct line numbers for the the
python sources:

        ...
        at org.python.core.PyObject.invoke(PyObject.java:2028)
        at org.python.pycode._pyx0.f$0(x.py:4)
        at org.python.pycode._pyx0.call_function(x.py)
        ...

Notice the "(x.py:4)" part.

>The same parser for $py.class files can also fish for all
>the setline() arguments.

Correct, it is the same information.

>This works fine when called interactively.
>However when I use it via execfile() i get a:
>NameError: tracer
>on the "return tracer" statement.

Can you an example. Again, it works for me:

Jython 2.1a3 on java1.3.0 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> execfile("tracer.py")
call 0 None
line 6 None
line 7 None
line 7 None
line 8 None
line 7 None
line 8 None
line 7 None
line 8 None
line 7 None
line 8 None
line 7 None
line 8 None
line 7 None
line 8 None
line 7 None
line 8 None
line 7 None
line 8 None
line 7 None
line 8 None
line 7 None
line 8 None
line 7 None
line 9 None
return 9 None
>>>

regards,
finn

Re: [Jython-dev] ord([123]) bug in 21a3

[<bc...@wo...>]

[Ype Kingma]

>ord([123])
>
>raises java.lang.ClassCastException instead of a TypeError.

It must be poetic justice! Right after I bitched about Brian Zimmer's
use of __tojava__ on jython-users, you discover the exact same error in
the core of jython.

>I you prefer, I'll can report this on sourceforge, too.

Of course I prefer a SF bugreport! You know it is a bug and you even
have a failing test program.

regards,
finn

Re: [Jython-dev] Jython code coverage testing: setline() calls, linenumbertable question, execfile() bug?

[Ype K. <yk...@xs...>]

Finn,

(A bit long, two subjects, a maybe bug and a likely bug.)

I wrote:

> Finn,
>
> >[Ype Kingma]
> >
> >>Hello jythoneers,
> >>
> >>I've been using pyunit for a while in jython and would like to
> > >have code coverage reports from some of the tests.
>
> <...>
>
> >But if all you need to know is whether a line have some executable
> >python code, then the line number table in the $py.class file can answer
> >that with the same certainty as the setline() method calls can.
>
> It would probably be a lot easier to parse, so I'll give it a try.
>
> >Each time a call to PyFrame.setline() is generated, we also add a line
> >number entry to the line number table. See the Code.setline() for
> >details.
>
> Ok.

>
> >
>
> >Line tracing is implemented in dynamic compilation. Unless you have
> >discovered a bug of course. This works for me:
>

My class file parser does not find a linenumber table in java $py.class
(21a3) files. It finds is an UTF8 'LineNumberTable' in the constant pool,
but no attribute seems to be using that name.
(Is this a bug?).
In fact the only attributes it finds are an  'UTF8' 'SourceFile'
attribute with the file name in the constant pool and
an  'UTF8' 'org.python.APIVersion' with a 4byte of value 9.

This coincides with Module.java in the compiler package.
Method write() (line 550) does the attributes above,
but no LineNumberTable. Adding it to a $py.class file
would take 4 bytes per 'executable' line.

The line number table is completely
redundant given the setline() calls in the code.
The same parser for $py.class files can also fish for all
the setline() arguments.

The parser is written in jython, but it
is ugly (it ends with an exception because it is confused
about u4 bytes read already from the DataInputStream,
even though readInt() works fine reading directly.)
Also it is quite untested, and the internal workings
are, uhh, not write to home about (Dutchism).
If anyone is interested I'll gladly provide a copy.

>
> >
> >
> >def tracer(frame, why, arg):
> >    print why, frame.f_lineno, arg
> >    return tracer
> >
> >def foo():
> >    for i in range(10):
> >        a = "abc"
> >    a = "Done"
> >
> >import sys
> >sys.settrace(tracer)
> >foo()
>
> I'll give it a try,
>

This works fine when called interactively.
However when I use it via execfile() i get a:
NameError: tracer
on the "return tracer" statement.

The following works when called via execfile(),
the interesting difference is probably the fact
that the tracer function is now not in a module
name space when returned for line tracing,
so I think there is a bug related to the new nested
namespaces in execfile():


class LineInfoCollector:
    def __init__(self):
       self.lineInfos = {}

    def addLineInfo(self, lineInfo):
        self.lineInfos[lineInfo] = 1

    def getLineInfos(self):
        infos = self.lineInfos.keys()
        infos.sort()
        return infos

    def tracer(self, frame, why, arg):
        self.addLineInfo((frame.f_code.co_filename,
                          frame.f_code.co_name,
                          frame.f_lineno,
                          why, arg))
        return self.tracer 

def foo():
    for i in range(2):
       a = "abc"
    a = "Done"

if __name__ == '__main__':
    import sys

    ltrac = LineInfoCollector() 
    sys.settrace(ltrac.tracer)
    try: foo()
    finally: sys.settrace(None)

    print 'Lines covered:'
    for lineInfo in map(str, ltrac.getLineInfos()):
        print lineInfo


Regards,
Ype

[Jython-dev] ord([123]) bug in 21a3

[Ype K. <yk...@xs...>]

ord([123])

raises java.lang.ClassCastException instead of a TypeError.

I you prefer, I'll can report this on sourceforge, too.

Regards,
Ype

Re: [Jython-dev] jythons module os

[<bc...@wo...>]

[Phil Surette]

>Yeah, I'm doing that. 
>
>I'm writing three tasks:
>1) a jython task. This is like the ant <script> task
>but it has several improvements:
>   - handles indented jython scripts.
>   - massages stack traces to make line numbers relative to
>the ant build file, rather than relative to the start of the
>script within the build file
>   - auto-imports some packages (right now, os, os.path,
>sys, string, re) - interetes to hear if others think
>this is a good/evil feature

This part is evil. Rule #2 clearly says that explicit is better than
implicit. 
   http://www.python.org/doc/Humor.html#zen

If you really think it is usefull, I can't see any problems with adding
a xml attribute to the jython tag to enable automatic import of some
modules, say:

<jython autoimports="true">
os.mkdir("newdir")
</jython>

>   - saves typing <jython> vs <script language="jpython">
>The jython task is working.

Cool.

>2) jythonc task - haven't started yet, hopefully tonight.
>
>3) jytaskdef task - define an ant task from within an
>ant build file. You write some jython code with (at
>the minimum) an execute(self) method; this code gets
>embedded in a jython class that extends org.apache.an.main.Task;
>this gets compiled  with jythonc; 

Is the jythonc step needed? Maybe I misunderstand the intention behind
jytaskdef, but from your description, I imagine something like:

<jytaskdef taskname="MyTask">
import org
class MyTask(org.apache.tools.ant.Task):
    def execute(self):
	print "Do something usefull"
</jytaskdef>

jytaskdef would then call project.addTaskDefinition("MyTask", MyTask). 

So, how big is my misunderstanding?

>then the task is defined
>and available everywhere in the build script.
>
>I do not expect that the ant developers will like/want to
>maintain task #3... however I think it will be very useful.

I'm not sure I would want the jakarta people to maintain #2. We should
be able to add new options to the jythonc and ensure that ant
integration is in sync. Assuming it is technically possible to include
the task in jython.jar, I would prefer to maintain it outself.

OTOH, I have never maintained an ant Task. If the ant project changes
the API between every ant release it may to big a job to keep up with
them. 

>Anyway, I'll submit the tasks to the jython group when
>they're ready.

Nice.

regards,
finn

Re: [Jython-dev] jythons module os

[<bc...@wo...>]

On Thu, 1 Nov 2001 10:26:25 -0500, you wrote:

>On Thu, Nov 01, 2001 at 03:14:17PM +0000, Finn Bock wrote:
>| [Phil Surette]
> 
>| >I haven't thought writing through yet, but I figure 
>| >reading will hit a large percentage of the use cases.
>| 
>| Writing is normally done by inserting values in the os.environment dict
>| and using that dict for the os.system() call. We can't possibly do any
>| better in core jython.
>
>How does os.setenv() figure into this? 

I guess you mean os.putenv()?

>I imagine that this operation is probably not possible in Java,

Right.

>thus the call should either silently
>ignore it, or raise an exception.

Either os.putenv() shouldn't be implemented at all or it should be
implemented as:

def putenv(varname, value):
   os.enviroment[varname] = value


>I was thinking that, perhaps, some environment values can be gotten
>from Java directly.  For example,
>
>    print os.eviron[ "HOME" ]
>
>    could be the same as
>
>    System.out.println( System.getProperty( "user.home" ) ) ;
>
>though on some JVMs, user.home is really messed up (ex, jdk1.1.8 on
>win).  I think Java ignores the user's preferences (ie setting $HOME
>in bash or cmd.exe) and using whatever it feels like, so this may not
>be a good idea.

The property route should not be mixed with the "sh -c env" route. Using
properties have been mentioned before, but nobody have picked it up and
suggested a patch. I guess it is because the defined set of properties
is too limited to be really usefull as a replacement for environment
variables.

regards,
finn

RE: [Jython-dev] jythons module os

[Phil S. <psu...@es...>]

Yeah, I'm doing that. 

I'm writing three tasks:
1) a jython task. This is like the ant <script> task
but it has several improvements:
   - handles indented jython scripts.
   - massages stack traces to make line numbers relative to
the ant build file, rather than relative to the start of the
script within the build file
   - auto-imports some packages (right now, os, os.path,
sys, string, re) - interetes to hear if others think
this is a good/evil feature
   - saves typing <jython> vs <script language="jpython">
The jython task is working.

2) jythonc task - haven't started yet, hopefully tonight.

3) jytaskdef task - define an ant task from within an
ant build file. You write some jython code with (at
the minimum) an execute(self) method; this code gets
embedded in a jython class that extends org.apache.an.main.Task;
this gets compiled  with jythonc; then the task is defined
and available everywhere in the build script.

I do not expect that the ant developers will like/want to
maintain task #3... however I think it will be very useful.
Anyway, I'll submit the tasks to the jython group when
they're ready.

-----Original Message-----
From: bc...@wo... [mailto:bc...@wo...]
Sent: Thursday, November 01, 2001 10:14 AM
To: jyt...@li...
Subject: Re: [Jython-dev] jythons module os


[Phil Surette]

>I was hoping to do a lazy initialization thing. 
>
>I'm open to suggestions as to how to do it... 
>I haven't actually written anything yet and I'm working
>on writing some jython tasks for ant first. 

On that note, if anyone feels like writing an ant task to invoke the
jythonc compiler it would be quite a usefull.

>My thoughts were to create a class that implements all the
>dictionary things - __getitem__ etc - and populate
>the dictionary the first time any of these are called.

Good. Maybe we should add a system property to completely disable all
attempt to fill os.environment. I can image there is environments where
a security manager does not allow calls to Runtime.exec().

>By cribbing from ant I should be able to put in support 
>for a read-only environment for unices and win32.

That's fine for a start. Beyond win&unix we have to let users send
patches for their own platforms anyway.

Does anyone know how ant decide that it is running on unix?

>I haven't thought writing through yet, but I figure 
>reading will hit a large percentage of the use cases.

Writing is normally done by inserting values in the os.environment dict
and using that dict for the os.system() call. We can't possibly do any
better in core jython.

regards,
finn

_______________________________________________
Jython-dev mailing list
Jyt...@li...
https://lists.sourceforge.net/lists/listinfo/jython-dev

Re: [Jython-dev] jythons module os

[dman <ds...@ri...>]

On Thu, Nov 01, 2001 at 03:14:17PM +0000, Finn Bock wrote:
| [Phil Surette]
 
| >I haven't thought writing through yet, but I figure 
| >reading will hit a large percentage of the use cases.
| 
| Writing is normally done by inserting values in the os.environment dict
| and using that dict for the os.system() call. We can't possibly do any
| better in core jython.

How does os.setenv() figure into this?  I imagine that this operation
is probably not possible in Java, thus the call should either silently
ignore it, or raise an exception.

I was thinking that, perhaps, some environment values can be gotten
from Java directly.  For example,

    print os.eviron[ "HOME" ]

    could be the same as

    System.out.println( System.getProperty( "user.home" ) ) ;

though on some JVMs, user.home is really messed up (ex, jdk1.1.8 on
win).  I think Java ignores the user's preferences (ie setting $HOME
in bash or cmd.exe) and using whatever it feels like, so this may not
be a good idea.

-D

Re: [Jython-dev] jythons module os

[<bc...@wo...>]

[Phil Surette]

>I was hoping to do a lazy initialization thing. 
>
>I'm open to suggestions as to how to do it... 
>I haven't actually written anything yet and I'm working
>on writing some jython tasks for ant first. 

On that note, if anyone feels like writing an ant task to invoke the
jythonc compiler it would be quite a usefull.

>My thoughts were to create a class that implements all the
>dictionary things - __getitem__ etc - and populate
>the dictionary the first time any of these are called.

Good. Maybe we should add a system property to completely disable all
attempt to fill os.environment. I can image there is environments where
a security manager does not allow calls to Runtime.exec().

>By cribbing from ant I should be able to put in support 
>for a read-only environment for unices and win32.

That's fine for a start. Beyond win&unix we have to let users send
patches for their own platforms anyway.

Does anyone know how ant decide that it is running on unix?

>I haven't thought writing through yet, but I figure 
>reading will hit a large percentage of the use cases.

Writing is normally done by inserting values in the os.environment dict
and using that dict for the os.system() call. We can't possibly do any
better in core jython.

regards,
finn

Re: [Jython-dev] shutdowns in jython / atexit

[Samuele P. <ped...@bl...>]

[Finn]
> [Samuele] 
> 
> >Make sense, but I would not consider that a reverse of initialize
> >but an helper for calling sys.exitfunc when deemed necessary
> >(e.g. in presence of a SystemExit exception).
> 
> It may eventually do other stuff (such as close open files) so I will
> prefer a generic name. Something like "finalize".

My point is that initialize must be called only once.
For exitfunc we should either choose to make it static or an
instance field of PySystemState, so it would there could be
multiple versions of it. 
In particular I can imagine
(rare) situations where you possibly have multiple interpreters and
you would call finalize more than once. So I don't see it
as the symmetric of initialize.

I should admit I have checked CPython wrt these aspects.

Samuele.

Re: [Jython-dev] shutdowns in jython / atexit

[<bc...@wo...>]

[me]

> - from a new method on PythonInterpreter which an application that 
>   embed jython can call (as the reverse of initialize).

[Samuele] 

>Make sense, but I would not consider that a reverse of initialize
>but an helper for calling sys.exitfunc when deemed necessary
>(e.g. in presence of a SystemExit exception).

It may eventually do other stuff (such as close open files) so I will
prefer a generic name. Something like "finalize".

>Further:
>I think we should deal with the jythonc generated main methods
>or runMain perhaps too.

Yes indeed. Good catch.

regards,
finn

RE: [Jython-dev] jythons module os

[Phil S. <psu...@es...>]

I was hoping to do a lazy initialization thing. 

I'm open to suggestions as to how to do it... 
I haven't actually written anything yet and I'm working
on writing some jython tasks for ant first. 

My thoughts were to create a class that implements all the
dictionary things - __getitem__ etc - and populate
the dictionary the first time any of these are called.

By cribbing from ant I should be able to put in support 
for a read-only environment for unices and win32.
I haven't thought writing through yet, but I figure 
reading will hit a large percentage of the use cases.


-----Original Message-----
From: bc...@wo... [mailto:bc...@wo...]
Sent: Wednesday, October 31, 2001 2:26 PM
To: jyt...@li...
Subject: Re: [Jython-dev] jythons module os


[Phil Surette]

>I am planning to look into adding os.environment and os.system
>support in, basically cribbing from how ant does this, but
>I have not had time yet.

I hope you and/or someone else find some time to look into it, it would
be a valuable addition. I'm not sure what the right way to enable the
os.environment should be. It isn't right to fill the os.environment dict
whenever "os" is imported, that would be way to slow for all the program
that never uses any enviroment variables. Any thoughts?

>However, I think jnios is probably the better way to go for
>you. Someone needs to finish it though. 

Any volunteers?

regards,
finn

_______________________________________________
Jython-dev mailing list
Jyt...@li...
https://lists.sourceforge.net/lists/listinfo/jython-dev

Re: [Jython-dev] shutdowns in jython / atexit

[Samuele P. <ped...@bl...>]

[Finn]
> I disagree about calling sys.exitfunc from the JVM shutdown hooks. I
> think sys.exitfunc should be called from 
> 
> - the execption handling in Py.maybeSystemExit()
> - from the end of jython.java and 

Agreed.

> - from a new method on PythonInterpreter which an application that 
>   embed jython can call (as the reverse of initialize).
 
Make sense, but I would not consider that a reverse of initialize
but an helper for calling sys.exitfunc when deemed necessary
(e.g. in presence of a SystemExit exception).

Further:
I think we should deal with the jythonc generated main methods
or runMain perhaps too.

regards.

Re: [Jython-dev] Jython code coverage testing: setline() calls

[Ype K. <yk...@xs...>]

Finn,

>[Ype Kingma]
>
>>Hello jythoneers,
>>
>>I've been using pyunit for a while in jython and would like to
> >have code coverage reports from some of the tests.

<...>

>But if all you need to know is whether a line have some executable
>python code, then the line number table in the $py.class file can answer
>that with the same certainty as the setline() method calls can.

It would probably be a lot easier to parse, so I'll give it a try.

>Each time a call to PyFrame.setline() is generated, we also add a line
>number entry to the line number table. See the Code.setline() for
>details.

Ok.

>

>Line tracing is implemented in dynamic compilation. Unless you have
>discovered a bug of course. This works for me:

>
>
>def tracer(frame, why, arg):
>    print why, frame.f_lineno, arg
>    return tracer
>
>def foo():
>    for i in range(10):
>        a = "abc"
>    a = "Done"
>
>import sys
>sys.settrace(tracer)
>foo()

I'll give it a try,

Thanks,
Ype

[Jython-dev] shutdowns in jython / atexit

[<bc...@wo...>]

[From a bug report]

>Bugs item #476772, was opened at 2001-10-31 06:27
>You can respond by visiting: 
>http://sourceforge.net/tracker/?func=detail&atid=112867&aid=476772&group_id=12867
>
>Category: Core
>Group: None
>Status: Open
>Resolution: None
>Priority: 5
>Submitted By: Itamar Shtull-Trauring (itamar)
>Assigned to: Nobody/Anonymous (nobody)
>Summary: shutdowns in jython / atexit
>
>Initial Comment:
>In cpython I can use signals to detect when python is
>shut down, but there is no way to do this in jython.
>It'd be nice to be able to do this using atexit module,
>but it doesn't work in jython.
>
>It should be possible to make atexit work by making
>jython use Java 1.3's Runtime.addShutdownHook to
>execute sys.exitfunc. Not sure what to do about older
>versions of Java, however.

I disagree about calling sys.exitfunc from the JVM shutdown hooks. I
think sys.exitfunc should be called from 

- the execption handling in Py.maybeSystemExit()
- from the end of jython.java and 
- from a new method on PythonInterpreter which an application that 
  embed jython can call (as the reverse of initialize).

That will make it available on all JVM's and it will match CPython (I
believe).

Any comments to such a scheme?

regards,
finn

Re: [Jython-dev] Q: how to run cleanup method when interpreterexits?

[<bc...@wo...>]

[Samuele]

>Maybe I'm just very confused, but despite of the News
>jython does not support effectively atexit .

That's true.

The "atexit" module was added because it is imported by the "threading"
module and the "threading" module contained so much usefull stuff that
the missing daemon thread cleanup was a minor issue.

At the moment, "atexit" is completely useless.

[I guess there is a lesson about which CPython modules we include with
Jython hidden here. Unfortunately I don't have a clue of what the lesson
was.]

regards,
finn

Re: [Jython-dev] Jython code coverage testing: setline() calls

[<bc...@wo...>]

[Ype Kingma]

>Hello jythoneers,
>
>I've been using pyunit for a while in jython and would like to 
>have code coverage reports from some of the tests.
>I started from Skip Montanaro's code coverage module
>(http://musi-cal.mojam.com/~skip/python/trace.py) and found out
>that the biggest hurdle probably is obtaining line numbers that have code
>assigned to them. He uses the CPython parser to get actual lines
>from the SETLINENO instructions in CPython byte code.
>That won't work for jython, so I started piecing together a java class
>file disassembler in jython to find out how it's done in jython class files.
>I got to the point of disassembling individual
>instructions more or less correctly, and then I found out
>that setline() is called only at the beginning of each function,
>which is a showstopper in this case.

For dynamicly compiled python code, a call to PyFrame.setline(int) is
generated for each statement (including expr_stmt). Maybe the call to
setline is missing for some statement types, but it can't be that many.

>There is also a line number table, but I wouldn't know how
>to interpret this while the JVM is executing.

The line number table maps java bytecode PC addresses to source line
numbers, but as you say, it is useless while running since we don't have
access to the java bytecode PC.

But if all you need to know is whether a line have some executable
python code, then the line number table in the $py.class file can answer
that with the same certainty as the setline() method calls can.

Each time a call to PyFrame.setline() is generated, we also add a line
number entry to the line number table. See the Code.setline() for
details.

>  http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/jython/jython/org/python/compiler/Code.java?annotate=2.4#517

>Some questions:
>- Is there an option for jythonc to have setline() called for each line
>  of jython code? I checked the sources a bit and even found the
>  point at which the setline call is emitted for a jython function,
>  but I got no further than that.

For jythonc? No.

>- Is line tracing implemented in jython? Function tracing works fine
>  in the profiler, but line tracing is a bit more involved, and I did
>  not check whether this can actually be done in jython.

Line tracing is implemented in dynamic compilation. Unless you have
discovered a bug of course. This works for me:

def tracer(frame, why, arg):
    print why, frame.f_lineno, arg
    return tracer

def foo():
    for i in range(10):
        a = "abc"
    a = "Done"

import sys
sys.settrace(tracer)
foo()


regards,
finn

Re: [Jython-dev] Safe Jython?

[<bc...@wo...>]

[M. Ranganathan]

>Can I restrict an imbedded jython interpreter in the same fashion
>that I can restrict an imbedded python interpreter. Do I have
>to resort to the java.security properties to do this? If there is a
>better way, I would appreciate a tip on how to proceed.

The best way (IMO) is to use a security manager. Simply because the java
security manager feature have been put through more testing and review
than any similar feature we could add to jython.

>Is there some way to restrict loop execution in a jython program
>to a maximum number of iterations?

Generally no. Maybe the trace hooks:

  http://www.python.org/doc/current/lib/module-sys.html#l2h-250

can be abused for this purpose, but I'm sure an evil program can also
defeat that.

Security (in any form) was never in the design spec for jython. Because
of that, jython will never (IMO) be a secure environment.

regards,
finn

Re: [Jython-dev] Enhancement Applet ClassLoader

[<bc...@wo...>]

[Sells, Fred]

>I am using jython to customize the behavior of an applet written in Java.
>
>I am loading jython source from a URL and use that source to define 90% of
>the appearance of the applet.
>
>I decided to load the source (rather than go the jythonc route) because the
>file is much smaller and I have the pazazz (sp?) of being able to modify
>applet appearance on the fly.  The speed and capability is a big hit with
>management.
>
>I read several Jython files and .exec() them in the same instance of the
>Interpreter.  these files are jython source but the files the client edits
>hide that fact.  They contain no "class" or "def" statements, nor do they
>import anything.
>
>In order to make this work I had to grant permissions to the applet that It
>would not normally (default) need. 

Right, you will need to do that.

>Not a politically acceptable thing to do in today's world.
>
>I do not know the internals of Jython or Java  enough to know if there is a
>potential solution for a future release.  Things I would consider as a
>solution, include:
>     the ability to "exec()" without requiring a change to applet security
>     or
>     the abiltiy to execute conventional C-python (.pyc) byte-code in the
>Java environment.

The solution is to make an engine in jython which performs a traditional
interpretation of bytecodes. It could be .pyc byte-codes or it could be
a homebrew set of byte-codes. Running python fully interpreted would:

- be very slow. Obviously I don't know how slow, but I guess at least
  10 times slower than current execution.
- not allow any subclassing of java classes or interfaces. 

>The "big picture" reason to consider such a chage to the core product 

It will *never* be a change. It might be an *addition*.

>is
>that applets (and webstart) applications can then extend their features with
>jython; allowing the core product to be very generic in Java and the
>deployed implementation to be customized in Jython.

For application where the restrictions above isn't a problem, a fully
interpreted engine would be valuable. It is a big task to begin, so
please don't hold your breath.

>That's just my opinion, I could be wrong.

But you are not.

regards,
finn

Re: [Jython-dev] Query

[<bc...@wo...>]

[Abraham, Shabu]

>Dear Sir,
>
>When I try to compile the following lines in java script , before making an
>instance of PythonInterpreter, 
>
>Properties props = new Properties();
>props.setProperty("python.path", "C:/dist/comjob:lib.custom");
>PythonInterpreter.initialize(System.getProperties(), props, new String[]
>{""});
>
>I get an error that intialize(Properties, Properties, String[]) method not
>found in PythonInterpreter.

Maybe you forgot to import the java.util package?

  import java.util.*;

Without java.util and using the jikes compiler, I get this error message
(among other errors):

>     8. PythonInterpreter.initialize(System.getProperties(), props, new String[] {""});
>*** Error: No match was found for method "initialize(java.util.Properties, Properties, java.lang.String[])".

It kind of look like the message you describe.

regards,
finn