jython-dev — Discussion of developing and enhancing Jython itself(!) (not how to use Jython)

[Jython-dev] PHP to Jython

[rmr <roc...@ho...>]

Does a PHP to JYTHON conversion utility exist??

Thanks

Re: [Jython-dev] sre_constants.MAGIC

[Ben B. <be...@ac...>]

> - Depending completely on modules from CPython is also wrong. One of the
> problems is the socket modules that gets imported by other modules from
> Lib. Running the SimpleHTTPServer script f.ex will pick up the socket.py
> from CPython and that can't work with Jython.

I should add that socket.py is one of the modules that *is* included in 
debian jython's Lib/ directory (see Kevin's list), so AFAIU jython should 
pick up the correct socket.py.

Ben.

-- 

Ben Burton
be...@ac...  |  ba...@de...
http://baasil.humbug.org.au/bab/
Public Key: finger ba...@de...

A man cannot be too careful in the choice of his enemies.
	- Oscar Wilde, The Picture of Dorian Gray

Debian jython packaging (was: [Jython-dev] sre_constants.MAGIC)

[Ben B. <be...@ac...>]

> I think Ben is on this list already.

Ben is on this list but is kind of busy and hadn't noticed this thread until 
just now. :)

> - Searching for a different CPython release is wrong. It must be 2.1.1.

I'll fix this.

> and I think dman's suggestion was correct. The CPython modules have to
> be carefully duplicated as done by installer/mklist.py.

The procedure currently used by the debian package is:

- Have a python path that places jython's Lib/ before the CPython directory;
- Only duplicate modules that require jython-specific patches, and place 
these patches in jython's Lib/

Is this doomed to failure, or have I just missed some jython-specific patches 
in the debian packaging?  You can see the (small) list of duplicated modules 
in the package contents list that Kevin posted.

Ben.

-- 

Ben Burton
be...@ac...  |  ba...@de...
http://baasil.humbug.org.au/bab/
Public Key: finger ba...@de...

I go into a real vulnerable side of myself. That's where I am finding
a lot of hidden stuff, as a woman afraid to be vulnerable, because I think
I will be weak, thinking I'll be taken advantage of, thinking I won't know
where to draw the line. But I am finding that vulnerability gives me great
strength, because you're not hiding anymore!
	- Tori Amos

Re: [Jython-dev] sre_constants.MAGIC

[Kevin B. <kb...@ca...>]

Finn Bock wrote:
> 
> [Kevin Butler]
>
> WTF? Doesn't the debian package of jython include the CPython modules in
> its ./Lib directory? Your report about sre_constants.MAGIC indicate that
> it doesn't.

I've got jython deb version 2.1-alpha3-5.

> Does anyone have a list of files in the debian jython release? I could
> only find a .deb file and don't know how to list its content.

dpkg -L jython   (at least for an installed .deb)

output attached.

> Maybe it would be helpful if we improve mklist.py to be platform
> portable. Right now it is mainly a script that works on my machine.

:-)

kb

Re: [Jython-dev] sre_constants.MAGIC

[<bc...@wo...>]

[Kevin Butler]

>Ahh, it looks like the magic is in /usr/bin/jython, which chooses the
>highest-numbered CPYTHON_LIB:
>
>CPYTHON_LIB=
>for pydir in /usr/lib/python?.?; do
>	if [ -d "$pydir" ]; then
>		if [ "$CPYTHON_LIB" \< ":$pydir" ]; then
>			CPYTHON_LIB=":$pydir"
>		fi
>	fi
>done

WTF? Doesn't the debian package of jython include the CPython modules in
its ./Lib directory? Your report about sre_constants.MAGIC indicate that
it doesn't.

Does anyone have a list of files in the debian jython release? I could
only find a .deb file and don't know how to list its content.

>I couldn't see this code in Jython CVS - is this created by my Debian 
>jython package? If so, anything else I should tell the Debian jython
>package maintainer?

I think Ben is on this list already.

Well,

- Searching for a different CPython release is wrong. It must be 2.1.1.
- Depending completely on modules from CPython is also wrong. One of the
problems is the socket modules that gets imported by other modules from
Lib. Running the SimpleHTTPServer script f.ex will pick up the socket.py
from CPython and that can't work with Jython.

It seems the search for CPython was added based on this bugreport. 

   http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=118429

and I think dman's suggestion was correct. The CPython modules have to
be carefully duplicated as done by installer/mklist.py.

Maybe it would be helpful if we improve mklist.py to be platform
portable. Right now it is mainly a script that works on my machine.

regards,
finn

Re: [Jython-dev] new system and environ support in javaos

[<bc...@wo...>]

[me]

> Ok, as long as the os.error contain all the information from the java
> exception. Including information about the type of exception
> (IOException or SecurityException) and the exception message.
> 
> The patch below doesn't and it certainly should.

[kevin]

>In general I agree.  The reasons I didn't in this case were:
>
>1- I didn't know if we wanted to have a standard Python module expose Java exception info
>2- the errors weren't giving any useful information 
>
>And given your opinion, #1 becomes a non-issue, and #2 could be a 
>platform-specific thing, so it probably would be good to include it...

In win2k the message is sometimes very usefull:

Jython 2.1b1 on java1.4.0-beta3 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> import java
>>> java.lang.Runtime.getRuntime().exec("foobar")
Traceback (innermost last):
  File "<console>", line 1, in ?
java.io.IOException: CreateProcess: foobar error=2

where error #2 means 

   #define ENOFILE         2       /* No such file or directory */

If we get a bugreport, we want to know the actual error number.

>Is there a way to easily incorporate a Java exception into an OSError?  
>Should I just include the .toString(), 

yes.

>or try to incorporate the stack trace info?

No.

>Should errors.java include a "JavaError" as a subclass of "OSError"?

No.

>> Do you have a SF account? Make one and I'll add you as committer. Then
>> you don't have to make excuses every time.
>
>I'm 'kevinbutler'

Added. Let me know if I missed to check some access-right boxes.

>> >! pardir = '..'
>> ..
>> I liked the old defensive comment.
>
>:-)  We can put it back in, but nobody sees it unless they look at the source 

True. I just hope that users that find the value wrong, will try and
hunt down where the value is defined. Maybe even suggest patches.

>We could be really defensive with some assertions/warnings about 
>"getParentFile().getCanonicalPath()", etc., but we'd want to have them
>disable-able...

Or more if-statements that test for os.name and python.os.

regards,
finn

Re: [Jython-dev] zxJDBC into Jython

[<bc...@wo...>]

[brian]

>Since I'm not sure who's interested, I thought I'd just mail the group.
>
>I finally rolled zxJDBC into the Jython mainline.  

And thank you for that.

>I wanted to bring to your attention the following:
>
>* I added a top-level directory 'com'
>
>* I added two new files to 'Lib', dbexts.py and isql.py.
>
>* I made some changes to build.xml.  In particular:
>  - I moved the reading of ant.properties to the beginning so local vars
>can override global ones.
>  - I added additional conditional compilations of zxJDBC datahandlers.
>If you do not have the correct JDBC drivers (such as MySQL, Oracle,
>Postgresql, ...) they will not compile.  This really has no affect for
>general development, but they should get compiled for releases.  I can
>help gather the JDBC drivers as needed.  If there is a licensing issue
>with this, I can make them available outside of the core release.

Some of the drives must be available freely on the net. Tell us about
the links you already have.

>  - I added five new properties to ant.properties (again, these can be
>safely ignored) as referenced in the 'main.classpath' path element.

Please update the Doc/compile.ht file with a list of the new properties.

>* I have numerous pyunit tests that I have not checked in yet.  I'm
>deciding on the best way to handle it.  One problem with developing
>across so many db's is testing them since you have to have running
>instances.  For the time being I'll handle testing manually.

We really should get our testing more together. It seems like we have to
run the tests in:

- Lib/test/testall.py
- bugtests/driver.py
- Some of the tests in CPython's Lib/tests
- and now some zxJDBC tests.

>* I have not checked in any documentation.  I would like to re-style it
>to fit into whatever flavor the rest of the Jython documentation uses.
>
>* In installer/mklist.py, the 'javafiles' filter does not include the
>*.dat nor the new *.properties files, should it? 

Yes, either that or maybe add a new 'zxJDBCfiles' filter.

>On this note, I added
>the new top-level 'com' package and its subdirectories as needed.  I did
>not check in an update version of the generated files in that directory,
>should I have?

Yes, but only if the generated liftoff.filelist does not deviate severly
from the CVS version. At least you should try to run it, because your
checkin contains a little typo. 

>* Please let me know if something doesn't compile.

I get a warning with jikes-1.15:

    [javac] Compiling 217 source files to D:\jython\CVS\build
    [javac]
    [javac] Issued 1 semantic warning compiling
"D:/jython/CVS/com/ziclix/python/sql/PyCursor.java":
    [javac]
    [javac]     59.             } catch (Exception e) {}
    [javac]                                  <--------->
    [javac] *** Caution: This try block cannot throw a "checked
exception" (JLS section 14.7) that can be caught here. You may have
intended to catch a RuntimeException instead of an Exception.

>I have only compiled
>under 1.3 and 1.4, so I hope that it works on an earlier VM (is anyone
>still using one?).

Yes. It is fundamental requirement for me that jython also compiles and
runs on java1. I don't demand that all new feature is backward
compatible all the way down to JDK1.1 but the jython sources should
still compile cleanly on JDK1.1.

The <exclude name=".."> thing does not work for me. If I rewrite the
exclude to 
   <javac .... excludes="${exclude.java2.files}">
It compiles again for JDK1.1.  I'm using ant-1.2.

Some of the zxJDBC sources depends on java2 collections (like Map,
HashSet and ArrayList)and JDBC20DataHandler depends on Types.CLOB and
getBigDecimal() and getArray().

It is ok if zxJDBC require java2, then it only have to be part of the
exlude.java2.files property in build.xml.

>* I removed the license information from within all the files.  I kept
>only a personal copyright.

Very good.

In order to avoid a prolonged discussion about the original license of
zxJDBC, you might want to add a note on the license webpage. Or we could
add a LICENSE.txt file to the com/ziclix directory. Thoughs?

regards,
finn

Re: [Jython-dev] new system and environ support in javaos

[Kevin B. <kb...@ca...>]

Finn Bock wrote:
> 
> [Samuele Pedroni]
> 
> > Some notes:
> >
> > - if something goes wrong with Runtime.exec a java exception is thrown,
> >  I think this should be somehow wrapped in an os.error. Opinions?
> 
> [Kevin Butler]
> 
> >Agreed.
> 
> Ok, as long as the os.error contain all the information from the java
> exception. Including information about the type of exception
> (IOException or SecurityException) and the exception message.
> 
> The patch below doesn't and it certainly should.

In general I agree.  The reasons I didn't in this case were:

1- I didn't know if we wanted to have a standard Python module expose Java exception info
2- the errors weren't giving any useful information 

And given your opinion, #1 becomes a non-issue, and #2 could be a platform-specific thing, so it probably would be good to include it...

Is there a way to easily incorporate a Java exception into an OSError?  
Should I just include the .toString(), or try to incorporate the stack trace info?  
Should errors.java include a "JavaError" as a subclass of "OSError"?

[snip: platform-specifics]
> Thats why I haven't touched added this feature myself. It is so easy to
> get working for oneself but so hard to get it working correctly for all.

Java: Write once, debug everywhere?

> >As is becoming all too frequent, here's a patch (but this is the last one for a
> >long time...  *wink*),
> 
> Do you have a SF account? Make one and I'll add you as committer. Then
> you don't have to make excuses every time.

I'm 'kevinbutler'

I think the peer review is helpful, so I'd like to keep submitting significant changes like this last one to the list...

> >! pardir = '..'
> 
> You think that is right on macs and AS/400? 

Nope.  :-)

Looks like Python's os.py uses the following for Mac:

    curdir = ':'; pardir = '::'; sep = ':'; pathsep = '\n'
    defpath = ':'

No idea for other exotic (to me) platforms...

> Maybe it actually is, but I
> doubt it. I liked the old defensive comment.

:-)  We can put it back in, but nobody sees it unless they look at the source 

We could be really defensive with some assertions/warnings about "getParentFile().getCanonicalPath()", etc., but we'd want to have them disable-able...

Makes me want an install-time OS-chooser function that enables/disables particular functions & warns about conflicts! Isn't that exactly what Java is supposed to avoid?

> >!         except UnboundLocalError:
> >!             # 'value' is unbound because we didn't find '=' on first line
> 
> Maybe it is just my personal preference, but I would prefer an explicit
> test instead of catching UnboundLocalError.

I prefer an explicit test, but weighed against the cost of executing the check in each iteration...

I guess we could structure it like this:

           lines = self._readLines( p.getInputStream() )
           if '=' not in lines[0]:
  	       raise OSError( 0,
  		   "Command did not print environment.\n"
  		   "Command=%s\nOutput=%s\n" % (
  		   self.getEnv, '\n'.join( lines )
  		   ))
           for line in lines:
		...

That feels better.  :-)

[Samuele Pedroni Re: shell os causes failure]
> I got this problem running the CPython 2.1 test suite, it uses
> tempfile and the failing os.environ made some tests fail.

Ouch.

Looks like the code was probably:

    # set up a bunch of default 'attempdirs'
    # ...
    # then try to get a couple more from the environment
    for envname in 'TMPDIR', 'TEMP', 'TMP':
        if os.environ.has_key(envname):
            attempdirs.insert(0, os.environ[envname])

Pretty innocuous, especially because it already had various standard and os-specific "attempdirs" in place.

Which makes it a good case for your preference:

> So why I would prefer falling back to NullEnv + warning.

I wouldn't have a problem issuing a warning - well, I'd have to figure out how to use warnings.py  ;-)

What category would be appropriate? etc?

kb

Re: [Jython-dev] sre_constants.MAGIC

[Kevin B. <kb...@ca...>]

Finn Bock wrote:
> 
> [Kevin Butler]
> >
> >Instead, I just modified my python.path in /usr/share/jython/registry to point to
> >python2.1
> >instead of letting jython (somehow?) find python2.2...
> 
> How indeed! It should not attempt do that automaticly. Maybe it is done
> by site.py that is reading .pth files?

It was finding the python2.2/site.py file, for some reason (python2.2 was in the sys.path, though I had no python.path in the registry).

Ahh, it looks like the magic is in /usr/bin/jython, which chooses the highest-numbered CPYTHON_LIB:

CPYTHON_LIB=
for pydir in /usr/lib/python?.?; do
	if [ -d "$pydir" ]; then
		if [ "$CPYTHON_LIB" \< ":$pydir" ]; then
			CPYTHON_LIB=":$pydir"
		fi
	fi
done

---

I couldn't see this code in Jython CVS - is this created by my Debian jython package? If so, anything else I should tell the Debian jython package maintainer?

kb

Re: [Jython-dev] new system and environ support in javaos

[Samuele P. <ped...@bl...>]

> >The defaults are python.enviroment = shell and if unknown then unix.
> >A fragile mixture although it is a good way to get things tested.
> 
> Yeah, it is clearly not any of my programs that is failing <wink>, so I
> can afford to be aggresive.
> 
> >In particular if I'm writing an application that should run under
> >jython under a potentially unknown os, now I should care
> >about python.enviroment settings.
> >2nd: an old straightforward application importing os can fail.
> 
> Just importing os? Surely it must also access os.environ before it
> fails.
> 
> >I imagine a possible compromise would be to issue a warning
> >and not raise an expception in this particular case of execute usage.
> 
> A warning is certainly better than a silent fallback to NullEnv. 
> 

Oops I skipped over the important point, the problem is not
importing os - environ is lazy and I know- but using a CPython module that
uses os.environ which worked fine with the empty environ
(e.g. tempfile) and now can fail miserably if os.environ
fails miserably.

I got this problem running the CPython 2.1 test suite, it uses
tempfile and the failing os.environ made some tests fail.

So why I would prefer falling back to NullEnv + warning.

Samuele.

Re: [Jython-dev] new system and environ support in javaos

[<bc...@wo...>]

>> >I'm inclined to throw an OS exception here, too - let the user know that
>things
>> >aren't working as expected, and if the user wants to change configuration to
>use
>> >NullEnv, that's great.
>>
>> I agree.
>>

[Samuele]

>I'm skeptical about this. You're making a strong point about the user expecting
>os.environ to work. Given my first experience I would expect nothing :).

Hehehe.

>The defaults are python.enviroment = shell and if unknown then unix.
>A fragile mixture although it is a good way to get things tested.

Yeah, it is clearly not any of my programs that is failing <wink>, so I
can afford to be aggresive.

>In particular if I'm writing an application that should run under
>jython under a potentially unknown os, now I should care
>about python.enviroment settings.
>2nd: an old straightforward application importing os can fail.

Just importing os? Surely it must also access os.environ before it
fails.

>I imagine a possible compromise would be to issue a warning
>and not raise an expception in this particular case of execute usage.

A warning is certainly better than a silent fallback to NullEnv. 

Would it be worth making a difference of defensive calls? Letting the 3
argument os.environment.get() and has_key() fail silently but throwing
exceptions for (most of?) the rest? Probably not!

regards,
finn

Re: [Jython-dev] sre_constants.MAGIC

[<bc...@wo...>]

[Kevin Butler]
>
>> (somehow my Linux jython
>> install is now giving 'AssertionError: SRE module mismatch', and I haven't chased
>> that down yet).
>
>Looks like the reason is that I've installed python2.2 on this machine, so now jython
>is using the python2.2 sre, etc. modules, instead of python2.1 which I had been using.
>
>This causes a problem because the sre_constants 'MAGIC' number was bumped from 20010320
>to 20010701.
>
>Seems like jython should probably ship with an sre_constants.py that is compatible with
>_sre?

It does. Jython-21a3 contains a sre magic of 20010320. The CVS version
does not contain copies of CPython modules (except the module that we
have had to modify to make them work with jython).

>I assume the right thing to do for now would be to enhance the jython _sre module to be
>20010701-compatible.

Absolutely not. Jython-2.1 work with the modules form CPython-2.1.1. It
is not tested or intended to work with CPython-2.2 modules.

20010701-compatibility should be added after the release of jython-2.1
final.

>Instead, I just modified my python.path in /usr/share/jython/registry to point to
>python2.1 

And that is the right way to run the CVS version of jython. I do it by
having this in my %HOME%/.jython file:

    python.path=d:\\python\\Python-2.1.1\\Lib

>instead of letting jython (somehow?) find python2.2...

How indeed! It should not attempt do that automaticly. Maybe it is done
by site.py that is reading .pth files?

regards,
finn

Re: [Jython-dev] new system and environ support in javaos

[Samuele P. <ped...@bl...>]

> >I'm inclined to throw an OS exception here, too - let the user know that
things
> >aren't working as expected, and if the user wants to change configuration to
use
> >NullEnv, that's great.
>
> I agree.
>

I'm skeptical about this. You're making a strong point about the user expecting
os.environ to work. Given my first experience I would expect nothing :).

The defaults are python.enviroment = shell and if unknown then unix.
A fragile mixture although it is a good way to get things tested.
In particular if I'm writing an application that should run under
jython under a potentially unknown os, now I should care
about python.enviroment settings.
2nd: an old straightforward application importing os can fail.

I imagine a possible compromise would be to issue a warning
and not raise an expception in this particular case of execute usage.

regards.

Re: [Jython-dev] new system and environ support in javaos

[<bc...@wo...>]

[Samuele Pedroni]

> Some notes:
>
> - if something goes wrong with Runtime.exec a java exception is thrown,
>  I think this should be somehow wrapped in an os.error. Opinions?

[Kevin Butler]

>Agreed.

Ok, as long as the os.error contain all the information from the java
exception. Including information about the type of exception
(IOException or SecurityException) and the exception message.

The patch below doesn't and it certainly should.

>> - in particular if something goes wrong when populating environ, we should
>>  decide whether to throw an exception or gracefully fall back to a NullEnv.
>
>I'm inclined to throw an OS exception here, too - let the user know that things
>aren't working as expected, and if the user wants to change configuration to use
>NullEnv, that's great.

I agree.

>Should we be trying to provide some sort of meaningful 'errno' codes?  I've just
>been raising OSError( 0...) like the other javaos functions...

Good enough. The IOException thrown from exec() sometimes contain the
operating system error code (as text). 

>> - environ populating does not work (as far as my installation is concerned)
>> with win 98:
>>   "command.com /c set should be invoked, "command /c set" fails.
>
>I get similar behavior on Win95/JDK1.1.8.  (had to dig up my Win95 laptop & install
>a JDK (figured I'd install one I didn't have elsewhere), replace the old jpython
>installation w/ a new jython version, & get the new javaos.py...  Speedy it was
>not...)

Thats why I haven't touched added this feature myself. It is so easy to
get working for oneself but so hard to get it working correctly for all.

>As is becoming all too frequent, here's a patch (but this is the last one for a
>long time...  *wink*),

Do you have a SF account? Make one and I'll add you as committer. Then
you don't have to make excuses every time.

>! pardir = '..'

You think that is right on macs and AS/400? Maybe it actually is, but I
doubt it. I liked the old defensive comment.

>!         except UnboundLocalError:
>!             # 'value' is unbound because we didn't find '=' on first line

Maybe it is just my personal preference, but I would prefer an explicit
test instead of catching UnboundLocalError.

regards,
finn

[Jython-dev] [ jython-Patches-461151 ] Simple java class import

[<no...@so...>]

Patches item #461151, was opened at 2001-09-13 02:04
You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=312867&aid=461151&group_id=12867

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Otmar Humbel (otmarhumbel)
>Assigned to: Samuele Pedroni (pedronis)
Summary: Simple java class import

Initial Comment:

The following lines added just at the beginning of 
method importFromAs in org/python/core/imp.java (from 
the 2.0 codebase) would make the import of single java 
classes a bit more tolerant, especially if the package 
manager encounters a 'bad' .jar File:


public static void importFromAs(String mod, String[] 
names, String[] asnames, PyFrame frame) {
  if ( names.length==1 && names.length==names.length 
&& asnames[0].equals(names[0]) ) {
    // this is a candidate for simple java class import
    String packageName = mod;
    String fullClassName = packageName + "." + names
[0];
    try {
      Class.forName( fullClassName );
      PySystemState.add_package( packageName );
    } catch( Throwable t ) {}
  }
  // rest of method left untouched
  // ...
}



----------------------------------------------------------------------

You can respond by visiting: 
http://sourceforge.net/tracker/?func=detail&atid=312867&aid=461151&group_id=12867

Re: [Jython-dev] sre_constants.MAGIC

[Kevin B. <kev...@bi...>]

Kevin Butler wrote:

> (somehow my Linux jython
> install is now giving 'AssertionError: SRE module mismatch', and I haven't chased
> that down yet).

Looks like the reason is that I've installed python2.2 on this machine, so now jython
is using the python2.2 sre, etc. modules, instead of python2.1 which I had been using.

This causes a problem because the sre_constants 'MAGIC' number was bumped from 20010320
to 20010701.

Seems like jython should probably ship with an sre_constants.py that is compatible with
_sre?

I assume the right thing to do for now would be to enhance the jython _sre module to be
20010701-compatible.

Instead, I just modified my python.path in /usr/share/jython/registry to point to
python2.1 instead of letting jython (somehow?) find python2.2...

kb

Re: [Jython-dev] new system and environ support in javaos

[Kevin B. <kev...@bi...>]

Samuele Pedroni wrote:

> Some notes:
>
> - if something goes wrong with Runtime.exec a java exception is thrown,
>  I think this should be somehow wrapped in an os.error. Opinions?

Agreed.

> - in particular if something goes wrong when populating environ, we should
>  decide whether to throw an exception or gracefully fall back to a NullEnv.

I'm inclined to throw an OS exception here, too - let the user know that things
aren't working as expected, and if the user wants to change configuration to use
NullEnv, that's great.

Should we be trying to provide some sort of meaningful 'errno' codes?  I've just
been raising OSError( 0...) like the other javaos functions...

> - environ populating does not work (as far as my installation is concerned)
> with win 98:
>   "command.com /c set should be invoked, "command /c set" fails.

I get similar behavior on Win95/JDK1.1.8.  (had to dig up my Win95 laptop & install
a JDK (figured I'd install one I didn't have elsewhere), replace the old jpython
installation w/ a new jython version, & get the new javaos.py...  Speedy it was
not...)

>  I think java version is also concerned here regarding to which extensions are
> tried
>   and which os calls are made. I used jdk 1.3.0.
>   Anyone did try this already with different results?

As is becoming all too frequent, here's a patch (but this is the last one for a
long time...  *wink*), and now __test() works on NT and 95 (somehow my Linux jython
install is now giving 'AssertionError: SRE module mismatch', and I haven't chased
that down yet).

kb

$ cvs diff -c javaos.py
Index: javaos.py
===================================================================
RCS file: /cvsroot/jython/jython/Lib/javaos.py,v
retrieving revision 2.11
diff -c -r2.11 javaos.py
*** javaos.py 2001/11/15 08:17:17 2.11
--- javaos.py 2001/11/20 05:49:10
***************
*** 23,29 ****
             "defpath", "name"]

  import java
! from java.io import File, BufferedReader, InputStreamReader
  import javapath as path
  from UserDict import UserDict
  import string
--- 23,29 ----
             "defpath", "name"]

  import java
! from java.io import File, BufferedReader, InputStreamReader, IOException
  import javapath as path
  from UserDict import UserDict
  import string
***************
*** 35,44 ****

  error = OSError

! name = 'java' # descriminate based on JDK version?
  curdir = '.'
! pardir = '..' #This might not be right...
! #curdir, pardir??
  sep = java.io.File.separator
  altsep = None
  pathsep = java.io.File.pathSeparator
--- 35,43 ----

  error = OSError

! name = 'java' # discriminate based on JDK version?
  curdir = '.'
! pardir = '..'
  sep = java.io.File.separator
  altsep = None
  pathsep = java.io.File.pathSeparator
***************
*** 234,241 ****
              env = self._formatEnvironment( self.environment )
          else:
              env = None
!         p = java.lang.Runtime.getRuntime().exec( shellCmd, env )
!         return p

      ########## utility methods
      def _readLines( self, stream, func=None ):
--- 233,243 ----
              env = self._formatEnvironment( self.environment )
          else:
              env = None
!         try:
!             p = java.lang.Runtime.getRuntime().exec( shellCmd, env )
!             return p
!         except IOException:
!             raise OSError( 0, "Failed to execute command: %s" % shellCmd )

      ########## utility methods
      def _readLines( self, stream, func=None ):
***************
*** 269,286 ****
          This allows multi-line variables as long as subsequent lines do
          not have '=' signs.
          """
!         p = self.execute( self.getEnv )
!         env = {}
!         key = 'firstLine' # in case first line had no '='
!         for line in self._readLines( p.getInputStream() ):
!             try:
!                 i = line.index( '=' )
!                 key = self._keyTransform(line[:i])
!                 value = line[i+1:]
!             except ValueError:
!                 # found no '=', so this line is part of previous value
!                 value = '%s\n%s' % ( value, line )
!             env[ key ] = value
          return env

  def _getOsType( os=None ):
--- 271,295 ----
          This allows multi-line variables as long as subsequent lines do
          not have '=' signs.
          """
!         try:
!             p = self.execute( self.getEnv )
!             env = {}
!             for line in self._readLines( p.getInputStream() ):
!                 try:
!                     i = line.index( '=' )
!                     key = self._keyTransform(line[:i])
!                     value = line[i+1:]
!                 except ValueError:
!                     # found no '=', so this line is part of previous value
!                     value = '%s\n%s' % ( value, line )
!                 env[ key ] = value
!         except UnboundLocalError:
!             # 'value' is unbound because we didn't find '=' on first line
!             raise OSError( 0,
!                 "Command did not print environment.\n"
!                 "Command=%s\nOutput=%s\n" % (
!                 self.getEnv, line
!                 ))
          return env

  def _getOsType( os=None ):
***************
*** 312,318 ****
      if osType == "nt":
          return _ShellEnv( ["cmd", "/c"], "set", string.upper )
      elif osType == "dos":
!         return _ShellEnv( ["command", "/c"], "set", string.upper )
      elif osType == "mac":
          return _NullEnv()
      else: # osType == "unix":
--- 321,327 ----
      if osType == "nt":
          return _ShellEnv( ["cmd", "/c"], "set", string.upper )
      elif osType == "dos":
!         return _ShellEnv( ["command.com", "/c"], "set", string.upper )
      elif osType == "mac":
          return _NullEnv()
      else: # osType == "unix":
***************
*** 348,356 ****
          # should print PATH (on NT)
          ("echo PATH=%PATH%", "(PATH=.*;.*)|(PATH=%PATH%)"),
          # should print 'testKey=%testKey%' on NT before initialization,
          # and 'testKey=testValue' after
          ("echo %s=%%%s%%" % (key,key),
!                 "(%s=%%%s%%)|(%s=%s)" % (key, key, key, value)),
          # should print PATH (on Unix)
          ( "echo PATH=$PATH", "PATH=.*" ),
          # should print 'testKey=testValue' on Unix after initialization
--- 357,366 ----
          # should print PATH (on NT)
          ("echo PATH=%PATH%", "(PATH=.*;.*)|(PATH=%PATH%)"),
          # should print 'testKey=%testKey%' on NT before initialization,
+         # should print 'testKey=' on 95 before initialization,
          # and 'testKey=testValue' after
          ("echo %s=%%%s%%" % (key,key),
!                 "(%s=)" % (key,)),
          # should print PATH (on Unix)
          ( "echo PATH=$PATH", "PATH=.*" ),
          # should print 'testKey=testValue' on Unix after initialization
***************
*** 359,369 ****
          # should output quotes on NT but not on Unix
          ( 'echo "hello there"', '"?hello there"?' ),
          # should print 'why' to stdout.
!         ( r'''python -c "import sys;sys.stdout.write( 'why\n' )"''', "why" ),
          # should print 'why' to stderr, but it won't right now.  Have
          # to add the print to give some output...empty string matches every
          # thing...
!         ( r'''python -c "import sys;sys.stderr.write('why\n');print " ''', "" )
          ]

      assert not environ._populated, \
--- 369,379 ----
          # should output quotes on NT but not on Unix
          ( 'echo "hello there"', '"?hello there"?' ),
          # should print 'why' to stdout.
!         ( r'''jython -c "import sys;sys.stdout.write( 'why\n' )"''', "why" ),
          # should print 'why' to stderr, but it won't right now.  Have
          # to add the print to give some output...empty string matches every
          # thing...
!         ( r'''jython -c "import sys;sys.stderr.write('why\n');print " ''', "" )
          ]

      assert not environ._populated, \
***************
*** 403,405 ****
--- 413,431 ----
              "expected environment to have PATH attribute " \
              "(this may not apply to all platforms!)"

+     # attempt to get an environment with a shell that is not startable
+     try:
+         print _ShellEnv( ["badshell", "-c"], "set" ).environment
+     except OSError, val:
+         assert val.strerror.startswith( "Failed to execute" )
+         print "caught correct error for bad shell"
+
+     # attempt to get an environment with a command that does not print an
environment
+     try:
+         se2 = _getShellEnv()
+         se2.getEnv="echo This command does not print environment"
+         print se2.environment
+     except OSError, val:
+         assert val.strerror.startswith( "Command did not" )
+         print "caught correct error for bad command"
+

[Jython-dev] zxJDBC into Jython

[brian z. <bz...@zi...>]

Since I'm not sure who's interested, I thought I'd just mail the group.

I finally rolled zxJDBC into the Jython mainline.  I wanted to bring to
your attention the following:

* I added a top-level directory 'com'

* I added two new files to 'Lib', dbexts.py and isql.py.

* I made some changes to build.xml.  In particular:
  - I moved the reading of ant.properties to the beginning so local vars
can override global ones.
  - I added additional conditional compilations of zxJDBC datahandlers.
If you do not have the correct JDBC drivers (such as MySQL, Oracle,
Postgresql, ...) they will not compile.  This really has no affect for
general development, but they should get compiled for releases.  I can
help gather the JDBC drivers as needed.  If there is a licensing issue
with this, I can make them available outside of the core release.
  - I added five new properties to ant.properties (again, these can be
safely ignored) as referenced in the 'main.classpath' path element.

* I have numerous pyunit tests that I have not checked in yet.  I'm
deciding on the best way to handle it.  One problem with developing
across so many db's is testing them since you have to have running
instances.  For the time being I'll handle testing manually.

* I have not checked in any documentation.  I would like to re-style it
to fit into whatever flavor the rest of the Jython documentation uses.

* In installer/mklist.py, the 'javafiles' filter does not include the
*.dat nor the new *.properties files, should it?  On this note, I added
the new top-level 'com' package and its subdirectories as needed.  I did
not check in an update version of the generated files in that directory,
should I have?

* Please let me know if something doesn't compile.  I have only compiled
under 1.3 and 1.4, so I hope that it works on an earlier VM (is anyone
still using one?).

* I removed the license information from within all the files.  I kept
only a personal copyright.

* I removed the 'log' and 'pipe/xml' packages.  I didn't like the
logging implementation and the pipe/xml package required an additional
third-party package which I didn't want to include.

It should be noted this is a fresh checkin to CVS and not a merge of the
two repositories.  I actually found a bug while testing against a
Paradox database that is now in the Jython project and not in zxJDBC.

Also, please make sure to do a 'cvs update -d' to get the new
directories.

I'm really excited about finally merging zxJDBC into Jython.  Please
feel free to check out the code and make any comments, I'm interested in
feedback.

thanks,

brian

[Jython-dev] new system and environ support in javaos

[Samuele P. <ped...@bl...>]

Some notes:

- if something goes wrong with Runtime.exec a java exception is thrown,
 I think this should be somehow wrapped in an os.error. Opinions?
- in particular if something goes wrong when populating environ, we should
 decide whether to throw an exception or gracefully fall back to a NullEnv.
- environ populating does not work (as far as my installation is concerned)
with win 98:
  "command.com /c set should be invoked, "command /c set" fails.
  I think java version is also concerned here regarding to which extensions are
tried
  and which os calls are made. I used jdk 1.3.0.
  Anyone did try this already with different results?

regards, Samuele.

[Jython-dev] NameError: tracer. Giving up

[Ype K. <yk...@xs...>]

Finn,

Some weeks ago I reported the bug below. I tried to isolate it but
this proved to take too much of my time, given the workaround.


Since the workaround is not really straightforward it might be
worthwhile to somehow make it available until someone else runs into
it the bug and is able to isolate it.
The workaround is to use a method of an object as the trace function
instead of a function defined in module scope.

Is a bug report the right way to make the workaround available?

Regards,
Ype :(


>
>>
>> >
>> >
>> >def tracer(frame, why, arg):
>> >    print why, frame.f_lineno, arg
>> >    return tracer
>> >
>> >def foo():
>> >    for i in range(10):
>> >        a = "abc"
>> >    a = "Done"
>> >
>> >import sys
>> >sys.settrace(tracer)
>> >foo()
>>
>> I'll give it a try,
>>
>
>This works fine when called interactively.
>However when I use it via execfile() i get a:
>NameError: tracer
>on the "return tracer" statement.

It is not only via execfile(), a lot more is going in between,
ao. the dictionary against which the execfile() is executed
is the __dict__ of a new.module(). This is used within several
try/except and try/finally blocks on the stack frames of
several method invocations.

>
>The following works when called via execfile(),
>the interesting difference is probably the fact
>that the tracer function is now not in a module
>name space when returned for line tracing,
>so I think there is a bug related to the new nested
>namespaces in execfile():

I looked around in the jython source code for the nested scopes, but
could not find anything that might relate to this.

>
>
>class LineInfoCollector:
>    def __init__(self):
>       self.lineInfos = {}
>
>    def addLineInfo(self, lineInfo):
>        self.lineInfos[lineInfo] = 1
>
>    def getLineInfos(self):
>        infos = self.lineInfos.keys()
>        infos.sort()
>        return infos
>
>    def tracer(self, frame, why, arg):
>        self.addLineInfo((frame.f_code.co_filename,
>                          frame.f_code.co_name,
>                          frame.f_lineno,
>                          why, arg))
>        return self.tracer
>
>def foo():
>    for i in range(2):
>       a = "abc"
>    a = "Done"
>
>if __name__ == '__main__':
>    import sys
>
>    ltrac = LineInfoCollector()
>    sys.settrace(ltrac.tracer)
>    try: foo()
>    finally: sys.settrace(None)
>
>    print 'Lines covered:'
>    for lineInfo in map(str, ltrac.getLineInfos()):
>        print lineInfo
>

Re: [Jython-dev] jythonc "from blah import stuff, stuff" bug

[<bc...@wo...>]

[Chris Monson]

>First of all, I love Jython!  I have done some amazing things with it in 
>the two days of experience that I have :)
>
>I have an application working in Jython, and I am currently trying to 
>get the applet to work, as well.  I am using Jython 2.1a3 with JDK 
>1.3.1_01 from Sun.
>
>I did the following:
>
>jythonc --core --deep --jar myapplet.jar myapplet.py
>
>And it produced the appropriate jar file with all of the fixings, as 
>expected.
>
>However, when trying to invoke the applet with appletviewer (or in a 
>browser plugin), I got the following:
>
>    java.lang.NoClassDefFoundError: org/python/compiler/JavaMaker
>
>I finally isolated it to this line of python code:
>
>    from java.awt import Panel, Font, Button, Panel, Color, Label
>
>Notice that 'Panel' is imported twice in that list.  That was causing 
>the problem (but the error was not very helpful at all).  Once I removed 
>the spurious Panel import, I stopped getting that particular error.
>
>I am currently struggling with a different error, but the one above is 
>clearly (IMHO) a Jython bug, and I figured I should let you guys know.

I have added a bugreport about it, but I doubt I'll bother to fix it.
Jythonc does a static analysis of all the assignments going on in the
python script and this analysis does not handle re-assignments to the
same name. I agree that jythonc can be improved so it recognizes that
Panel is obviously bound to the same value.

regards,
finn

Re: [Jython-dev] jythonc breaks "import random"

[<bc...@wo...>]

>>[...] you can't use eval() or compile() in a restricted environment
>>such as applets.

[Chris Monson]

>I'll make the logical leap that says I can't do an 'exec' on uncompiled 
>code, as well.

Right.

>I was looking at that, and noticed that Jython is building java code 
>from the dynamic code in the eval, which makes sense from a performance 
>standpoint, but obviously breaks the applet security manager when it 
>tries to compile and dynamically load it.
>
>Is there any talk of allowing an option that would bundle the jython 
>interpreter when compiling with jythonc, 

The only 'interpreter' we have is the one that gets included when you
specify the --all option.

>so that eval's, exec's, and 
>compile's can be used in applets?  I believe that embedding the 
>interpreter would allow for that, though I can't be certain without 
>digging  into the source a bit more.

We would need a whole new subsystem that included a traditional
interpreter like CPython's ceval loop. And a whole new compiler that
creates whatever bytecode the interpreter is interpreting.

Yes, we have talked about it.

In Tools/jpythonc/PythonInterpreter.py is the beginning of an
interpreter that is running directly off the parse tree. That way we
don't need yet another compiler, but the result is likely slower than
python bytecode would be. PythonInterpreter is written by JimH and it is
some way away from being usefull.

regards,
finn

Re: [Jython-dev] jythonc breaks "import random"

[Chris M. <ch...@bo...>]

Finn Bock wrote:

>[Chris Monson]
>
>>My program works fine in jython, but I can't use jythonc to create an 
>>applet.
>>
>>I narrowed the problem down to this line of python code:
>>
>>   import random
>>
>>The stack trace given when I try to run the applet is as follows (it's 
>>longer than that, but I'm trying to keep the message short).
>>
>>Java Traceback:
>>
>>       at org.python.core.__builtin__.eval(__builtin__.java)
>>...
>>       at random$_PyInner._verify$1(random.java:377)
>>
>>   <snip>
>>
>>So, it appears that the _verify method in the random module is doing an 
>>eval!
>>
>
>Indeed. Rather thoughless bit of code there. I suggest you edit your
>copy of random.py to have an empty _verify() function:
>
>def _verify(name, expected):
>    pass
>
>I have posted a patch to CPython and will include a version of random.py
>in 2.1b1 that does not use eval().
>
Thanks.  That worked (but, you knew it would...).  I was not certain 
that I could safely change that function in my own module, so I 
appreciate the help.

>>So, I tried not importing random at all, but rather doing a simple eval:
>>
>>print eval("1 + 1")
>>
>>Again, I am required to use --all (which kind of makes sense), and 
>>again, I get an AccessControlException just like above.
>>
>>Does this mean that I can't use eval or any module that uses eval in my 
>>applets?
>>
>
>Correct, you can't use eval() or compile() in a restricted environment
>such as applets.
>
I'll make the logical leap that says I can't do an 'exec' on uncompiled 
code, as well.

I was looking at that, and noticed that Jython is building java code 
from the dynamic code in the eval, which makes sense from a performance 
standpoint, but obviously breaks the applet security manager when it 
tries to compile and dynamically load it.

Is there any talk of allowing an option that would bundle the jython 
interpreter when compiling with jythonc, so that eval's, exec's, and 
compile's can be used in applets?  I believe that embedding the 
interpreter would allow for that, though I can't be certain without 
digging  into the source a bit more.

Anyway, it's a thought.  I'll be following the list more closely now and 
really spending some time in the Jython source.  Jython is an incredibly 
cool project, and I am interested in contributing where I can.

Thanks again for your help.

C

Re: [Jython-dev] jythonc breaks "import random"

[<bc...@wo...>]

[Chris Monson]

>My program works fine in jython, but I can't use jythonc to create an 
>applet.
>
>I narrowed the problem down to this line of python code:
>
>    import random
>
>The stack trace given when I try to run the applet is as follows (it's 
>longer than that, but I'm trying to keep the message short).
>
> Java Traceback:
>
>        at org.python.core.__builtin__.eval(__builtin__.java)
>...
>        at random$_PyInner._verify$1(random.java:377)
>
>    <snip>
>
>So, it appears that the _verify method in the random module is doing an 
>eval!

Indeed. Rather thoughless bit of code there. I suggest you edit your
copy of random.py to have an empty _verify() function:

def _verify(name, expected):
    pass

I have posted a patch to CPython and will include a version of random.py
in 2.1b1 that does not use eval().

>That's all well and good, but I don't have org.python.core.parser 
>anywhere in my jar file.  Oops.
>
>It turns out that adding that file doesn't help.  If I use the --all 
>flag, I get this:
>Java Traceback:
>
>java.security.AccessControlException: access denied 

Using --all with applets is a topic for a different post.

>So, I tried not importing random at all, but rather doing a simple eval:
>
>print eval("1 + 1")
>
>Again, I am required to use --all (which kind of makes sense), and 
>again, I get an AccessControlException just like above.
>
>Does this mean that I can't use eval or any module that uses eval in my 
>applets?

Correct, you can't use eval() or compile() in a restricted environment
such as applets.

regards,
finn

[Jython-dev] Re: [Jython-users] PyDictionary and Java Hashtables

[<bc...@wo...>]

[Fred Sells on jython-users. I moved it to jython-dev because my reply
concern the development of jython]

>I'm using the python function call
>
>foo(**settings) to allow any options to be set in the call.
>
>foo then calls javabar(settings) where
>
>public void javabar(PyDictionary dictionary)

There is no garantee that the **setting object is a PyDictionary. It can
also be a PyStringMap or a PyInstance.

>I would like to convert dictionary to a Hashtable without stepping through
>all the elements.  Is there a way?

Not at the moment, but first I want to make sure I understand your
request correctly. From java you want to something like:

   PyObject o = ... // Assigned from some python code.
   Hashtable h = Py.py2hashtable(o);

If we decide to add a method like Py.py2hashtable(..), what should it
do? Here is my take (completely untested):

   public static Hashtable py2hashtable(PyObject o) {
        Hashtable htable = new Hashtable();
        Class Class = Object.class;

        PyObject keys = o.invoke("keys");

        int len = key.__len__();
        for (int i = 0; i < len; i++) {
            PyObject key = keys.__finditem__(i);
            PyObject value = o.__finditem__(key);
            htable.put(Py.tojava(key, oClass),
                       Py.tojava(value, oClass));
        }
        return htable;
    }


regards,
finn