jython-dev

[Jython-dev] jythons module os

[Dawurske,Jens <JDa...@oh...>]

Hi,

will the next jython-release (I'm using jython 2.0 on solaris) have a more
complete os-module - right now missing eg utime?

I did read FAQ 3.9. The Jython's os module is missing some functions, why?
(http://www.jython.org/cgi-bin/faqw.py?req=show&file=faq03.009.htp) and
downloaded the mentioned jnios.

Reading the doc jnios is more or less: tested for Win, experienced C++ and
so on. Doesn't made me really happy.

I would like to use a nice os-module on solaris. I also run for tests
Jython2.1a3 and Python2.1 on W2K

Thanks in advance for an answer or hints!

Jens

Jens Dawurske       mailto:JDa...@oh...
ohltec AG           www.ohltec.de
web campus Kiel
Kaistrasse 101      Tel.: 0431-7755 500
24114 Kiel          Fax:  0431-7755 555

Re: [Jython-dev] jythons module os

[Phil S. <phi...@ho...>]

I am planning to look into adding os.environment and os.system
support in, basically cribbing from how ant does this, but
I have not had time yet.

However, I think jnios is probably the better way to go for
you. Someone needs to finish it though. 

"Dawurske,Jens" wrote:
> 
> Hi,
> 
> will the next jython-release (I'm using jython 2.0 on solaris) have a more
> complete os-module - right now missing eg utime?
> 
> I did read FAQ 3.9. The Jython's os module is missing some functions, why?
> (http://www.jython.org/cgi-bin/faqw.py?req=show&file=faq03.009.htp) and
> downloaded the mentioned jnios.
> 
> Reading the doc jnios is more or less: tested for Win, experienced C++ and
> so on. Doesn't made me really happy.
> 
> I would like to use a nice os-module on solaris. I also run for tests
> Jython2.1a3 and Python2.1 on W2K
> 
> Thanks in advance for an answer or hints!
> 
> Jens
> 
> Jens Dawurske       mailto:JDa...@oh...
> ohltec AG           www.ohltec.de
> web campus Kiel
> Kaistrasse 101      Tel.: 0431-7755 500
> 24114 Kiel          Fax:  0431-7755 555
> 
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev

Re: [Jython-dev] jythons module os

[<bc...@wo...>]

[Phil Surette]

>I am planning to look into adding os.environment and os.system
>support in, basically cribbing from how ant does this, but
>I have not had time yet.

I hope you and/or someone else find some time to look into it, it would
be a valuable addition. I'm not sure what the right way to enable the
os.environment should be. It isn't right to fill the os.environment dict
whenever "os" is imported, that would be way to slow for all the program
that never uses any enviroment variables. Any thoughts?

>However, I think jnios is probably the better way to go for
>you. Someone needs to finish it though. 

Any volunteers?

regards,
finn

Re: [Jython-dev] jythons module os

[Kevin B. <kb...@ca...>]

Finn Bock wrote:
> 
> [Phil Surette]
> 
> >I am planning to look into adding os.environment and os.system
> >support in, basically cribbing from how ant does this, but
> >I have not had time yet.
> 
> I hope you and/or someone else find some time to look into it, it would
> be a valuable addition. I'm not sure what the right way to enable the
> os.environment should be. It isn't right to fill the os.environment dict
> whenever "os" is imported, that would be way to slow for all the program
> that never uses any enviroment variables. Any thoughts?

So I started playing with this, and didn't stop until I was basically functional.

Here's the code I came up with (it even has tests!)

I'm afraid I wrote in on NT, so testing on Unixes would be appreciated...

Comments/suggestions appreciated.

kb

Re: [Jython-dev] kevin's environ module

[Phil S. <phi...@ho...>]

Thanks Kevin! Great stuff.

I tested it out on my Win 2000 box. 
My linux box is unavailable right now, but I can test
it on that soon if no one else does.

I'm a version of the script with
the following modifications:

The os checks were limited to Windows NT, so I went over
to http://www.tolstoy.com/samizdat/sysprops.html
and got a list of the os.name strings there and
added them in along with some guesses.

Under the Win95 stream I believe the correct command
is 'command /c' so I poked that in, hope someone can
verify that this works. I'm relying on 
http://www.easydos.com/dosindex.html for this info.

Under win32, os.environ should be case-insensitive,
so I added that in as well (this follows
cpython's behaviour). Case-sensitivity should be
maintained for everything else (but this is untested).

I didn't really take an in-depth look at the module
other than to verify that it seems to work as
expected. 

Kevin Butler wrote:
> 
> Finn Bock wrote:
> >
> > [Phil Surette]
> >
> > >I am planning to look into adding os.environment and os.system
> > >support in, basically cribbing from how ant does this, but
> > >I have not had time yet.
> >
> > I hope you and/or someone else find some time to look into it, it would
> > be a valuable addition. I'm not sure what the right way to enable the
> > os.environment should be. It isn't right to fill the os.environment dict
> > whenever "os" is imported, that would be way to slow for all the program
> > that never uses any enviroment variables. Any thoughts?
> 
> So I started playing with this, and didn't stop until I was basically functional.
> 
> Here's the code I came up with (it even has tests!)
> 
> I'm afraid I wrote in on NT, so testing on Unixes would be appreciated...
> 
> Comments/suggestions appreciated.
> 
> kb
> 
>   ------------------------------------------------------------------------
> # os module enhancements for Jython
> # includes System command and lazily populated environ object
> # Kevin Butler <kev...@bi...>
> #
> # Features:
> # - transparently populates environment on first access
> # - transparently disables lazy evaluation for subsequent accesses
> # - passes changed environment to system command
> 
> #
> # Does not:
> # - ensure thread-safe initial population of environment
> # - read both stderror & stdin, probably in separate threads
> # - pass changed environment to child processes started with Runtime.exec (is this possible?)
> # - handle commands with '"' in them correctly on systems that use sh
> from java.lang import Runtime, System
> from java.io import BufferedReader, InputStreamReader
> from UserDict import UserDict
> 
> class ShellExec:
>     defaultTemplate = 'sh -c "%s"'
>     templates = {
>         "Windows NT": 'cmd /c %s',
>         }
> 
>     def __init__( self, template=None ):
>         """template is a string formatting template to format command to execute."""
>         if template:
>             self.template = template
>         else:
>             # determine the appropriate command template for the current OS
>             self.template = ShellExec.templates.get(
>                 System.getProperty( "os.name" ),
>                 ShellExec.defaultTemplate
>                 )
> 
>     def execute( self, cmd ):
>         """Execute cmd in a shell, and then return process"""
>         shellCmd = self.formatCmd( cmd )
>         if _environPopulated:
>             # already populated environment, so pass environment to subprocess
>             env = self.formatEnvironment( environ )
>             p = Runtime.getRuntime().exec( shellCmd, env )
>         else:
>             # use default environment
>             p = Runtime.getRuntime().exec( shellCmd )
>         return p
> 
>     def system( self, cmd ):
>         """Act like the standard library 'system' call.
>         Execute a command in a shell, and send output to stdout.
>         """
>         p = self.execute( cmd )
> 
>         # should read both stderror and stdout in separate threads...
>         stream = BufferedReader( InputStreamReader( p.getInputStream() ))
>         # we want immediate output, so don't call readLines
>         while 1:
>             line = stream.readLine()
>             if line is None: break
>             print line
> 
>         return p.waitFor()
> 
>     def readLines( self, stream ):
>         lines = []
>         while 1:
>             line = stream.readLine()
>             if line is None: break
>             lines.append( line )
>         return lines
> 
>     def formatCmd( self, cmd ):
>         """Format a command for execution in a shell.
>         """
>         return self.template % cmd
> 
>     def formatEnvironment( self, env ):
>         """Format enviroment in lines suitable for Runtime.exec"""
>         lines = []
>         for keyValue in env.items():
>             lines.append( "%s=%s" % keyValue )
>         return lines
> 
> _shellExec = ShellExec()
> 
> def system( cmd ):
>     return _shellExec.system( cmd )
> 
> def _populateEnviron():
>     """Populate this module's 'environ' variable.
>     This unwraps the 'envelope' pattern of the default lazy dictionary.
>     """
>     defaultEnvironCmd = "env"
>     environCmds = {
>         "Windows NT": "set"
>         }
>     # determine the appropriate command template for the current OS
>     environCmd = environCmds.get(
>         System.getProperty( "os.name" ),
>         defaultEnvironCmd
>         )
> 
>     p = _shellExec.execute( environCmd )
>     env = {}
>     stream = BufferedReader( InputStreamReader( p.getInputStream()))
>     for line in _shellExec.readLines( stream ):
>         i = line.index( '=' )
>         env[ line[:i]] = line[i+1:]
> 
>     # assign to os.environ, so future accesses incur no overhead
>     global environ, _environPopulated
>     environ = env
>     _environPopulated = 1
>     return env
> 
> class LazyDict( UserDict ):
>     """A lazy-populating User Dictionary.
>     Lazy initialization is not thread-safe.
>     """
>     def __init__( self, dict=None, populate=lambda: {} ):
>         """populate is a function that returns the populated dictionary"""
>         UserDict.__init__( self, dict )
>         self.initialized = 0
>         self.populate = populate
> 
>     def __populate( self ):
>         if not self.initialized:
>             self.initialized = 1 # not thread-safe!
>             # store as self.data so any 'set' sets in original as well
>             self.data = self.populate()
> 
>     ########## extend methods from UserDict by pre-populating
>     def __repr__(self):
>         self.__populate()
>         return UserDict.__repr__( self )
>     def __cmp__(self, dict):
>         self.__populate()
>         return UserDict.__cmp__( self, dict )
>     def __len__(self):
>         self.__populate()
>         return UserDict.__len__( self )
>     def __getitem__(self, key):
>         self.__populate()
>         return UserDict.__getitem__( self, key )
>     def __setitem__(self, key, item):
>         self.__populate()
>         UserDict.__setitem__( self, key, item )
>     def __delitem__(self, key):
>         self.__populate()
>         UserDict.__delitem__( self, key )
>     def clear(self):
>         self.__populate()
>         UserDict.clear( self )
>     def copy(self):
>         self.__populate()
>         return UserDict.copy( self )
>     def keys(self):
>         self.__populate()
>         return UserDict.keys( self )
>     def items(self):
>         self.__populate()
>         return UserDict.items( self )
>     def values(self):
>         self.__populate()
>         return UserDict.values( self )
>     def has_key(self, key):
>         self.__populate()
>         return UserDict.has_key( self, key )
>     def update(self, dict):
>         self.__populate()
>         UserDict.update( self, dict )
>     def get(self, key, failobj=None):
>         self.__populate()
>         return UserDict.get( self, key, failobj )
>     def setdefault(self, key, failobj=None):
>         self.__populate()
>         return UserDict.setdefault( self, key, failobj )
>     def popitem(self):
>         self.__populate()
>         return UserDict.popitem( self )
> 
> # initialize environ to the placeholder
> environ = LazyDict( populate=_populateEnviron )
> _environPopulated = 0
> 
> def putenv( key, value ):
>     environ[ key ] = value
> 
> def getenv( key ):
>     return environ[ key ]
> 
> def __test():
>     key, value = "testKey", "testValue"
>     org = environ
>     testCmds = [
>         "echo hello there",             # no quotes, should output both words
>         "echo PATH=%PATH%",             # should print PATH (on NT)
>         "echo %s=%%%s%%" % (key,key),   # should print 'testKey=%testKey%' on NT before initialization, and 'testKey=testValue' after
>         "echo PATH=$PATH",              # should print PATH (on Unix)
>         "echo %s=$%s" % (key,key),      # should print 'testKey=testValue' on Unix after initialization
> 
> # the following tests have double quotes in the commands. They work w/ NT's cmd shell, but not w/ sh. Escaping the quotes doesn't seem to help.
> #        'echo "hello there"',           # should output quotes on NT, no quotes on Unix.
> #        r'''python -c "import sys;sys.stdout.write( 'why\n' )"''', # should print 'why' to stdout.
> #        r'''python -c "import sys;sys.stderr.write( 'why\n' )"''', # should print 'why' to stderr, but it won't right now.  Should complete, though!  :-)
>         ]
> 
>     assert not _environPopulated, "before population, _environPopulated should be false"
> 
>     # test system - we should really grab the output of the system command, but we aren't
>     for cmd in testCmds:
>         print "\nExecuting %s with default environment" % cmd
>         assert not __shellExec.system( cmd ), "%s failed with default environment" % cmd
> 
>     # trigger initialization of environment
>     environ[ key ] = value
> 
>     assert _environPopulated, "after population, _environPopulated should be true"
>     assert org.get( key, None ) == value, "expected stub to have %s set" % key
>     assert environ.get( key, None ) == value, "expected real environment to have %s set" % key
> 
>     # test system using the non-default environment - should really grab output, but oh, well.
>     for cmd in testCmds:
>         print "\nExecuting %s with initialized environment" % cmd
>         assert not __shellExec.system( cmd ), "%s failed with initialized environment" % cmd
> 
>     assert environ.has_key( "PATH" ), "expected environment to have PATH attribute (this may not apply to all platforms!)"
> 
>

Re: [Jython-dev] kevin's environ module

[chuck c. <cc...@zi...>]

Kevin...
I've got a real interest in getting your os module to work so I began tonight
trying to get it to run on my unix boxes - one solaris and one linux.  I
started testing on the linux box first which is running mandrake 8.1,
jython2.1a3 and the IBM JDK 1.3. When I first ran the module there were errors
because you call a method on __shellExec when you define it as _shellExec.
Simple typo - but once fixed the tests fail for me with the message: 
AssertionError: echo hello there failed with default environment

I started looking into it a little deeper and it appears than any command with 
arguments fail.  If i add "ls", "date" and "uptime" to the beginning of the
testCmds list they work.  However when I add in "ls -l" I get the assertion.
It appears the shell returns with an exit code of 2.  sh on my machine is bash.
I tried it with csh and tcsh and the exit code is 1.  This is going to be some
quirky unix shell thing it looks like.  I'm experiementing now and reading man
pages.  Anyone else with more unix knowledge know what might be happening here?

chuck

On Thu, 01 Nov 2001 23:51:32 -0800
Phil Surette <phi...@ho...> wrote:

> Thanks Kevin! Great stuff.
> 
> I tested it out on my Win 2000 box. 
> My linux box is unavailable right now, but I can test
> it on that soon if no one else does.
> 
> I'm a version of the script with
> the following modifications:
> 
> The os checks were limited to Windows NT, so I went over
> to http://www.tolstoy.com/samizdat/sysprops.html
> and got a list of the os.name strings there and
> added them in along with some guesses.
> 
> Under the Win95 stream I believe the correct command
> is 'command /c' so I poked that in, hope someone can
> verify that this works. I'm relying on 
> http://www.easydos.com/dosindex.html for this info.
> 
> Under win32, os.environ should be case-insensitive,
> so I added that in as well (this follows
> cpython's behaviour). Case-sensitivity should be
> maintained for everything else (but this is untested).
> 
> I didn't really take an in-depth look at the module
> other than to verify that it seems to work as
> expected. 
> 

Re: [Jython-dev] kevin's environ module

[Phil S. <phi...@ho...>]

Looks like we are each doing some of the same work.
Hopefully we can merge our changes - I made a number
of them.

I haven't got a linux box to test with tonight; can
you try wrapping the arguments with quotes?

i.e. instead of '/bin/sh -c %s' use '/bin/sh -c "%s"'

chuck clark wrote:
> 
> Kevin...
> I've got a real interest in getting your os module to work so I began tonight
> trying to get it to run on my unix boxes - one solaris and one linux.  I
> started testing on the linux box first which is running mandrake 8.1,
> jython2.1a3 and the IBM JDK 1.3. When I first ran the module there were errors
> because you call a method on __shellExec when you define it as _shellExec.
> Simple typo - but once fixed the tests fail for me with the message:
> AssertionError: echo hello there failed with default environment
> 
> I started looking into it a little deeper and it appears than any command with
> arguments fail.  If i add "ls", "date" and "uptime" to the beginning of the
> testCmds list they work.  However when I add in "ls -l" I get the assertion.
> It appears the shell returns with an exit code of 2.  sh on my machine is bash.
> I tried it with csh and tcsh and the exit code is 1.  This is going to be some
> quirky unix shell thing it looks like.  I'm experiementing now and reading man
> pages.  Anyone else with more unix knowledge know what might be happening here?
> 
> chuck
> 
> On Thu, 01 Nov 2001 23:51:32 -0800
> Phil Surette <phi...@ho...> wrote:
> 
> > Thanks Kevin! Great stuff.
> >
> > I tested it out on my Win 2000 box.
> > My linux box is unavailable right now, but I can test
> > it on that soon if no one else does.
> >
> > I'm a version of the script with
> > the following modifications:
> >
> > The os checks were limited to Windows NT, so I went over
> > to http://www.tolstoy.com/samizdat/sysprops.html
> > and got a list of the os.name strings there and
> > added them in along with some guesses.
> >
> > Under the Win95 stream I believe the correct command
> > is 'command /c' so I poked that in, hope someone can
> > verify that this works. I'm relying on
> > http://www.easydos.com/dosindex.html for this info.
> >
> > Under win32, os.environ should be case-insensitive,
> > so I added that in as well (this follows
> > cpython's behaviour). Case-sensitivity should be
> > maintained for everything else (but this is untested).
> >
> > I didn't really take an in-depth look at the module
> > other than to verify that it seems to work as
> > expected.
> >
> 
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev

Re: [Jython-dev] kevin's environ module

[chuck c. <cc...@zi...>]

Hmm...Kevin actually had 'sh -c "%s"' in the original code
I switched to '%s' and everything worked.  And then I tried
'sh -c %s' and that worked.  I was curious and tried
"sh -c '%s'" but that failed as well.  So it definitely has something
to do with the quotes.  In the bash man page I found the following:

       Shell builtin commands return a status of 0 (true) if suc=AD
       cessful,  and  non-zero  (false)  if an error occurs while
       they execute.  All builtins return an exit status of 2  to
       indicate incorrect usage.

So I'm guessing maybe the shell is having issues quoting?  I get a
return code of 2 whether I use a builtin or call a command.

chuck

On Fri, 02 Nov 2001 01:40:26 -0800
Phil Surette <phi...@ho...> wrote:

> Looks like we are each doing some of the same work.
> Hopefully we can merge our changes - I made a number
> of them.
>=20
> I haven't got a linux box to test with tonight; can
> you try wrapping the arguments with quotes?
>=20
> i.e. instead of '/bin/sh -c %s' use '/bin/sh -c "%s"'
>=20

[Jython-dev] environ module, revision 2

[Kevin B. <kev...@bi...>]

OK, I'm getting back to environ now, integrating the feedback I received,
and testing on my Linux box.

From Chuck Clark:
> __shellExec should be _shellExec

*ouch* Sorry about that! That's what I get for making a change, then
running the tests w/o restarting the VM...

> 'sh -c "%s"'  should be 'sh -c %s'

The "%s" worked for me under cygwin, honest! The no-quotes acted like it
worked, but really ignored all the arguments to the command (ls -l == ls,
etc.).  Workaround is to use the exec( String[] ) form (can't use
Python's % operator *sigh*)

> Phil Surette wrote:

> The os checks were limited to Windows NT, so I went over
> to http://www.tolstoy.com/samizdat/sysprops.html
> and got a list of the os.name strings there and
> added them in along with some guesses.

Cool - thanks!

Looks like WinCE uses 'cmd' (google shows lotsa hits for WinCE and
cmd.exe together), so I'm classifying it as 'nt'.

>Under win32, os.environ should be case-insensitive,
> so I added that in as well (this follows
> cpython's behaviour). Case-sensitivity should be
> maintained for everything else (but this is untested).

Darn it, that ruins all hope of "unwrapping" the environment dictionary,
doesn't it?  *sigh* Guess we could... Nope, too ugly.  Functionality &
clarity trump optimization. Who ever said environment variables would be
fast?   :-)  :-(

Finn Bock wrote:
> A gracefull "do nothing" is needed for clasic macs.

Done.

> It must also be posible to
> override the os.name property in the registry file; something like
>
>   python.os=Windows

How about 'nt|dos|mac|unix'?

>Then we need to decide how to enable this. A registry entry like
>
>   python.environment=Shell

Sounds good - I'm not sure where that would hook in?

>should be the default and other values could be "None" which is what we
have
>today and maybe "Properties" which could fill the environment with just
$HOME
>and $USER and $PATH.

I didn't do anything along this line.

New code attached.

Note that we can call this whatever would be appropriate for integration
into jython (shellos?) - I'm not attached to 'environ.py' by any means.
:-)

Thanks

kb

Re: [Jython-dev] environ module, revision 2

[<bc...@wo...>]

[Kevin Butler]

>> It must also be posible to
>> override the os.name property in the registry file; something like
>>
>>   python.os=Windows
>
>How about 'nt|dos|mac|unix'?

Very good.

>>Then we need to decide how to enable this. A registry entry like
>>
>>   python.environment=Shell
>
>Sounds good - I'm not sure where that would hook in?

Untested, but something like:

_envType = sys.registry.getProperty("python.environment", "Shell")
if _envType == "Shell":
    _shellEnv = _getShellEnv()

    # support environ, putenv, getenv
    environ = _shellEnv.environment
    putenv = environ.__setitem__
    getenv = environ.__getitem__
elif _envType == "None":
    ...

>>should be the default and other values could be "None" which is what we
>have
>>today and maybe "Properties" which could fill the environment with just
>$HOME
>>and $USER and $PATH.
>
>I didn't do anything along this line.

That's fine, for now we just need a logical place to put it, if someone
wants to add it later.

>New code attached.
>
>Note that we can call this whatever would be appropriate for integration
>into jython (shellos?) - I'm not attached to 'environ.py' by any means.
>:-)

A copy & paste into javaos.py.

># Does not:
># - read both stderror & stdin, probably in separate threads

An absolute requirement for the system() call.

># - pass changed environment to child processes started with Runtime.exec (impossible?)

An old and out of date comment?

>        p = Runtime.getRuntime().exec( shellCmd, env, self._cwd )

The 3 argument version of exec() is a jdk1.3 feature. It is ok to try
and use such advanced features but there must a fallback to jdk1.1
behaviour.

I wonder if the chdir support is worth it at all. It is fine that
subshells inherit the parents (faked) notion of the $CWD but it is
confusing that files open from jython (both java and python files) does
not honor the $CWD.

>def _getOsType( os=None ):
>    os = os or registry.getProperty( "python.os" ) or registry.getProperty( "os.name" )

It is better to get the os.name from the System.getProperty() method. It
is common to initialize the registry with the System.getProperties() but
it is not certain that the application embedding jython does so.

regards,
finn

[Jython-dev] updated javaos.py

[Kevin B. <kev...@bi...>]

One update per week, pretty slow.  :-(

OK, I've incorporated environ.py into javaos.py (CVS 2.8).

I've attached the modified file and a registry w/ a descriptive comment

Since the last post, I made the following changes:
- enable based on registry settings
- remove chdir (made os.chdir throw OSError)
- added subthread to read stderror (one line of code - gosh I love Python *grin*)
- get os.name from System properties instead of registry

I also changed the javapath import to be "import javapath as path" because jython supports
that form now...

Feedback welcome

kb

Re: [Jython-dev] updated javaos.py

[<bc...@wo...>]

[Kevin]

>One update per week, pretty slow.  :-(

Don't worry about that. We have got by without it for almost 4 years
despite frequent request for both compatible environment and system().
It is so good to finally have both. Thanks.

>OK, I've incorporated environ.py into javaos.py (CVS 2.8).

>Feedback welcome

I did some minor reformatting of long lines to keep them below 79
characters and committed it in CVS.

regards,
finn

Re: [Jython-dev] updated javaos.py

[Kevin B. <kb...@ca...>]

Finn Bock wrote:
> 
> [Kevin]
> 
> >One update per week, pretty slow.  :-(
> 
> Don't worry about that. We have got by without it for almost 4 years
> despite frequent request for both compatible environment and system().
> It is so good to finally have both. Thanks.

My pleasure. :-)

But I just realized I should have started the stderr-reading thread 
before reading stdout, so that we don't see all of stdout before seeing 
any of stderr.

Sorry about that!  Hopefully no more changes to this for a long time.

Here's the patch:

cvs diff -c javaos.py
Index: javaos.py
===================================================================
RCS file: /cvsroot/jython/jython/Lib/javaos.py,v
retrieving revision 2.9
diff -c -r2.9 javaos.py
*** javaos.py	2001/11/14 16:25:20	2.9
--- javaos.py	2001/11/14 17:12:35
***************
*** 219,229 ****
          # _readLines( ... println )
          def println( arg, write=sys.stdout.write ):
              write( arg + "\n" )
-         # read stdin in main thread
-         self._readLines( p.getInputStream(), println )
          # read stderr in secondary thread
          thread.start_new_thread( self._readLines,
                                  ( p.getErrorStream(), println ))
          return p.waitFor()
  
  
--- 219,229 ----
          # _readLines( ... println )
          def println( arg, write=sys.stdout.write ):
              write( arg + "\n" )
          # read stderr in secondary thread
          thread.start_new_thread( self._readLines,
                                  ( p.getErrorStream(), println ))
+         # read stdin in main thread
+         self._readLines( p.getInputStream(), println )
          return p.waitFor()
  

kb

Re: [Jython-dev] updated javaos.py

[<bc...@wo...>]

[Kevin]

>But I just realized I should have started the stderr-reading thread 
>before reading stdout, so that we don't see all of stdout before seeing 
>any of stderr.
>
>Sorry about that! Hopefully no more changes to this for a long time.

Ohh, you are such an optimist.

Don't worry about changes; development is what we do here. I feel the
fundamentals is fine, the details can be straighten out over time and
with user feedback. If you are volunteering to deal with the feedback
and questions, I will gladly commit the patches.

>Here's the patch:

Has been committed.

regards,
finn

RE: [Jython-dev] updated javaos.py

[brian z. <bz...@zi...>]

Well I'm a pessimist and I have the first bug in javaos.py.

I'm on a Win2K box running Cygwin bash and when I execute `env` I get
the following (abbreviated for readability):

WINDIR=C:\WINNT
USERPROFILE=C:\Documents and Settings\brian zimmer
PS1=\[\033]0;\w\007
\033[32m\]\u@\h \[\033[33m\w\033[0m\]
$
ANT_HOME=//d/java/third/apache/jakarta-ant-1.3
PROGRAMFILES=C:\Program Files
USER=bzimmer

You'll notice the PS1 variable wraps to a new line and causes a
ValueError to be thrown:

Jython 2.1b1 on java1.3.0 (JIT: null)
Type "copyright", "credits" or "license" for more information.
>>> import os
>>> os.environ
Traceback (innermost last):
  File "<console>", line 1, in ?
  File "d:\home\development\sourceforge\jython\Lib\javaos.py", line 127,
in __repr__
  File "d:\home\development\sourceforge\jython\Lib\javaos.py", line 122,
in _LazyDict__populate
  File "d:\home\development\sourceforge\jython\Lib\javaos.py", line 273,
in _getEnvironment
ValueError: substring not found in string.index
>>>

For the moment I've just wrapped it in a try/except block but then the
value of the PS1 variable is truncated in the .environ dictionary.

thanks,

brian


> -----Original Message-----
> From: jyt...@li...
> [mailto:jyt...@li...] On Behalf Of Finn Bock
> Sent: Wednesday, November 14, 2001 12:49 PM
> To: jyt...@li...
> Subject: Re: [Jython-dev] updated javaos.py
> 
> 
> [Kevin]
> 
> >But I just realized I should have started the stderr-reading thread 
> >before reading stdout, so that we don't see all of stdout
> before seeing
> >any of stderr.
> >
> >Sorry about that! Hopefully no more changes to this for a long time.
> 
> Ohh, you are such an optimist.
> 
> Don't worry about changes; development is what we do here. I
> feel the fundamentals is fine, the details can be straighten 
> out over time and with user feedback. If you are volunteering 
> to deal with the feedback and questions, I will gladly commit 
> the patches.
> 
> >Here's the patch:
> 
> Has been committed.
> 
> regards,
> finn
> 
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/j> ython-dev
> 

Re: [Jython-dev] updated javaos.py

[Kevin B. <kev...@bi...>]

brian zimmer wrote:

> Well I'm a pessimist and I have the first bug in javaos.py.

:-)  No, no - my 'stderr shows up at the end of stdout' is first.  ;-)

> I'm on a Win2K box running Cygwin bash and when I execute `env` I get
> the following (abbreviated for readability):

> WINDIR=C:\WINNT
> USERPROFILE=C:\Documents and Settings\brian zimmer
> PS1=\[\033]0;\w\007
> \033[32m\]\u@\h \[\033[33m\w\033[0m\]
> $
> ANT_HOME=//d/java/third/apache/jakarta-ant-1.3
> PROGRAMFILES=C:\Program Files
> USER=bzimmer
>
> You'll notice the PS1 variable wraps to a new line and causes a
> ValueError to be thrown:

Ah, yes. I remember wondering about that case. Unfortunately, I don't know
that there's much we can do about it, except catch the ValueError and append
the value to the previous variable...

How about this?

    def _getEnvironment( self ):
        """Get the environment variables by spawning a subshell.
        This allows multi-line variables as long as subsequent lines do
        not have '=' signs.
        """
        p = self.execute( self.getEnv )
        env = {}
        key = 'firstLine' # in case first line had no '='
        for line in self._readLines( p.getInputStream() ):
            try:
                i = line.index( '=' )
                key = self._keyTransform(line[:i])
                value = line[i+1:]
            except ValueError:
                # found no '=', so this line is part of previous value
                value = '%s\n%s' % ( value, line )
            env[ key ] = value
        return env

If a subsequent line has an equals sign, this code will treat it as a new
variable assignment. We could get complex and define what characters are
valid in shell variable names, but there would still be cases where it
wouldn't work (e.g., X='My\nname=Kevin')

I think allowing some multi-line variables is useful, and handling the other
multi-liners w/o raising errors is helpful.

Thoughts?

kb

Re: [Jython-dev] updated javaos.py

[<bc...@wo...>]

[Kevin]

>How about this?
>...

Committed.

>If a subsequent line has an equals sign, this code will treat it as a new
>variable assignment. We could get complex and define what characters are
>valid in shell variable names, but there would still be cases where it
>wouldn't work (e.g., X='My\nname=Kevin')
>
>I think allowing some multi-line variables is useful, and handling the other
>multi-liners w/o raising errors is helpful.

Indeed. Still, we end up with a corrupt environment and when passed to
os.system() it may cause other errors. Such as when PATH is reset:

  X='My\nPATH=Kevin'

[In cygwin I had actually enter ctrl-M ctrl-J at the command prompt to
include a CRLF in the output of "cmd.exe /c set"]

So it seems like "cmd.exe /c set" is loosing the significant embedded
newline. Is this also the case for "sh -c env" on unixes? I suspect it
is.

It appears that the bash builtin "set" command writes its output in a
way that can be used for input. So while commands we execute now must
remain the portable default, we could have a mode that uses "bash -c
set" and convert the output back into real values:

  X=$'My\nPATH=Kevin'

It seems it would have to deal with the $'..' and backslash quoting.

It is not a requirement to fix this far-fetched problem before beta1.

regards,
finn

Re: [Jython-dev] updated javaos.py

[Kevin B. <kev...@bi...>]

Finn Bock wrote:

> [Kevin]
>
> [In cygwin I had actually enter ctrl-M ctrl-J at the command prompt to
> include a CRLF in the output of "cmd.exe /c set"]

What I've been doing is typing:

export X='asdf

zxcv'


> So it seems like "cmd.exe /c set" is loosing the significant embedded
> newline. Is this also the case for "sh -c env" on unixes? I suspect it
> is.

Yes - both "cmd.exe /c set" and "sh -c env" output the same, broken way.  :-(

> It appears that the bash builtin "set" command writes its output in a
> way that can be used for input. So while commands we execute now must
> remain the portable default, we could have a mode that uses "bash -c
> set" and convert the output back into real values:
>
>   X=$'My\nPATH=Kevin'
>
> It seems it would have to deal with the $'..' and backslash quoting.

On my cygwin system (bash 2.02.1(2)), bash -c set outputs:
X='asdf
zxcv'
(no the $ or backslash escapes), but my Linux system (bash 2.05.0(1)) outputs as
you described.

Easy enough to add a 'gnu' os type that does 'bash -c set', but the quoting
requires a rewrite of _getEnvironment (or a subclass of _ShellEnv) - probably
read the whole file, us re.split/findall to parse it out, and ??? to unescape the
escaped chars, and maybe something to handle the $/no$/escaped-chars.  Yuck.

>It is not a requirement to fix this far-fetched problem before beta1.

Yes - I don't think the niche of 'gnu users using multi-line environment
variables that have equal signs in lines after the first' is big enough to worry
about it at this point...  (Anyone who disagrees is welcome to implement it...
:-)  )

kb

Re: [Jython-dev] kevin's environ module

[<bc...@wo...>]

[Kevin & Phil]

>class ShellExec:
>    unixTemplate = 'sh -c "%s"'
>    ntTemplate = 'cmd /c %s'
>    dosTemplate = 'command /c %s' # from http://www.easydos.com/command.html
>    winceTemplate = dosTemplate # just guessing here
>    defaultTemplate = unixTemplate
>    # the keys for these templates taken from 
>    # http://www.tolstoy.com/samizdat/sysprops.html
>    templates = {
>        "Windows 95": dosTemplate, # does this catch 98, ME?
>        "Windows 98": dosTemplate, # guessing
>        "Windows ME": dosTemplate, # guessing
>        "Windows NT": ntTemplate,
>        "Windows NT 4.0": ntTemplate,
>        "WindowsNT": ntTemplate,
>        "Windows 2000": ntTemplate,
>        "Windows CE": winceTemplate, # I do wince when I contemplate CE
>        "Windows XP": ntTemplate # guessing
>        }

A gracefull "do nothing" is needed for clasic macs. It must also be posible to
override the os.name property in the registry file; something like

   python.os=Windows

and then there should some generic entries like

        "Windows": ntTemplate,
        "Unix": unixTemplate,
        "Mac": None,
 

Then we need to decide how to enable this. A registry entry like

   python.environment=Shell

should be the default and other values could be "None" which is what we have
today and maybe "Properties" which could fill the environment with just $HOME
and $USER and $PATH.

regards,
finn

Re: [Jython-dev] jythons module os

[<bc...@wo...>]

[Dawurske,Jens]

>will the next jython-release (I'm using jython 2.0 on solaris) have a more
>complete os-module 

At the moment the only change to the "os" module is fix to this bug:

>http://sourceforge.net/tracker/?group_id=12867&atid=112867&func=detail&aid=230021

>- right now missing eg utime?

We can at best to half a job in the utime() function. Pure Java2 only
allow changing the modification time of a file, not the access time.

Try to put this in your Lib/javaos.py:

def utime(path, times):
    if times and hasattr(File, "setLastModified"):
        File(path).setLastModified(long(times[1] * 1000.0))


regards,
finn

Re: [Jython-dev] kevin's environ module

[Phil S. <phi...@ho...>]

This reminds me of the bad 'ol days of shell scripting...
escaping special characters was always a black art.

chuck clark wrote:
> 
> Hmm...Kevin actually had 'sh -c "%s"' in the original code
> I switched to '%s' and everything worked.  And then I tried
> 'sh -c %s' and that worked.  I was curious and tried
> "sh -c '%s'" but that failed as well.  So it definitely has something
> to do with the quotes.  In the bash man page I found the following:
> 
>        Shell builtin commands return a status of 0 (true) if suc­
>        cessful,  and  non-zero  (false)  if an error occurs while
>        they execute.  All builtins return an exit status of 2  to
>        indicate incorrect usage.
> 
> So I'm guessing maybe the shell is having issues quoting?  I get a
> return code of 2 whether I use a builtin or call a command.
> 
> chuck
>

Re: [Jython-dev] kevin's environ module

[chuck c. <cc...@zi...>]

The good news is that with the same 'sh -c %s' all the tests passed on my
Solaris box too.  So I think we just need to have kevin update it and things
look to work reasonably well on unix.  I'll continue to play with it and report
back any other findings.

chuck

On Fri, 02 Nov 2001 02:04:27 -0800
Phil Surette <phi...@ho...> wrote:

> This reminds me of the bad 'ol days of shell scripting...
> escaping special characters was always a black art.
>