jython-dev

RE: [Jython-dev] javaos.py for OS/400

[Humbel O. <Otm...@bi...>]

[  Kevin Butler ]

> > On the positive side, I can type
> >         >>> import os
> >         >>> os.system( 'env' )
> >         >>> os.system( 'java' )
> > and get the correct output.
>=20
> Wow - I certainly wasn't expecting that!  :-)
>=20
> I believe that is using 'sh' as its sub-shell.  Interesting -=20
> can you run 'sh' from the normal AS/400 shell?

Yes and no. From the native shell, I have to type STRQSH to get a
POSIX-like shell named Qshell
	http://www.as400.ibm.com/developer/qshell/
Once there, I can type:
	env
	sh -c env
	qsh -c env
all resulting in the same output.
Now, using the latest checked in version from javaos.py, I get (the
output of alias is not human readable):
   Jython 2.1a3 on java1.3.0 (JIT: NONE)
   Type "copyright", "credits" or "license" for more information.
   >>>
 $ import os
   >>>
 $ os.system( 'alias' )
   =A3  ~  =A7@`=C1%
   0
   >>>
I managed to translate the output correctly to=20
   >>>
 $ os.system( 'alias' )
   tar=3Dpax -A
   0
   >>>

=20
> Could you please send me the output of the following? (You=20
> may need javaos.py v2.12 from CVS - I did some renaming to=20
> make things more consistent with other modules)
>=20
> >>> import os
> >>> print os._shellEnv.getEnv, os._shellEnv.cmd,=20
> os._shellEnv._formatCmd( "echo hello world" )
>=20
> I expect it to print:
> env ['sh', '-c'] ['sh', '-c', 'echo hello world']

Here it is (your version):
 $ print os._shellEnv.getEnv, os._shellEnv.cmd, \
    os._shellEnv._formatCmd( "echo hello world" )
   env ['sh', '-c'] ['sh', '-c', 'echo hello world']
   >>>


> Some ideas (brainstorm mode):
> - a problem with quoting?  try os.system( '"java -version"' )
Ooops, now I remeber once quoting all but the command itself:
 $ os.system( 'java "-version"' )
nicely prints the output.... see your comments on echo below !

> - a problem with the encoding of the space or the hyphen? =20
> maybe try a "java helloWorld"?
> - a path problem (different java executables?)
> - from the shell, can you do (I don't know if the "" is right=20
> to keep the command in a single arg on AS/400...): =20
> 	sh -c "java -version"
Yes, without a problem.

< snipped suggestions about Runtime.exec() >=20

> > 4.
> >         >>> os.system( 'echo hello' )
> > gives no output at all - difficult to pass os.__test() this way :-(
>=20
> Yip, I added those tests when I discovered similar behavior...  :-)
>=20
> This is probably the same problem as #3 - the secondary=20
> arguments are not getting passed correctly to echo command,=20
> so echo is cheerfully echoing a blank line...
>=20
> When I encountered this behavior, I was working with the=20
> String form of Runtime.exec, executing strings like 'sh -c=20
> "my command"'  and the behavior was different on NT,=20
> NT-cygwin, and Linux - some of those platforms worked with=20
> the quoted command, others would fail with the quotes, but=20
> give the 'one command, no args' behavior you're seeing if=20
> they didn't have the quotes. =20
>=20
> Using the exec( String[] ) form works better, because you=20
> specify the argv array for the shell w/o worrying about=20
> quoting, so the shell gets exactly:  "-c", "java -version",=20
> ...  At least in theory...
I still haven't found a combination to get this echo.=20
So I will further investigate, using your suggestions.
And trying to solve the different encodings..

I'll attach javaos.py - but do not checkin yet, please :-) to give you
an idea what I am doing. The basic change is to use a subclass
_QShellEnv of _ShellEnv which handles all the os400 specifics.

Best wishes,
Oti.

RE: [Jython-dev] javaos.py for OS/400

[Humbel O. <Otm...@bi...>]

Hi Kevin, others

I slowly begin to understand why Finn did not want to write this one :-)

I can successfully issue quite complex commands like:
    os.system('alias')
    os.system('alias tar')
    os.system('echo this is my echo !')
    os.system('env')
    os.system('java')
    os.system('java -version')
    os.system('java -classpath /bison/neighborhood/iSeriesInfra/java/lib
CH.obj.Libraries.Utility.TestOutput', jvm=3D1)
    os.system('ls')
    os.system('ls -l')
    os.system('set')
    os.system('system -v ping rmtsys(alpsemsrv6)') # this issues the
native 'PING' command

However, there are some commands for which I didn't find a way to pass
hyphenated options to, like:
    os.system('alias -p')
- no idea why. Now I have to check each command if it accepts options
starting with '-', this will take some time ... . I see no solution at
the moment, I can only document these commands.

Please also notice the jvm=3D1 flag in the 'java -classpath' command:
os.system needs to be telled if a JVM is going to be started, because in
this case the encoding is different. In fact, IF a jvm is started, I can
use your (Kevin's) code unchanged :-)

Now my questions:=20
Is this 'extension' to os.system() - it will silently ignore the jvm
flag on most platforms - acceptable ?
Shall I proceed ?
If you tell me this has little chances to be checked into cvs, then I'll
give up my 'two-hours-a-day-research'.
I attach the current javaos.py FYI (It still needs to be worked over,
though).

Thanks for your feedback !
Oti.

PS. I think it would not be a big think to replace re, since we are only
using re.match(), won't it ?

Re: [Jython-dev] javaos.py for OS/400

[Kevin B. <kb...@ca...>]

Sorry for the slow reply - we've moved houses and had all sorts of
complications, the most minor of which is I no longer have a home 
internet connection (I'm going back to 56K RSN).  This has not 
been a productive couple of weeks.

Humbel Otmar wrote:
> 
> Hi Kevin, others
> 
> I slowly begin to understand why Finn did not want to write this one :-)

:-)

> I can successfully issue quite complex commands like:

cool!

> However, there are some commands for which I didn't find a way to pass
> hyphenated options to, like:
>     os.system('alias -p')
> - no idea why. Now I have to check each command if it accepts options
> starting with '-', this will take some time ... . I see no solution at
> the moment, I can only document these commands.

That is strange.  My guess is it is another character set issue, and that the EBCDIC (?) string argument passed to 'qsh -c "EBCDIC STRING -HERE"' has the wrong character.  Hmm.  Yeah, sounds crazy to me, too.

Maybe you could try echoing the commands, or maybe change the 'println' and 'printlnStdErr' functions to write( map( ord, arg ) + "\n" ), and see what the actual characters passed were?  


The following may also yield interesting information:

	Runtime.getRuntime().exec( ["qsh", "-c", "echo alias -p"] )
	Runtime.getRuntime().exec( ["qsh", "-c", "echo", "alias", "-p"] )
	Runtime.getRuntime().exec( ["echo", "alias", -p"] )

	system( "python printOrd.py -p" )  where printOrd.py does:  import sys;print map( ord, sys.argv[1] )

Also, the use of string.split & not passing to qsh is worrisome - it
means that trying to quote arguments and/or use environment variables
in commands would fail (unless we use expandvars & complicate
'split'):

	os.system( 'jvm LookupInfo "Kevin Butler" "$USERNAME"' )

will get mapped to:

        Runtime.exec([ 'jvm', 'LookupInfo', '"Kevin', 'Butler"', '"$USERNAME"' ])

How does it fail if it is executed as follows?

        Runtime.exec( ['qsh', '-c', 'jvm LookupInfo "Kevin Butler" "$USERNAME"']) 


> Please also notice the jvm=1 flag in the 'java -classpath' command:
> os.system needs to be telled if a JVM is going to be started, because in
> this case the encoding is different. In fact, IF a jvm is started, I can
> use your (Kevin's) code unchanged :-)

If we could make the 'jvm=' flag something more like an encoding
specification, I'd like it better. This may require figuring out
exactly what is causing the two cases to behave differently.

I did a little searching/reading:

http://www.as400.ibm.com/developer/java/devkit/rzaha116.htm#HDRRZAHA-JCERF

Sounds like reading from stdin and writing to stdout encode character
data using the AS/400 job CCSID (coded character set identifier),
while reading with an InputStreamReader and writing to an
OutputStreamWriter encode/decode using the file.encoding property.

The linked pages:

http://www.as400.ibm.com/developer/java/devkit/rzaha118.htm#HDRRZAHA-DFVRF
- shows many CCSIDs mapping to ISO 8859_1 as the default

http://www.as400.ibm.com/developer/java/devkit/rzaha117.htm#HDRRZAHA-FEVACRF
- shows ISO_8859_1 mapping to CCSID 819 as the closest map - but 819 isn't listed above (!)?

Also, it looks like the default CCSID is 37 - which looks like it
should be EBCDIC, and the default file.encoding for CCSID 37 is ISO
8859_1.  Hmm.

Looks like i18n.jar has a file encoding for international EBCDIC (Cp500),
and the AS/400 page indicates there is an encoding for IBM US EBCDIC (Cp037).

So how about this?

...

class _StreamInfo:
    """Holds the buffered stream and some other useful info.
    Handy to pass around as a parameter.
    """
    def __init__( self, stream, jvm=0 ):
        encoding = sys.registry.getProperty( "jython.encoding", System.getProperty( "file.encoding" ) )    
        self.setBufferedStream( BufferedReader(InputStreamReader(stream), encoding) )
        self.setJvm( 0 )

And then try the following:

sys.registry.setProperty( "jython.encoding", "Cp037" )
system( "ls -l" )
system( "java -Dfile.encoding=Cp037 HelloWorld" ) # requires a HelloWorld
system( "java -Dfile.encoding=Cp500 HelloWorld" ) # requires a HelloWorld

> Now my questions:
> Is this 'extension' to os.system() - it will silently ignore the jvm
> flag on most platforms - acceptable ?

If we can't make it generally useful, we may want to put most of it in a 'javaos400.py' or some such, but we can determine that as needed...

> Shall I proceed ?
> If you tell me this has little chances to be checked into cvs, then I'll
> give up my 'two-hours-a-day-research'.
> I attach the current javaos.py FYI (It still needs to be worked over,
> though).

I think it is valuable - and I'm learning things from your research, too!  :-)

> PS. I think it would not be a big think to replace re, since we are only
> using re.match(), won't it ?

No, that shouldn't be hard if we decide we need to, but it doesn't sound like there is a need at this point.

kb