Porting Regina REXX to System/370

Help
2009-03-02
2013-05-14
  • Robert O'Hara
    Robert O'Hara
    2009-03-02

    I am working to port Regina REXX to VM/370 release 6 on the Hercules platform.  I have successfully compiled Regina on VM/370 using the Gnu C compiler.  But when I use Regina to interpret a REXX program, it immediately complains that there is an invalid character in the program.  But that invalid character is the letter 'S' of a 'SAY' instruction.

    I suspect that deep in lexsrc.c or yaccsrc.c the characters read from the REXX program being interpreted are being compared to the ASCII representations.  Thus the code expects the letter S to be x'53' (ASCII), but gets x'E2' (EBCDIC).

    Do I need to re-generate lexxsrc.c and yaccsrc.c?  I can't find instructions for doing this.

    Any advice would be appreciated!

    Thanks,

    Bob O'Hara

     
    • Mike Protts
      Mike Protts
      2009-03-02

      It's likely to be a problem.  Could be various stray characters or problems with line end.

      The first thought I have is to read the file into memory and convert, then use RexxStart using the in memory version.  It's not ideal though.

      Maybe a better way to debug is to force tracing on, and write a little trace function (RXIOTRC?) that will show what is being interpreted.  Probably need to dump output in hex to be useful.

      Cheers
      Mike

       
    • Robert O'Hara
      Robert O'Hara
      2009-03-02

      Thanks, Mike.

      The problem seems to be deeper than that.  This REXX program:

      SAY "Hi"

      produces these messages:

      Error 13 running "hello.rex.a", line 1: Invalid character in program
      Error 13.1: Invalid character in program "('e2'X)"

      As I mentioned in my previous post, x'E2' is the letter S in EBCDIC.  So it seems clear we are dying in the parsing phase of the interpreter, somewhere in lexsrc.c and/or yaccsrc.

      Bob

       
    • Mike Protts
      Mike Protts
      2009-03-02

      It may be something as simple as a validation routine with a hard
      coded table of hex values for valid ascii - as the message is pretty
      specific.  Can you try running an ascii rexx program to see if it
      works, a binary transfer of a single line file should be OK, or use
      xedit in hex mode (was xedit available with that release of VM?).

      Otherwise you could try with the in storage option, but code the rexx
      in ascii hex, just as a diagnostic.

      I've never looked at lex/yacc code, so I'd be starting from scratch there.

      Cheers
      Mike

       
    • Mark Hessling
      Mark Hessling
      2009-03-02

      Hi Bob,

      Are you aware that Paul Edwards <mutazilah@gmail.com> is also attempting porting Regina to MVS on Hercules?  It might be worth discussing issues with him.

      As far as the EBCDIC problem, then it may have to do with lexsrc and yaccsrc.  To regenerate them you obviously need flex and yacc on your platform.  There are targets in the Makefile:
      % make yaccsrc lexsrc
      should create new yaccsrc.c and lexsrc.c and a new header file.

      BTW Are you using the source code from the SVN repository of SourceForge?  It's just I'm in the process of releasing Regina 3.5 and wouldn't want you to have to make major changes to new code again.

      Cheers, Mark

       
    • Robert O'Hara
      Robert O'Hara
      2009-03-02

      Mark,
      I'm in regular contact with Paul Edwards, he has not gotten as far as I have.

      I suspect I need to port flex and yacc to VM/370 before I can Regina to work.

      Unfortunately I did not check out the sources using SVN; I just downloaded the source distribution.  But so far my work has been to comment out code with ifdefs, I don't think it will take too much time to add it to 3.5.

      So for now I will take a look at flex and yacc.

      Thanks, Bob

      PS.  Mike, I tried running an ASCII version of a simple REXX program on VM/370, but Regina still chokes on the newline character (x'15) that C places at the end of the line, even though I put a CRLF (x'0d0a') ahead of it...

       
    • Robert O'Hara
      Robert O'Hara
      2009-03-02

      Well, just to be sure I created a REXX program:
         SAY "ABC"
      and coded it in ASCII (except for ABC, left in EBCDIC) on VM/370.  I also added a couple of lines of code to lexsrc.c to replace x'15' (EBCDIC newline) characters with ASCII blanks.

      And it runs!  So the good news is that there is hope that I can get Regina REXX to work on VM/370.  The bad news is that I have to fix this ASCII/EBCDIC problem.

      Bob

       
      • Mark Hessling
        Mark Hessling
        2009-03-02

        Hi Bob,

        Did a quick Google and found that the ASCII/EBCDIC problem with flex has been looked at before.  Don't know how useful the info below will be however :-)

        Cheers, Mark

        Return-Path: Mark_Ganter@liz.com
        Received: from 192.216.61.11 by horse.ee.lbl.gov for vern (5.65/1.43r)
            id AA02152; Tue, 11 Jan 94 06:19:36 -0800
        Received: from melonville.radiomail.net (mayberry.radiomail.net) by radiomail.net with SMTP id AA20220
          (5.65c+/IDA-1.4.4 for <vern@horse.ee.lbl.gov>); Tue, 11 Jan 1994 06:19:35 -0800
        Message-Id: <199401111419.AA20220@radiomail.net>
        Received: from liz.com by melonville.radiomail.net with CCGW-1.7(930217);
                  Tue, 11 Jan 94 06:19:17
        From: Mark_Ganter@liz.com
        Date: 11 Jan 94 09:05
        To: vern@horse.ee.lbl.gov
        Subject: Re[2]: Flex on an AS400

                      I have no problem at all with being a contact for advice
                  on porting Flex to EBCDIC.  The code was put on an AS400, so
                  filing the message under /MISC/AS400 is more appropriate.
                  The port was an interesting, very educational experience.
                  Thanks again.

                                                   Mark

        ------------------------------------------------------------------------
        Date: Tue, 18 Apr 1995 12:33:48 -0400
        To: "Michael W. Duffy" <mduffy@netcom.com>, Mark_Ganter@liz.com,
                vern@ee.lbl.gov (Vern Paxson), slayten@cas.org
        From: slayten@cas.org (Steve Layten)
        Subject: RE: Porting LEX scanner on EBCDIC machine
        X-Mailer: <Windows Eudora Version 2.0.2>
        Content-Length: 918

        I came in late on this discussion - I don't follow comp.compilers very
        closely these days because my job has shifted somewhat.

        I ported (quite some time ago) flex 2.3.6 to an IBM 3090 under MVS, using
        the SAS C compiler. The approach I used was, as Vern suggested, was to
        translate the tables to reflect EBCDIC-based indices. I haven't even
        downloaded flex 2.4 yet, so I don't know what's in the current distribution.
        My patches were in the flex 2.3 distribution for a while in the MISC
        directory. If you want any more info feel free to drop me a line - I still
        have (somewhere) the patches that I created from version 2.3.6.

        Steve Layten
        --
        Steven W. Layten, Senior Research Scientist
        Chemical Abstracts Service PO BOX 3012, Columbus, OH 43210  +1 614 447 3600
        INET: slayten@cas.org                                  FAX: +1 614 447 3813
        # # Speaking only for myself, and NOT for Chemical Abstracts Service! # # #

         
    • Robert O'Hara
      Robert O'Hara
      2009-03-03

      First of all, thanks to everyone for posting responses...

      I have made some progress.  I have flex and bison on VM/370, courtesy of Paul Edwards.  Now I do understand what these tools do (although my last compiler class was over 30 years ago!) but I have never used them in practice.

      The BISON program I have wants 3 files specified:  The input file (yaccsrc.y), the output file (yaccsrc.c), and a "defines" file.  I don't know what that is, and have found nothing that might be it in the regina source distribution.  Can anyone point me in the right direction?

      For FLEX, does it require only the input file, lexsrc.l?  Or are there other files I need?

      Thanks for helping out this noobie...

      Bob O'Hara

       
    • Mark Hessling
      Mark Hessling
      2009-03-03

      Hi Bob,

      How about yaccsrc.h as the other output file required by bison? :-)  Use the yaccsrc and lexsrc targets rules in Makefile.in as a guide.

      Cheers, Mark

       
    • Robert O'Hara
      Robert O'Hara
      2009-03-06

      Well, it turns out that BISON and FLEX are part of Paul Edwards' VM/380 system, which I use.  I ran yaccsrc.y through Bison v1.35, and lexsrc.l through Flex v1.38, and rebuilt Regina.  But something is now wrong with the grammar, because I get these results:

      For the REXX program
         SAY "Hi"
      Regina says

      Error 64 running "hello.rex.a": [Syntax error while parsing]
      Error 64.1: [Syntax error at line 1]

      For the REXX program
         SAY HI
      Regina says

      Error 36 running "a.rex.a", line 1: Unmatched "(" in expression

      For the REXX program
         X = 57
         RETURN X
      Regina says

      Error 64 running "b.rex.a": [Syntax error while parsing]
      Error 64.1: [Syntax error at line 1]

      For the REXX program
         RETURN 57
      Regina says

      Error 21 running "c.rex.a", line 1: Invalid data on end of clause
      Error 21.1: The clause ended at an unexpected token; found "57"

      So it would seem that things have gotten quite messed up...  I am not really sure how to debug this.  Any suggestions are welcome!

      Bob

       
    • Robert O'Hara
      Robert O'Hara
      2009-03-06

      Well, it turns out that BISON and FLEX are part of Paul Edwards' VM/380 system, which I use.  I ran yaccsrc.y through Bison v1.35, and lexsrc.l through Flex v1.38, and rebuilt Regina.  But something is now wrong with the grammar, because I get these results:

      For the REXX program
         SAY "Hi"
      Regina says

      Error 64 running "hello.rex.a": [Syntax error while parsing]
      Error 64.1: [Syntax error at line 1]

      For the REXX program
         SAY HI
      Regina says

      Error 36 running "a.rex.a", line 1: Unmatched "(" in expression

      For the REXX program
         X = 57
         RETURN X
      Regina says

      Error 64 running "b.rex.a": [Syntax error while parsing]
      Error 64.1: [Syntax error at line 1]

      For the REXX program
         RETURN 57
      Regina says

      Error 21 running "c.rex.a", line 1: Invalid data on end of clause
      Error 21.1: The clause ended at an unexpected token; found "57"

      So it would seem that things have gotten quite messed up...  I am not really sure how to debug this.  Any suggestions are welcome!

      Bob

       
    • Robert O'Hara
      Robert O'Hara
      2009-03-25

      Well, I have made a lot of progress in the past few weeks.  I now have a usable subset of Regina REXX running on VM/370 release 6 under Hercules.

      I've been unable to get BISON to work properly on CMS, but I was able to run FLEX, and by adding a small macro to YACCSRC.C the interpeter now handles EBCDIC input without problems.

      But of course there is always another question:

      Does Regina REXX free all of the memory it allocates?  Or does it assume that the operating system will take care of this for it?

      Thanks, Bob O'Hara

       
      • Mark Hessling
        Mark Hessling
        2009-03-26

        Hi Bob,

        Good work!. I should try and find my old VM minidiscs for Hercules and try it out.

        Regina should free its memory on exit.  I do know there is some memory that isn't freed (using Purify), but this should be a small amount.

        Cheers, Mark

         
    • Robert O'Hara
      Robert O'Hara
      2009-03-29

      Mark,
      Here is a trace of memory usage when executing a REXX procuedure with the single line "SAY "Hello..." on Windows:

      main

      malloc   520 at 3639e0
      free            3639e0
      malloc   520 at 3639e0
      malloc   520 at 363c28
      free            363c28
      free            3639e0
      malloc   520 at 3639e0
      malloc   520 at 363c28
      free            363c28
      free            3639e0
      malloc     4 at 365e60
      malloc 18512 at 365ea0
      malloc 32776 at 4e0068
      malloc    24 at 36a730
      malloc  2056 at 36a788
      malloc    24 at 36afd0
      malloc 32776 at 4e80b0
      malloc    24 at 362850
      malloc 32776 at 4f00f8
      malloc    24 at 3628a8
      malloc 32776 at 4f8140
      malloc    24 at 362900
      malloc 32776 at 500188
      malloc    24 at 362958
      malloc 32776 at 5081d0
      malloc    24 at 3629b0
      malloc 32776 at 510218
      malloc    24 at 362a08
      malloc 32776 at 518260
      malloc    24 at 362a60
      malloc 32776 at 5202a8
      malloc    24 at 362ab8
      malloc 32776 at 5282f0
      malloc    24 at 362b10
      malloc 32776 at 530338
      malloc    24 at 362b68
      malloc 32776 at 538380
      malloc    24 at 362bc0
      malloc 32776 at 5403c8
      malloc    24 at 362d80
      malloc 32776 at 548410
      malloc    24 at 362dd8

      setup_system

      malloc 32776 at 550458
      malloc    24 at 362e30
      malloc 32776 at 5584a0
      malloc    24 at 362e88

      execute_file

      malloc    40 at 362f90
      malloc 16386 at 5604e8
      malloc 32776 at 565568
      malloc    24 at 362ff8
      free            5604e8
      free            362f90

      As you can see, most of the memory allocated is never freed.  Am I missing something there?