Help for migration MF/Adabas -> GC/Adabas

[Vita Compte]

Hello,

I'm discovering GnuCOBOL, and I'd like to migrate a project from MicroFocus/Adabas (database) to GnuCOBOL/Adabas on GNU/Linux.

I've already installed GnuCobol 3.1.2.0 and tried to compile some cbl successfully.

I now need some help to use the Adabas library and the precompiler with GnuCOBOL.

The cobpc precompiler produces a .cob, which I successfully compile with GnuCOBOL.

But, I'm stuck to uses the adabas functions

Namely :

cobcrun TESTCON
libcob: erreur: module 'sqbglaad' not found

sqbglaad is included in the provided cobmain.a

I'd like to reproduce the way I did with Microfocus :

1. make a custom rts with the library needed linked in - the MF command :
   cob -xge "" -o rtsada SQL.o /db/aad/v1501/lib/version.o /db/aad/v1501/lib/cobmain.a
   The SQL.cob is a cobol program which contains all calls to adabas functions (like sqbglaad) to generate static entry calls (if my words are correct)

2. uses precompiler cobpc and compile my programs without linking adabas libraries

3. launch programs with custom rtsada

Can you please help me ?

I guess I need to use cobcrun.o to generate my rtsada.

Thanks

[Mickey White]

Welcome Vita, Not a DB wiz, but here is some info that might relate https://db2twilight.blogspot.com/
https://sourceforge.net/p/gnucobol/contrib/HEAD/tree/trunk/samples/DBsample/
https://sourceforge.net/p/gnucobol/contrib/HEAD/tree/trunk/esql/
https://sourceforge.net/p/gnucobol/contrib/HEAD/tree/trunk/tools/JC-SQL-Precompiler/

[Simon Sobisch]

A you did not explicit wrote about which file you do have and a quick search did not show any docs from Adabas about that I'd try the following:

cobc -xg -o rtsada /build/dir/gnucobol-3.1.2/bin/.libs/cobcrun.o SQL.o /db/aad/v1501/lib/version.o /db/aad/v1501/lib/cobmain.a

If that doesn't work: Adabas seems to be licensed with support contracts only - in this case you may just want to check with its support.

[Vita Compte]

Thanks for your answer Mickey, my next step will be to migrate from Adabas to DB2. So it will be convenient when needed !

Thanks for your answer Simon, unfortunately Adabas support is terminated. But I still need to use it for some months.
I tried your command, and it generates the rtsada sucessfully.

But I still have the error :

./rtsada TESTCON
libcob: erreur: module 'sqbglaad' not found

Is it a problem with a function 'sqbglaad' provided in a shared object with an other name ? (cobmain.a)

If I do :

nm /db/aad/v1501/lib/cobmain.a | grep sqbglaad
0000000000000160 T sqbglaad

nm rtsada | grep sqbglaad
0000000000604ce8 b call_sqbglaad.4282

[Simon Sobisch]

It seems that there's an entry point of that name - so if you link it directly.... it may get optimized out by the linker.
Try telling the linker that you want to keep every symbol in the final executable, as you've mentioned GNU/Linux I guess you use gcc (or compatible) in this case you can do:

cobc -xg -o rtsada /build/dir/gnucobol-3.1.2/bin/.libs/cobcrun.o SQL.o /db/aad/v1501/lib/version.o /db/aad/v1501/lib/cobmain.a -v -Q "-Wl,-no-as-needed"
rtsada TESTCON

And check where this leads.
Another option would be to just use the normal RTS module runner and preload the symbols:

cobc -bg -o cobada.so /build/dir/gnucobol-3.1.2/bin/.libs/cobcrun.o SQL.o /db/aad/v1501/lib/version.o /db/aad/v1501/lib/cobmain.a -v -Q "-Wl,-no-as-needed"
COB_PRE_LOAD=cobada  cobcrun TESTCON

[Anonymous]

Hello Simon, and thanks for your reply.

I tried both solutions, but still not working.

I've attached an archive with DUMMY.cob (little program which just call 'sqbglaad') and the cobmain.a which contains the function.

The only way I made it work is :

• extract o files from cobmain.a. -> ar x cobmain.a
• cobc DUMMY.cob vpr40.o
• COB_PRE_LOAD="vpr40" cobcrun DUMMY

So it asks for another function
cobcrun: symbol lookup error: ./vpr40.so: undefined symbol: sql__ucmp
which shows that it founds the sqbglaad function

But this solution is awful as there is a lot of functions and a lot of .o files in the different .a of the adabas library.

With the file attached, would you be kind to have a look ?

[Simon Sobisch]

As the adabas integration is all closed-source I won't do that in my time working on the free GnuCOBOL. If you really want it you'd possibly have to look out for someone that helps remote - I'd do that but only on an half-hourly paid basis (half an hour is likely enough). If you need that then contact me via mail.

Concerning the "general linking" issue: cobmain.a seems to be a static library and version.o a necessary object. I'd have guessed that this may be a 32/64bit issue, but as partially unpacking did work this seems to not be the case.

According to this StackOverflow answer you possibly can create a dynamic link library out of the static library:

$ gcc -shared -o cobada.so   \
-Wl,--whole-archive          \
/db/aad/v1501/lib/cobmain.a  \
/db/aad/v1501/lib/version.o  \ 
-Wl,--no-whole-archive
$ ldd cobada.so

Note: you may or may not have to include some -lada functions if the ldd shows unresolved symbols.

If there are no unresolved symbols then you can do

COB_PRE_LOAD=cobada cobcrun -r  # ensure that cobada is both "set" and "resolved"  (= loaded)
COB_PRE_LOAD=cobada  cobcrun TESTCON

[Vita Compte]

Hello Simon,

Thanks, I achieved to launch my programs with your last suggestion !
Now it's time to test if everything works as expected !

Thanks again

[Simon Sobisch]

So "separate shared library (converted from the static library), pre-loaded", right?
Was there anything else needed to be linked against that library?

If possible please share the experience with this integration, we only had a single mention in the old discussion board but that was only "I did use it with success", not "how".

[Vita Compte]

Yes, I can share my experience :

First to be able to use Adabas :

1. I use this instruction to build a shared library, which contains everything needed :

   sh $ gcc -shared -o cobada.so \ -Wl,--whole-archive \ /db/aad/v1501/lib/cobmain.a \ /db/aad/v1501/lib/version.o \ -Wl,--no-whole-archive

   There are more .a to include, but the whole list depends of the Adabas version.
   I also added the other libraries I need like -lxml2 and -lssl.

2. ldd cobada.so - It's convenient to be sure a library is not forgotten.

3. COB_PRE_LOAD=cobada cobcrun TESTCON - runs perfectly my sample program test which just try to connect to the database.

Then, I tried to compile every program of the application.

---

In order to be able to compile them with Micro Focus or GnuCOBOL, I didn't modify them directly, but I use an own script which modifies the source code, then calls cobc.

What I had to do :

1. I have some hexadecimals written like this : X"0"
   GnuCOBOL needs two numbers

   "literal does not have an even number of digits

   sed -i 's/X"0"/X"00"/g' $1

2. I have some entries which starts with a underscore
   GnuCOBOL doesn't allow this.
   sed -i 's/_ENT_INIT/eENT_INIT/g' $1

3. I converted BINARY by COMP-5
sed -i 's/BINARY/COMP-5/g' $1

With this, the majority of programs compile.
I have now to look precisely if it remains some with errors.
I am at the beginning of the tests.
I also have a warning with arrays I need to investigate too.
Finally, when the application calls a EXIT PROGRAM, it seems that cobcrun doesn't stop.
I have to investigate this.

I'll update this post if I have to make other modifications.

But the most important is that the application starts and seems to work as expected.

[Simon Sobisch]

Thanks for the update. I've mod-edited it for some formatting and added some minor details.

For "What I had to do":

1. Is there a reason to not do this change directly to the original source? That non-standard odd digit should have never compiled, but I see at least a "reason" for it with a single-digit (will add that to e allowed with a warning when relaxed syntax checks are active.

2. Please provide an example that compiles with MF but does not with GC to highlight this issue. That sounds like something to be inspected.

3. Just add this as an alias -freserved=BINARY:COMP-5 - no need to adjust the code then.

4. EXIT PROGRAM - always only stopped the program if it was called, but not if it is the main program. Either do the "old style" EXIT PROGRAM. STOP RUN. in this case or the "newer" GOBACK. - both will ensure that the program is exited if called and to end the runtime unit if it was the "main" program. Both work with MF and GC identical so could be adjusted the same (you may want to do a global replace sed -i 's/EXIT\s+PROGRAM/GOBACK/g' in the original sources if this is what is expected (most COBOL programmers expect EXIT PROGRAM to leave the module - but as noted this is only true if the program was CALLed).

Out of curiosity: What are your compile commands with cob (MF) and cobc (GC)?

[Vita Compte]

Thanks Simon for your edit. I love to read people experiences in such cases, so it will be better for people who will read it.

1) *Is there a reason to not do this change directly to the original source? *
My application has so much cobol programs, it allows me to modify them at compile time, rather than modifying one by one.

2) Please provide an example that compiles with MF but does not with GC to highlight this issue.

      IDENTIFICATION DIVISION.
      PROGRAM-ID. "TESTENTRY".
      ENVIRONMENT DIVISION.
      CONFIGURATION SECTION.
      PROCEDURE DIVISION.
      DEBUT.
          ENTRY "_TPL_INIT".
          STOP RUN.

With Gc :
TESTENTRY.cob: dans le paragraphe « DEBUT » :
TESTENTRY.cob:7: erreur : ENTRY invalide « _TPL_INIT » - le nom ne peut pas débuter avec un espace ou un souligné

With MF : TESTENTRY.cob successfully compiled

3) Great ! I'll try this !

4) You are right, it was the main programs. So I'll try the

  sed -i 's/EXIT\s+PROGRAM/GOBACK/g'

5) MF : cob -Ww -P -u -C LINKCOUNT=2200 $1
GC : cobc -std=mf -fmfcomment $1

[Simon Sobisch]

My application has so much cobol programs, it allows me to modify them at compile time, rather than modifying one by one.

What about looping with find(direct execute by find) or grep (an actual for loop), then applying the totally clean changes with sed -i?

ENTRY "_TPL_INIT". hm...

Checking ACU Docs just reference that entry-names have to follow the program-name convention - but doesn't elaborate on this which leaves only user-defined words ("must contain at least one alphabetic character") and COBOL words in general (which specify: "Hyphens or underscores may not appear as the first or last character").

IBM docs are much more explicit:

Must be an alphanumeric literal that conform to the rules for the formation of a program-name in an outermost program.

and there:

The name can be up to 30 characters in length.
Only the hyphen, underscore, digits 0-9, and alphabetic characters are allowed in the name when it is specified as a user-defined word.
At least one character must be alphabetic.
The hyphen cannot be the first or last character.
If program-name is an alphanumeric literal [which is always the case for entry-point names], the rules for the name are the same except that the extension characters $, #, and @ can be included in the name of the outermost program and the underscore can be the first character.

MF doc is similar but lists different rules for different dialect options (mostly with "not enforced" comment) and longer names.

So - definitely something that should be adjusted in GnuCOBOL - I'm having a look at that.

[Simon Sobisch]

Both adjustments checked in with [r5044]. Using this version you can get rid of steps 1 (still would suggest to do this to update the code to be more standard-compliant and reduce warnings) and 2 (that earlier change of yours would likely break hart - every place where you CALL those entries).
3 can be replaced by command line option, 4 is best replaced in the original sources, too.

... so everything of this (actual unrelated to Adabas but to the MF migration) solved so far :-)

[Vita Compte]

Thanks Simon, and sorry for late reply.

For the moment, I test with GC 3.1.2.0
I'll test GC 3.2 soon

For debug, which solution do you recommend ?

[Simon Sobisch]

For debug, which solution do you recommend ?

If this is about source-level debugging: either GDB (needs cbl-gdb preprocessing; if one is unfamiliar with GDB it takes some time - but there are many "first step" docs available including the ones from the COBOLworx website and GDB is a tool in general that is good to know) or the Gix Debugger (if working on GUI tools is preferred - needs Gix-IDE setup).

[Vita Compte]

Thanks for your answer Simon.

I am used with the MF debugger in terminal. (cobanimsrv)
I imagine it will be difficult to compete with it.

I need to be able to debug only some programs.

Namely : prog1 calls prog2, which calls prog3.
I need to debug only prog3 for example.

I don't know if it is possible, but I'll try gix debugger asap.

[Simon Sobisch]

I imagine it will be difficult to compete with it.

Actually I had to use the animator several times - and find GDB with cbl-gdb from COBOLworx to be much more powerful and - after getting used to the differences faster to work with.
Note: you likely want to start with gdb --tui -ex "b prog3_:ENTRY_prog3" prog1 for your setup. If you want to stay in COBOL and use GDB then cbl-gdb is definitely a must.

[Anonymous]

Hello, some news about the migration.

1) Lot of programs work as expected, so I am confident for the future.

2) But, of course, I focus on those which do not work.
Here are some problems I solved "easily", although MF let them work.

  - Prog "A" calls Prog "B". 
    Prog B has a different program id than "B"
    MF executes B, GC says Entry point "B" not found.

  - Redefines with different level is allowed by MF, not by GC.

3) more difficult ones are " attempt to reference unallocated memory (signal SIGSEGV)"

Example I spent a lot of time to undesrstand :

Program A calls a function F which is an entry in a Program B.
Program A calls F with parameters : v1 -> a fixed structure (100 characters), the second one v2 was at different size (clearly a mistake, 100 characters, or 200, or 500, or 1000)
Program A also has a v3 (100 characters) defined.

Program B contains the entry with 2 fixed variables. : v1 -> 100 characters and v2 -> 500 characters.

Prog A calls F, so program B executes F, which did " move spaces to v2".
Return in Prog A, which makes an operation on V3 -> sigsesv.

I hope you will understand !

How can I find these type of problem in runtime ? ( linkage errors)

MF let them pass without errors.

I think I have to use "by reference" to deal with parameters which size can vary.

[Simon Sobisch]

1 :-)

2.1 - related is [bugs:884] - whenever this is implemented that will solve that issue - but fixing the name to match the module name (the CALLed one) is never wrong
2.2 -compile with -findirect-redefines would work around that, but clean code is always better

3 Did you compile with --debug? If not: do so and get the best diagnostic possible (but still the case you mention is not 100% implemented as a check).
I do find your text explanation confusing. Please write up a minimal demo program that shows the issue and it will be easy to understand.

Note: Overwriting data in LINKAGE that actually "isn't there" commonly won't do "no harm" with most compilers, but will overwrite "different" memory (for example MF has all WORKING STORAGE as if it would be in a single "level 0.5". Writing SPACESto an area bigger than passed will overwrite the data after it - if there is some COBOL area and you don't depend on its data, then you commonly get away with that; if there is not "enough" data [you pass a field at the end of the storage] then you get a chrash). I'd argue that a chrash is the best thing you can get - much better than overwriting some unrelated data (possibly numeric ones or important data).
Only data below level 01 is guaranteed to be consecutive storage.
With GnuCOBOL until 4.x (which has an option for consecutive data for all WS, a bit slower but matching different compilers) you'll overwrite other data, mostly depending on whatever the "C" storage layout on your environment is; the most common one is 01 fields "backwards" - a SIGEGV likely means that you don't have enough "sparse data area" before the level 01 you overwrite - and likely overwrite GnuCOBOL internal data (like the data pointers the field have - with the pointer being 0x2020202020 you'll have your SIGSEGV on the first use of that).
Things like that are easy to catch if --debug does the check for you, otherwise it can be found with minimal time in GDB using a watchpoint (but that needs some experience with GDB), possibly also when running under valgrind (slow execution).

[Vita Compte]

thanks for this so fast answer.
sorry if my explanation was confusing. I'll try to write a little demo reproducing it.

so your advice is to.compile programs with "cobc -d" ?

thanks

[Simon Sobisch]

My advice is: if something is strange add --debug to the compile command. And if it isn't important to get the minimal compute-performance time, leave it in.

[Vita Compte]

Thanks Simon, the --debug is very helpful, as it explains at what line it stops with errors.

I have more errors to review.

Especially errors like this, which are not triggered by MF, and not triggered without --debug :

       01 W-QTECAR.
               02 W-QTENUM                  PIC  Z(12)9V,999.

       77  W-QTECAR2                   PIC X(15).
       77  W-ZCAR                      PIC 9(2).       

---
                  MOVE 6                      TO W-ZCAR.  
                  MOVE W-QTECAR(W-ZCAR:15)    TO W-QTECAR2.

libcob: UNI.cobgc:1338: erreur: la longueur de << W-QTECAR >> est hors limites : 15, d?bute ? 13, maximum : 17

[Simon Sobisch]

Please run with LC_MESSAGES=C to get English error messages when posting to an international discussion board.
... checking in the message catalogs that was:

#: libcob/common.c:4005
#, c-format
msgid "length of '%s' out of bounds: %d, starting at: %d, maximum: %d"
msgstr "la longueur de « %s » est hors limites : %d, débute à %d, maximum : %d"

But the code does not match this issue, I guess the sample code is just wrong and W-ZCARis really 13.

If you run MF with all bounds checking, then you should get the same error. Note that you reference data outside of the 01 level; depending on the compiler (and possibly also version) this may either read into data fields before/after or automatically reduce the size to the maximum (if it does, then it needs to do a check on each reference-modification even without request, that would drop the performance).

If you just want to ignore this single check add -fno-ec=BOUND-REF-MOD to your cobc line, but "take with salt" as you likely will read/write data that don't belong there...

[Anonymous]

This seems odd the "V" indicating an assumed decimal point.
The "," is the decimal point.
Z(12),999 should be sufficient.
Albeit move data (13:15) to a 15 byte field will overlay data resident after the 15 bytes declaration.